An automatic calibration method for the eye-nose triangle

By automatically calibrating the eye-nose triangle area at five key feature points in the face area and performing secondary progressive registration, the problem of inefficiency and inaccurate manual selection of eye-nose triangle area in the prior art is solved, and the registration is improved Efficiency and accuracy are suitable for non-senior doctors and ordinary users, reducing the difficulty of medical imaging recognition.

CN115760934BActive Publication Date: 2024-11-26ARMY MEDICAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211275425.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-18
Publication Date
2024-11-26
Estimated Expiration
2042-10-18

AI Technical Summary

Technical Problem

When registering the clinical images of the patient's head with CVH, manual selection of the eye and nose triangle area is inefficient and inaccurate, resulting in insufficient automation processing level, which is especially inconvenient for non-senior medical workers and ordinary users. This may cause the problem of feature point mismatch.

Method used

The eye-nose triangle area is automatically calibrated by identifying five key feature points in the face area (left inner corner, left outer corner, right inner corner, right outer corner and human-center) and using the parallel queue competition recognition model for secondary progression Register, reduces the amount of calculation and improves accuracy.

Benefits of technology

It realizes automatic calibration of the triangle area of ​​the eye and nose, improves registration efficiency and accuracy, reduces user operation difficulty, is suitable for doctors and ordinary users in non-senior fields, and improves the universality of medical images recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115760934B_ABST
    Figure CN115760934B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of image registration, and in particular to an automatic calibration method for an eye-nose triangle, comprising: acquiring a head tomographic image; identifying a face region according to the head tomographic image; determining five key feature points of the face region; automatically calibrating the eye-nose triangle according to the five key feature points, and performing secondary progressive registration of the head tomographic image and a CVH head image according to the eye-nose triangle. The face region is identified according to the head tomographic image, and then the five key feature points of the face region are determined, and then the eye-nose triangle is automatically calibrated according to the five key feature points. In this way, manual selection of the eye-nose triangle is avoided, thereby improving the efficiency of secondary facial registration according to the eye-nose triangle, and improving intelligence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image registration, and in particular to an automatic calibration method for an eye-nose triangle area. Background Art

[0002] Clinical medical tomographic images such as CT and MRI are commonly used diagnostic and treatment methods in medicine. However, due to the abstract nature of the imaging method and the complexity of the human anatomical structure, clinical medical images often require the professional knowledge and experience of senior doctors, especially radiologists, to correctly read them. In particular, the complex clinical medical tomographic images of the head are too abstract and difficult to accurately identify for junior medical students, interdisciplinary researchers, medical engineers, and ordinary patients. Therefore, in the patent "An automatic positioning method and device based on clinical head images" previously applied by the applicant, the patient's head clinical image and CVH (Chinese Visible Human) are registered to obtain the CVH registered head anatomical structure, and then the patient's head clinical image is reconstructed in multiple planes to obtain the patient's head anatomical structure; there is a real-time mapping relationship between the patient's head anatomical structure and the CVH registered head anatomical structure; the region of interest is selected from the anatomical knowledge map, and the corresponding position of the region of interest in the patient's head anatomical structure is automatically located. Thereby, professional doctors in non-senior fields can also identify the various organ regions of the patient's head clinical image, which reduces the difficulty of identifying the patient's head clinical image and improves universality.

[0003] However, in the process of aligning the clinical head images of patients with CVH, the eye-nose triangle area is selected manually, which lowers the level of automated processing. For medical workers, although the feature area design based on 3D perspective is more user-friendly and intuitive than the traditional method of selecting each frame of the picture, if the steps of manually selecting the eye-nose triangle area need to be repeated for different data, the clinical operations of medical workers will be repeated and the scientific research efficiency will be low. For patients and their families, manually outlining the eye-nose triangle area is very inconvenient for some older users, resulting in a poor user experience.

[0004] Moreover, clinical applications have found that in the process of manually selecting the eye-nose triangle area, the subjective operations of some users vary greatly, which may cause a serious mismatch in the number of feature points between the patient's head tomography image and the CVH head tomography image, resulting in failure of spatial orientation. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes an automatic calibration method for the eye-nose triangle area, so as to automatically calibrate the eye-nose triangle area and improve the registration efficiency.

[0006] The technical solution adopted by the present invention is an automatic calibration method for the eye-nose triangle area.

[0007] In a first possible implementation, a method for automatically calibrating an eye-nose triangle includes: acquiring a head cross-sectional image; identifying a face region based on the head cross-sectional image; determining five key feature points of the face region; automatically calibrating the eye-nose triangle based on the five key feature points; and performing secondary progressive registration of the head cross-sectional image and a CVH head image based on the eye-nose triangle.

[0008] In combination with the first feasible method, in the second feasible method, the face area is identified based on the head tomographic image, including: performing volume rendering reconstruction based on the head tomographic image to obtain the three-dimensional structure of the head; performing projection mapping on the three-dimensional structure of the head to obtain a two-dimensional facial image; and inputting the contour of the two-dimensional facial image into a preset parallel queue competitive recognition model to obtain the face area.

[0009] In combination with the second feasible method, in the third feasible method, the parallel queue competition recognition model includes multiple algorithms, and the parallel queue competition recognition model is constructed in the following way: determining the weight of each algorithm for each type of image according to the robustness dimension, response efficiency dimension and accuracy dimension, establishing an automatic matching recommendation mechanism according to the weight of each algorithm for each type of image, recommending the optimal algorithm for the input image according to the automatic matching recommendation mechanism, and recognizing the input image according to the recommended algorithm to obtain the face area.

[0010] In combination with the third implementable method, in the fourth implementable method, the automatic matching recommendation mechanism automatically matches the optimal algorithm according to the resolution, modality, facial structure range and reconstruction method of the input image, and recommends the matched algorithm to the parallel queue competitive recognition model for face area recognition.

[0011] In combination with the second feasible method, in a fifth feasible method, the facial two-dimensional image contour is input into a preset parallel queue competitive recognition model to obtain a face area, including: when the parallel queue competitive recognition model fails to recognize, the three-dimensional structure of the head is rotated by a preset degree, re-projection mapping is performed, and face area recognition is performed again; when the parallel queue competitive recognition model recognizes a result, the result is used as the final recognized face area; when the parallel queue competitive recognition model recognizes two or three results, the optimal result is selected as the final recognized face area.

[0012] In combination with the second feasible method, in the sixth feasible method, the parallel queue competition recognition model identifies the face region in the following manner, including: preprocessing the facial two-dimensional image to obtain a corrected head tomographic image; obtaining a gradient amplitude map and a gradient direction map based on the corrected head tomographic image; obtaining facial feature information based on the gradient amplitude map and the gradient direction map; training the SVM classification model based on the facial feature information to obtain a target classifier; and classifying the facial two-dimensional image according to the target classifier to obtain the face region.

[0013] In combination with the first implementable method, in the seventh implementable method, the five key feature points include: left inner canthus, left outer canthus, right inner canthus, right outer canthus and philtrum.

[0014] In combination with the first feasible method, in the eighth feasible method, the eye-nose triangle area is automatically calibrated according to five key feature points, including: obtaining a minimum rectangle according to the five key feature points; expanding the minimum rectangle and moving the position of the minimum rectangle to obtain the eye-nose triangle area.

[0015] In combination with the second feasible manner, in a ninth feasible manner, the head sectional image and the CVH head image are registered according to the eye-nose triangle, including: obtaining the three-dimensional structure of the user's head according to the head sectional image; obtaining facial feature points according to the eye-nose triangle; pre-setting the facial feature points of the eye-nose triangle of the CVH; registering the three-dimensional structure of the user's head with the three-dimensional structure of the CVH head according to the facial feature points to obtain a CVH primary registered three-dimensional structure, the spatial direction of the user's head three-dimensional structure is consistent with that of the CVH primary registered three-dimensional structure; and performing a second registration on the user's head three-dimensional structure and the CVH primary registered three-dimensional structure.

[0016] In combination with the ninth possible implementation method, in the tenth possible implementation method, the user's head three-dimensional structure and the CVH primary registration three-dimensional structure are secondarily registered, including: obtaining a first annular contour of the user's head three-dimensional structure based on the eye-nose triangle; obtaining a second annular contour of the CVH primary registration three-dimensional structure; and performing registration based on the first annular contour and the second annular contour to obtain a CVH secondary registration head three-dimensional structure that is consistent with the spatial size of the user's head three-dimensional structure.

[0017] It can be seen from the above technical solution that the beneficial technical effects of the present invention are as follows:

[0018] 1. Identify the face area based on the head tomographic image, then determine the five key feature points of the face area, and then automatically calibrate the eye-nose triangle based on the five key feature points. In this way, there is no need to manually select the eye-nose triangle, which improves the accuracy of the eye-nose triangle selection, and also improves the efficiency of the secondary progressive registration based on the eye-nose triangle, and improves intelligence.

[0019] 2. The eye-nose triangle area is automatically calibrated according to the five key points to identify the front of a standard face. Compared with the existing technology that requires 68 facial feature points, this solution only needs the left inner corner of the eye, the left outer corner of the eye, the right inner corner of the eye, the right outer corner of the eye and the philtrum to identify the front of the face, which greatly reduces the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the prior art description. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0021] Figure 1 A schematic diagram of an automatic calibration method for the eye-nose triangle provided by the present invention;

[0022] Figure 2 A schematic diagram of the effect of different simulated lighting conditions on the gradient when the present invention does not adopt the present solution;

[0023] Figure 3 A schematic diagram of the effect of different simulated lighting conditions on the gradient after adopting the solution provided by the present invention;

[0024] Figure 4 A display diagram of the recognition effect of a natural face image provided by the present invention;

[0025] Figure 5 This is a typical representative recognition effect display diagram of a medical tomographic image provided by the present invention. DETAILED DESCRIPTION

[0026] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0027] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0028] Combination Figure 1 As shown, this embodiment provides an automatic calibration method for the eye-nose triangle area, including:

[0029] Step S01, obtaining a head cross-sectional image;

[0030] Step S02, identifying the face area according to the head tomographic image;

[0031] Step S03, determining five key feature points of the face area;

[0032] Step S04: Automatically calibrate the eye-nose triangle area based on the five key feature points.

[0033] Step S05: perform secondary progressive registration of the head section image and the CVH head image according to the eye-nose triangle.

[0034] In some embodiments, CVH (Chinese Visible Human) includes a Chinese Visible Human cross-sectional anatomical dataset and corresponding CT and MRI datasets, which include clinical images and anatomical structures of various parts of the body. CVH includes an anatomical knowledge graph, which includes the anatomical structure of each part, as well as the location identifiers of each organ region in the structure, organ names and other related knowledge.

[0035] The face area is identified based on the head tomographic image, and then the five key feature points of the face area are determined, and then the eye-nose triangle area is automatically calibrated based on the five key feature points. In this way, there is no need to manually select the eye-nose triangle area, which improves the accuracy of the eye-nose triangle area selection, and also improves the efficiency of facial registration based on the eye-nose triangle area, and improves intelligence.

[0036] In some embodiments, the head cross-sectional image is a clinical image of the patient's head acquired by CT (Computed Tomography) or MRI (Nuclear Magnetic Resonance Imaging).

[0037] Optionally, identifying a facial region based on a head tomographic image includes: performing volume rendering reconstruction based on the head tomographic image to obtain a three-dimensional head structure; performing projection mapping on the three-dimensional head structure to obtain a two-dimensional facial image; and inputting the contour of the two-dimensional facial image into a preset parallel queue competitive recognition model to obtain a facial region.

[0038] Optionally, volume rendering and reconstruction are performed based on the head tomographic image to obtain the three-dimensional structure of the head, including: using an RC algorithm to perform volume rendering based on the complete tomographic image data to obtain the three-dimensional structure of the head.

[0039] Optionally, obtaining the three-dimensional structure of the head includes: obtaining a facial contour by using a morphological-subtraction contour extraction method, and performing surface drawing according to the facial contour by using an MC algorithm to obtain a three-dimensional contour of the head.

[0040] In some embodiments, surface rendering is used when picking facial feature points and extracting facial contours, and volume rendering is used to obtain the three-dimensional structure of the head during facial recognition. Then, the feature points are identified, and the feature points found on the volume rendering are mapped to the facial contour drawn on the surface at the same perspective to obtain the facial feature points.

[0041] In some embodiments, the morphological-subtraction contour extraction method includes: reading DICOM (Digital Imaging and Communications in Medicine) raw data based on the C++ open source FO-DICOM library; converting the read image frame by frame into an 8-bit bitmap format; using the Ostu adaptive threshold segmentation algorithm for binarization processing; performing a closing operation on the processed image to fill small intracranial holes; using the FloodFill algorithm to fill the entire intracranial area; using a 3*3 corrosion template for morphological filtering, performing a subtraction operation on the data before and after corrosion to obtain the surface contour of a frame of data, saving the contour extraction result and processing the next frame of data until the batch processing is completed.

[0042] In some embodiments, the MC (Marching Cubes) algorithm is used to perform surface rendering based on the facial contour to obtain the three-dimensional contour of the head. The MC algorithm divides the three-dimensional volume data field constructed by the tomographic image into a number of voxels (cubes). The problem of extracting the isosurface of the three-dimensional data is decomposed into the problem of extracting the isosurface of each voxel. The MC algorithm processes the cubes in the data field one by one, finds the cubes that intersect with the isosurface, and uses linear interpolation to calculate the intersection of the isosurface and the edge of the cube. According to the relative position of each vertex of the cube and the isosurface, the intersection points on the isosurface and the edge of the cube are connected in a certain way to generate an isosurface, which is used as an approximate representation of the isosurface in the cube, and finally forms a three-dimensional contour structure, which serves as the morphological basis of the annular contour during the second registration.

[0043] In some embodiments, a RC (Ray Casting) algorithm is used to perform direct volume rendering based on the complete tomographic image data to obtain the three-dimensional structure of the head. The RC algorithm assumes that (at least) one ray (emitted in the direction of sight) is emitted from each pixel on the screen. This ray passes through the three-dimensional data field, and sampling points are set at equal distances between the intersection of the ray and the volume space. The data of each sampling point is calculated by trilinear or higher interpolation calculation. The color value and opacity of the sampling point are calculated according to the pre-set color and opacity and grayscale value corresponding gradient range. The color value and opacity value of each sampling point on each ray are synthesized from front to back or from back to front, so as to obtain the final image of the starting point position of the ray on the screen. By calculating each pixel according to the above process, a head image with a 3D stereoscopic effect can be obtained.

[0044] In some embodiments, since the three-dimensional structure of the head is obtained by volume rendering reconstruction in this application, and then the two-dimensional image of the face is obtained, such an image is not a natural human head tomographic image, but an image reconstructed by CVH or CT / MRI through volume rendering three-dimensional reconstruction. This leads to the fact that the accuracy of face recognition and non-face recognition in the prior art is unreliable when used in this application. Due to the multimodal characteristics of clinical images, and the influence of factors such as different resolutions, inter-layer intervals, and scanning site limitations in real clinical applications, face recognition of medical tomographic images is more difficult than natural images. Moreover, some images are local scanned images and may not even include the complete nose area, which will lead to the positioning and recognition of the face area. Therefore, the AdaBoost+Cascade cascade selection extraction method is adopted to form a strong classifier from weak classifiers of multiple face and non-face features, and the weight of the cascade classifier is determined according to the accuracy of the weak classifier, so as to construct a parallel queue competition recognition model to improve the accuracy and reliability of face recognition.

[0045] Optionally, the parallel queue competition recognition model includes multiple algorithms, and the parallel queue competition recognition model is constructed in the following manner: determining the weight of each algorithm for each type of image according to the robustness dimension, response efficiency dimension and accuracy dimension, establishing an automatic matching recommendation mechanism according to the weight of each algorithm for each type of image, recommending the optimal algorithm for the input image according to the automatic matching recommendation mechanism, and identifying the input image according to the matched algorithm to obtain the face area.

[0046] In some embodiments, the robustness dimension is to detect whether the atlas under the angle, resolution and mode can recognize the face and whether the required facial landmarks can be recognized according to the classification of different facial ranges and data modalities. The response efficiency dimension is to detect the response time of different algorithms under the same circumstances. The accuracy dimension is to return results through different algorithms, and users rank and score them according to their satisfaction with the positioning accuracy to obtain the accuracy of each algorithm. The recognition situation of each algorithm is analyzed through the robustness dimension, response efficiency dimension and accuracy dimension, and the optimal algorithm is selected, and the input image and the corresponding optimal algorithm are stored in the database to accumulate experience for the subsequent establishment of a self-matching recommendation database.

[0047] Optionally, the automatic matching recommendation mechanism automatically matches the optimal algorithm according to the resolution, modality, facial structure range and reconstruction method of the input image, and recommends the matched algorithm to the parallel queue competitive recognition model for face recognition.

[0048] In some embodiments, through the classic cases accumulated in the early stage, while the user inputs data, according to the resolution, modality, facial structure range and reconstruction method of the input image, the classic case with the highest similarity to the situation is searched in the preset case library. And the total score of the three dimensions under the case is automatically analyzed, and the top two algorithms in the total score are selected for accurate push. Once one of the algorithms successfully recognizes and detects a face, the other algorithm immediately stops running and returns the detection result to the user. Under the premise of ensuring the recognition accuracy, the recognition time is greatly reduced and the recognition efficiency is improved.

[0049] Optionally, the parallel queue competitive recognition model includes: Haar feature cascade, HOG+ linear SVM and Baidu AI. In some embodiments, after the facial two-dimensional image is input into the parallel queue competitive recognition model, the Haar feature cascade, HOG+ linear SVM and Baidu AI respectively perform competitive operations on the facial two-dimensional image to obtain each operation result, and analyze each operation result through the robustness dimension, response efficiency dimension and accuracy dimension to determine the best face area. At the same time, the weight of each algorithm for each type of image is determined according to the robustness dimension, response efficiency dimension and accuracy dimension, and an automatic matching recommendation mechanism is established according to the weight of each algorithm for each type of algorithm. According to the automatic matching recommendation mechanism, the optimal algorithm is recommended for the image input later, so as to ensure that all types of images can be accurately and quickly recognized. After a large number of experiments, it is found that the Haar algorithm is simple and fast, but the accuracy is not high, and it is difficult to meet the demand for automatic selection of feature point sets, but its high computing performance can be used to assist three-dimensional images to quickly correct the front face angle. The HOG+ linear SVM algorithm can be applied to face recognition and feature point calibration at almost all angles, but for partial scanned images that do not even include the complete nose area, it will cause the face area positioning to fail. Baidu AI platform is suitable for severely incomplete head tomographic images, but it cannot realize the "simplest case" full face image recognition. This solution combines and improves Haar feature cascade, HOG+ linear SVM and Baidu AI to form a parallel queue competition recognition model to realize the recognition of all types of images and ensure the recognition accuracy. During the test, 40 clinical CT / MRI head tomographic images were recognized, with a success rate of 97.5%.

[0050] Optionally, the facial two-dimensional image contour is input into a preset parallel queue competitive recognition model to obtain a face area, including: when the parallel queue competitive recognition model fails to recognize, rotating the three-dimensional structure of the head by a preset degree, re-projecting and mapping, and performing face area recognition again; when the parallel queue competitive recognition model recognizes one result, using the result as the final recognized face area; when the parallel queue competitive recognition model recognizes two or three results, selecting the best result as the final recognized face area.

[0051] In some embodiments, a two-dimensional facial image is input into a parallel queue competitive recognition model. In case of recognition failure, it is determined that the three-dimensional head structure does not fully display the standard face. The three-dimensional head structure is rotated by a preset degree, for example, 15 degrees, and re-projected to obtain a new two-dimensional facial image. The new two-dimensional facial image is input into the parallel queue competitive recognition model again for facial area recognition. The above recognition steps are repeated until the parallel queue competitive recognition model succeeds in recognition.

[0052] When the parallel queue competition recognition model successfully recognizes, the parallel queue competition recognition model outputs one, two or three results. When one result is output, the result is directly used as the final recognized face area; when two or three results are output, the best result is selected as the final recognized face area. In the face area recognition process of the parallel queue competition recognition model, experience is accumulated continuously. If a certain type of facial two-dimensional image has the highest recognition accuracy using a certain classifier, the weight of the classifier is increased so that the next time the same type of facial two-dimensional image is input into the parallel queue competition recognition model, the classifier is used for recognition first.

[0053] Optionally, the HOG+linear SVM in the parallel queue competition recognition model identifies the face region in the following manner, including: preprocessing the facial two-dimensional image to obtain a corrected head tomographic image; obtaining a gradient amplitude map and a gradient direction map based on the corrected head tomographic image; obtaining facial feature information based on the gradient amplitude map and the gradient direction map; training an SVM classification model based on the facial feature information to obtain a target classifier; and classifying the facial two-dimensional image according to the target classifier to obtain the face region.

[0054] Optionally, the facial two-dimensional image is preprocessed to obtain a corrected head tomographic image, including: grayscale processing the head tomographic image to obtain a grayscale head tomographic image; and adjusting the dynamic range of low grayscale areas and high grayscale areas in the grayscale head tomographic image to obtain a corrected head tomographic image.

[0055] In some embodiments, grayscale processing, i.e. ignoring color information, will have no effect on the tomographic image. Because medical images are grayscale images, the colors of common 3D reconstructions are artificially assigned pseudo colors, which do not contain any information of the original data, and grayscale processing does not change the information entropy of the image.

[0056] Optionally, the dynamic range of the low grayscale area and the high grayscale area in the grayscale head tomographic image is adjusted to obtain a corrected head tomographic image, including: performing gamma correction on the grayscale head tomographic image. In some embodiments, gamma correction is performed by the following formula f(x)=x ^γ; Wherein, x on the right side of the equation is the grayscale value of the original image pixel, r is the nonlinear coefficient of gamma correction, the smaller r is, the brighter the transformed image is, when r is 1, there is no change before and after the transformation, and f(x) on the left side of the equation is the grayscale value of the corresponding pixel after the original image grayscale value x is transformed.

[0057] By adjusting the value of the gamma correction nonlinear coefficient in the formula, the corresponding pixels before and after the image transformation are nonlinearized, thereby adjusting the dynamic range of the low gray value area and the high gray value area to achieve brightness compensation suitable for human visual characteristics and weaken the impact of contrast light sources (including local shadows or local overexposure) on image quality.

[0058] Optionally, a gradient amplitude map and a gradient direction map are obtained based on the corrected head tomographic image, including: using a preset horizontal gradient formula to perform a traversal operation on all pixel points in the corrected head tomographic image to obtain a horizontal gradient map; using a preset vertical gradient formula to perform a traversal operation on all pixel points in the corrected head tomographic image to obtain a vertical gradient map; synthesizing a gradient vector based on the horizontal gradient map and the vertical gradient map; and obtaining a gradient amplitude map and a gradient direction map based on the gradient vector.

[0059] Optionally, the horizontal gradient formula is Among them, Gx represents the first-order difference in the horizontal direction, and f(x, y) represents the grayscale value of the pixel at the (x, y) coordinate in the image.

[0060] Optionally, the vertical gradient formula is Wherein, Gy represents the first-order difference in the vertical direction.

[0061] The horizontal gradient is the gradient amplitude, which is used to characterize the value of the degree of grayscale change between adjacent pixels when there is an edge in the image. The vertical gradient is the gradient direction, which is used to characterize the direction in which the pixels around the pixel point change fastest.

[0062] Optionally, the first-order difference in the horizontal direction and the first-order difference in the vertical direction of each pixel are synthesized by vector addition to obtain the gradient vector corresponding to each pixel. Since the image is a two-dimensional array, Gx and Gy are regarded as mutually perpendicular vectors in the coordinate system, and the gradient vector of the point is synthesized by the idea of ​​vector addition. The horizontal gradient mainly highlights the horizontal edge features, and the vertical gradient mainly emphasizes the vertical edge features. The horizontal gradient and the vertical gradient are combined to obtain the gradient vector, which retains the contour edge information to the greatest extent, avoids information loss, and extracts useful edge gradient information, ignoring useless background and color information, thereby improving the recognition accuracy.

[0063] Optionally, obtaining a gradient magnitude map and a gradient direction map according to a gradient vector includes: obtaining a gradient magnitude and a gradient direction of each pixel point according to a modulus length formula and a phase angle formula of vector addition to form a gradient magnitude map and a gradient direction map.

[0064] Alternatively, the modulus length formula is Among them, mag(G) represents the gradient amplitude of the point.

[0065] Alternatively, the phase angle formula is Among them, θ represents the gradient direction angle of the pixel point, and the value range of the direction angle θ is [0°, 180°].

[0066] Optionally, facial feature information is obtained according to the gradient magnitude map and the gradient direction map, including: dividing the gradient magnitude map and the gradient direction map into a plurality of 8X8 sub-images; simplifying each sub-image into a multi-dimensional vector; performing sliding window trimming on each sub-image to obtain a plurality of blocks; and normalizing the multi-dimensional vectors in each block to obtain facial feature information.

[0067] Optionally, sub-graphs of the gradient magnitude map and the gradient direction map are simplified into multi-dimensional vectors, including: dividing the angle of the sub-graph of the gradient direction map into a plurality of interval angles, placing the gradient magnitudes corresponding to the pixel points that meet the interval angles into the corresponding intervals, and dividing the gradient magnitudes corresponding to the pixel points that do not meet the interval angles into the adjacent interval angles in proportion. In this way, the feature matrices of the plurality of sub-graphs are simplified into multi-dimensional vectors, which greatly simplifies the features and reduces the amount of computation.

[0068] In some embodiments, the interval angles are determined to be 20 0 , 40 0 , 60 0 , 80 0 , 100 0 , 120 0 , 140 0 , 160 0 , 180 0 , the gradient amplitude corresponding to the pixel points that meet each interval angle is placed in the interval corresponding to each interval angle, and the gradient amplitude corresponding to the pixel points that do not meet each interval angle is placed in the interval corresponding to the closest interval angle. After traversing all the pixel points once, a 9-dimensional vector can be obtained. Optionally, each sub-image is trimmed by sliding window to obtain several blocks, including: the facial two-dimensional image is evenly divided into several sub-images, and the sliding window is slid in the facial two-dimensional image, and each sliding forms a block, and each block includes several sub-images.

[0069] In some embodiments, each sub-image is a 9-dimensional vector, and the four sub-images are concatenated into a 36-dimensional vector. (where i∈[1,8]) performs average normalization on the sliding windows of 8 units, where X1-X8 represent the 36-dimensional vectors of the 1st to 8th sliding windows respectively, Xi old Represents the 36-dimensional vector of the original i-th sliding window, Xi new The 36-dimensional vector representing the new i-th sliding window is the facial feature information.

[0070] In some embodiments, the gradient of an image is sensitive to light. Often, the gradient amplitude changes due to changes in the overall light intensity of the image's surroundings, causing the feature extraction map of the same image to change due to different environments. If this feature is used as the input of a subsequent classifier, it will put great pressure on the robustness of the classifier algorithm. If the overall light source of the image is relatively strong, although the grayscale value is still proportional, the gradient changes greatly due to the increase in the overall value. Figure 2 As shown, the left half of the image simulates normal lighting conditions, the grayscale values ​​of two adjacent pixels are [3, 4], and the first-order difference of this point is 1; the right half of the image simulates overexposure, then the grayscale values ​​of these adjacent pixels become [6, 8], and the first-order difference of this point is 2. By contrast, it is found that the gradient changes due to changes in lighting. In this application, the facial two-dimensional image is divided into several sub-images, and a sliding window is used to slide to form multiple blocks, so that each block includes several sub-images, and each block is normalized to obtain facial feature information. Figure 3 As shown in the figure, for two adjacent pixel points of simulated normal light source and simulated strong light source, this scheme is used to calculate the geometric mean, and then normalize, and the first-order difference under the two light sources is 2. In this way, the gradient value does not change with the change of the overall illumination intensity of the surrounding, which can not only ensure that there is no lack of information, but also make the gradient extraction value of the same picture under different illumination unchanged, which has high robustness and reduces the influence of illumination on the results in the later SVM recognition detection.

[0071] Optionally, a linear SVM is used as a classification recognition algorithm, and facial feature information is extracted through a large number of positive samples (face regions) and negative samples (non-face regions), and the SVM classification model is trained based on the facial feature information. The parameters of the SVM model are continuously corrected in the application to achieve the best classification effect, and finally the face region is obtained.

[0072] Optionally, determining five key feature points of the face region includes: using a gradient boosting decision tree algorithm to determine the five key feature points of the face region.

[0073] In some embodiments, the gradient boosting decision tree algorithm adopts the fundamental idea of ​​regression tree, forms a GBDT tree set through multiple regression trees with residuals as leaf nodes, and forms a basic classification model through multiple GBDT cascades. The model is trained by manually labeled images, and the model parameters are changed to make the calibration as close as possible to the calibration "gold standard". The gradient boosting decision tree algorithm refers to the prior art.

[0074] In some embodiments, Figure 4 This is a demonstration of the recognition effect of natural face images. The real face image is identified through the parallel queue competition recognition model to identify the face area, and then the five key feature points of the face area are determined, and finally the following is obtained: Figure 4 The 5-point feature point marking diagram shown. Figure 5 This is a typical medical tomographic image recognition effect display diagram. Through volume rendering reconstruction, a two-dimensional face image is obtained after screenshot. The face area is identified through the parallel queue competition recognition model, and then the five key feature points of the face area are determined. Finally, the following is obtained: Figure 5 The 5-point feature point marking diagram shown. This application can recognize both real face images and face images obtained through body rendering through a parallel queue competition recognition model, and accurately identify feature points to further ensure the accuracy of registration, and can also be used in multiple fields such as face recognition.

[0075] Optionally, the five key feature points include: left inner eye corner, left outer eye corner, right inner eye corner, right outer eye corner and philtrum. The eye-nose triangle area is automatically calibrated according to the five key points, and the front of the standard face can be identified. Compared with the prior art that requires 68 facial feature points, the present application only requires the left inner eye corner, left outer eye corner, right inner eye corner, right outer eye corner and philtrum, which greatly reduces the amount of calculation.

[0076] Optionally, the eye-nose triangle area is automatically calibrated according to the five key feature points, including: obtaining a minimum rectangle according to the five key feature points; expanding the minimum rectangle and moving the position of the minimum rectangle to obtain the eye-nose triangle area.

[0077] Optionally, obtaining a minimum rectangle based on five key feature points includes: obtaining the maximum and minimum values ​​of the horizontal coordinate, and the maximum and minimum values ​​of the vertical coordinate among the five key feature points; combining the maximum and minimum values ​​of the horizontal coordinate with the maximum and minimum values ​​of the vertical coordinate, respectively, to obtain four coordinate points; and connecting the four coordinate points to obtain the minimum rectangle.

[0078] In some embodiments, the maximum horizontal coordinate Xmax, the minimum horizontal coordinate Xmin, the maximum vertical coordinate Ymax, and the minimum vertical coordinate Ymin are combined to obtain coordinate points (Xmax, Ymax), (Xmax, Ymin), (Xmin, Ymax), and (Xmin, Ymin).

[0079] Optionally, the minimum rectangle is expanded and the position of the minimum rectangle is moved to obtain the eye-nose triangle area, including: increasing the width of the minimum rectangle by a first preset percentage, shifting the minimum matrix to the left by a second preset percentage, and moving the minimum matrix up by a third preset percentage to obtain the eye-nose triangle area. In some embodiments, the width of the minimum rectangle is increased by 20% of the width, the minimum matrix is ​​shifted to the left by 10%, and the minimum matrix is ​​shifted up by 10% to obtain the eye-nose triangle area.

[0080] Optionally, the method further includes: performing spatial orientation registration of the patient's head three-dimensional structure and the head three-dimensional structure of the CVH according to the eye-nose triangle area, so as to obtain a head registration having a spatial orientation consistent with the head three-dimensional structure of the CVH.

[0081] Optionally, the head sectional image and the CVH head image are registered according to the eye-nose triangle, including: obtaining a three-dimensional structure of the user's head according to the head sectional image; obtaining facial feature points according to the eye-nose triangle; pre-setting facial feature points of the eye-nose triangle of the CVH; registering the three-dimensional structure of the user's head with the three-dimensional structure of the CVH head according to the facial feature points to obtain a CVH primary registered three-dimensional structure, the spatial directions of the user's head three-dimensional structure and the CVH primary registered three-dimensional structure being consistent; and performing a second registration on the user's head three-dimensional structure and the CVH primary registered three-dimensional structure.

[0082] Optionally, acquiring facial feature points according to the eye-nose triangle area includes: sampling the eye-nose triangle area with a preset step length to obtain a plurality of facial feature points.

[0083] In some embodiments, the user's facial feature points are obtained after sampling the eye-nose triangle area of ​​the user's facial two-dimensional image calibration with a preset step length. The CVH facial feature points are obtained after sampling the eye-nose triangle area of ​​the CVH facial two-dimensional image calibration with a preset step length. The user's facial feature points and the CVH facial feature points are both feature point cloud matrices with 3 rows and 7 columns. The spatial direction is aligned based on the two to obtain the CVH initial alignment three-dimensional structure. In this way, the problem of serious mismatch in the number of feature points in the same area due to inaccurate positioning of the feature area or large differences in spatial resolution of different modal data is solved, and the row and column feature point acquisition spacing in the eye-nose triangle area is adaptively adjusted, so that the eye-nose triangle area acquisition of various image data is always a 3*7 matrix, thereby improving the stability of the subsequent alignment navigation of the secondary progressive alignment algorithm.

[0084] Optionally, the user's head three-dimensional structure is secondarily registered with the CVH primary registration three-dimensional structure, including: obtaining a first annular contour of the user's head three-dimensional structure based on the eye-nose triangle; obtaining a second annular contour of the CVH primary registration three-dimensional structure; and performing registration based on the first annular contour and the second annular contour to obtain a CVH secondary registration head three-dimensional structure that is consistent with the spatial size of the user's head three-dimensional structure.

[0085] Optionally, the three-dimensional structure is cut from 2D to 3D normal vectors according to the eye-nose triangle to obtain ring features.

[0086] Optionally, there is a mapping relationship between each region in the CVH secondary registration head three-dimensional structure and each region in the user's head three-dimensional structure; each region in the CVH secondary registration head three-dimensional structure has a label, and the label is the name of each region; the region of interest is selected from the label and automatically located to the corresponding position in the user's head three-dimensional structure through the mapping relationship.

[0087] The CVH secondary registration head 3D structure is obtained through the secondary decremental registration method, and the automatic positioning of each facial area is achieved based on the real-time mapping relationship between the CVH secondary registration head 3D structure and the user's head 3D structure. In this way, non-experienced professional doctors can also identify the various organ areas of the patient's head clinical image, which reduces the difficulty of medical image recognition, is conducive to interdisciplinary professionals, young doctors, medical students, patients and their families to understand the lesions in medical images, lowers the threshold of medical imaging, and improves universality.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An automatic calibration method for the eye-nose triangle, characterized in that: include: Obtain head cross-sectional images; recognizing a face region according to the head tomographic image; Determine five key feature points of the face area, the five key feature points including: left inner corner of the eye, left outer corner of the eye, right inner corner of the eye, right outer corner of the eye and philtrum; Automatically calibrating the eye-nose triangle area according to the five key feature points, wherein the automatic calibration of the eye-nose triangle area according to the five key feature points includes: obtaining a minimum rectangle according to the five key feature points; expanding the minimum rectangle and moving the position of the minimum rectangle to obtain the eye-nose triangle area; Performing secondary progressive registration of the head tomographic image and the CVH head image according to the eye-nose triangle; The head tomographic image recognition of the face area includes: Perform volume rendering reconstruction based on the head tomographic image to obtain a three-dimensional structure of the head; Performing projection mapping on the three-dimensional structure of the head to obtain a two-dimensional facial image; Inputting the facial two-dimensional image contour into a preset parallel queue competition recognition model to obtain a face region; The parallel queue competition identification model includes multiple algorithms, including: Haar feature cascade, HOG+linear SVM; The parallel queue competition recognition model determines the best face area by performing competition calculations through the multiple algorithms; The parallel queue competition identification model is constructed in the following way: The weights of the algorithms for each type of image are determined according to the robustness dimension, the response efficiency dimension and the accuracy dimension, an automatic matching recommendation mechanism is established according to the weights of the algorithms for each type of image, an optimal algorithm is recommended for the input image according to the automatic matching recommendation mechanism, and the input image is recognized according to the recommended algorithm to obtain a face area.

2. The method according to claim 1, characterized in that The automatic matching recommendation mechanism automatically matches the optimal algorithm according to the resolution, modality, facial structure range and reconstruction method of the input image, and recommends the matched algorithm to the parallel queue competitive recognition model for face area recognition.

3. The method according to claim 1, characterized in that Inputting the facial two-dimensional image contour into a preset parallel queue competition recognition model to obtain a face region, including: In the case where the parallel queue competition recognition model fails to recognize, the three-dimensional head structure is rotated by a preset degree, projection mapping is performed again, and face area recognition is performed again; When the parallel queue competition recognition model recognizes a result, the result is used as the final recognized face region; When the parallel queue competition recognition model recognizes two or three results, the best result is selected as the final recognized face area.

4. The method according to claim 1, characterized in that: The HOG+ linear SVM in the parallel queue competition recognition model recognizes the face area in the following ways, including: Preprocessing the facial two-dimensional image to obtain a corrected head tomographic image; Acquire a gradient amplitude map and a gradient direction map according to the corrected head tomographic image; Obtain facial feature information according to the gradient magnitude map and the gradient direction map; Training the SVM classification model according to the facial feature information to obtain a target classifier; The facial two-dimensional image is classified according to a target classifier to obtain a human face region.

5. The method according to claim 1, characterized in that The head tomographic image and the CVH head image are registered according to the eye-nose triangle, including: Acquire the three-dimensional structure of the user's head according to the head tomographic image; Acquire facial feature points according to the eye-nose triangle area; Pre-set facial feature points of the eye-nose triangle area of ​​CVH; Aligning the user's head three-dimensional structure and the CVH head three-dimensional structure according to the facial feature points to obtain a CVH primary alignment three-dimensional structure, wherein the user's head three-dimensional structure and the CVH primary alignment three-dimensional structure have the same spatial direction; A second registration is performed on the user head three-dimensional structure and the CVH primary registration three-dimensional structure.

6. The method according to claim 5, characterized in that Performing a second registration on the user head three-dimensional structure and the CVH primary registration three-dimensional structure, including: Acquire a first annular contour of the three-dimensional structure of the user's head according to the eye-nose triangle area; Acquire a second annular contour of the primary registration three-dimensional structure of the eye-nose triangle area preset by CVH; Alignment is performed according to the first annular contour and the second annular contour to obtain a CVH secondary registration head three-dimensional structure that is consistent with the spatial size of the user's head three-dimensional structure.