Methods and systems for measuring exophthalmos

Abstract

The exophthalmos measuring method and system of the present application simplify the complexity of computer vision processing by first converting a three-dimensional face file into a two-dimensional map. Then the eye region is extracted to remove the redundant part of the picture and improve the accuracy. Then the eye contour is extracted, the pupil is detected after the orbit is detected, and the coordinates of the temporal orbital edge and the pupil edge on the two-dimensional map are extracted. Then map back to the three-dimensional photo, matrix the three-dimensional spherical coordinate equation, and use the least square method to fit the spherical center coordinates and the eyeball radius length, so as to calculate the exophthalmos distance. The present application uses convenient and non-contact three-dimensional face images as input, and realizes the automatic and accurate measurement of exophthalmos by means of artificial intelligence, computer vision and mathematical calculation based on three-dimensional coordinates.

Description

Technical Field

[0001] This invention relates to the field of ocular protrusion measurement, and in particular to a method and system for measuring ocular protrusion. Background Technology

[0002] Proptosis refers to the relative position of the eyeball within the bony orbit, determined by the relationship between orbital volume and orbital morphology, and influenced by the tightness of the orbital septum and intraorbital vascular pressure. Various orbital diseases, such as acute and chronic orbital inflammation, tumors, and carotid-cavernous fistula, can cause proptosis, with thyroid-associated ophthalmopathy (TAO) being the most common. Orbital fractures, orbital varicose veins, or orbital fat atrophy can lead to enophthalmos. Proptosis measurement is an essential basic clinical examination for diagnosing orbital diseases and evaluating treatment effectiveness. When a patient presents with proptosis, intraorbital lesions should be suspected, such as inflammatory changes like TAO or intraorbital space-occupying lesions; early screening for these conditions is crucial.

[0003] Eye exotropia refers to the vertical distance from the temporal orbital rim to the corneal apex. Clinically, the most commonly used methods are the standard ruler method and the Hertel exotropia method. The standard ruler method involves placing the zero mark of a transparent or special ruler on the temporal orbital rim, and the measurer observes from the side, noting the distance at the corneal apex. This method is simple but less accurate and often heavily reliant on the doctor's experience. Because eye exotropia is measured by human observation, accurately locating the highest point of the eyeball is difficult. Furthermore, the angle and placement of the ruler can significantly affect the measurement. The Hertel exotropia method uses an isosceles right-angled triangular optical prism to project both the corneal apex and the scale onto the prism through reflection from the inclined plane, allowing for direct observation by the human eye. However, due to the thickness of the soft tissue in the orbit, the position cannot be guaranteed to be perfectly aligned during manual positioning, leading to slight deviations that further amplify measurement errors. Furthermore, this method, which is also based on human visual observation, has similar problems to the ordinary ruler measurement method in determining the corneal apex, thus introducing a large error.

[0004] Currently, there are four main categories of methods for measuring proptosis: exophthalmometry, traditional computed tomography (CT) measurement, 3D CT measurement, and other methods. Among these, there are four types of exophthalmometers: Hertel, Naugle, Oculus, and Mourts. Some studies have shown that these versions may have higher accuracy than the Hertel exophthalmometer, but these have not been widely validated and used, and they cannot completely solve the inherent non-reproducibility problem of manual measurement. Traditional computed tomography (CT) measurement is mainly divided into two methods: direct measurement and indirect measurement. Direct measurement involves reconstructing the coronal, sagittal, and oblique planes based on CT scans, and selecting the distances between several points on these planes to represent the degree of proptosis. Indirect measurement, on the other hand, represents proptosis by calculating area ratios, using complex geometric calculations to quantify the axial degree of proptosis in TAO patients. The D-CT reconstruction method uses CT data to perform three-dimensional reconstruction for measurement. There are also many other attempts, such as using optical three-dimensional imaging, digital photographs of the front and sides, slit lamp measurement, and calculations using Heron's formula and simple Pythagorean theorem.

[0005] Each of the above methods for measuring eye protrusion has its own advantages and limitations. Eye protrusion meters are simple to operate but have large measurement errors and individual differences exist between measurers; CT measurements are more accurate than eye protrusion meters but are expensive, and patients are exposed to radiation damage, making them unsuitable as a standard method requiring repeated measurements throughout the course of the disease; both eye protrusion meters and CT scans mostly use the line connecting the outer margins of both orbits or the line connecting the inner and outer margins of the same orbit as a reference line, evaluating the degree of eye protrusion by measuring the distance from the corneal apex to this reference line. When the subject's head is tilted or the eyes are deviated, the two eyes are not on the same plane in the CT scan, leading to large deviations in the test results, and the detection method is complex; measurements using three-dimensional reconstruction technology have high accuracy and can be used in some special cases such as head misalignment or strabismus, but the measurement method is complex and time-consuming.

[0006] In today's era of rapid development in artificial intelligence technology, we are inspired to explore new methods for measuring ocular protrusion that are easy to operate, highly accurate, and widely applicable. Currently, applications of artificial intelligence in this field are limited to low-noise, relatively easy-to-analyze images such as CT / MRI scans, without directly analyzing human faces. Other patents also focus their innovative perspectives on improvements to the Hertel device. Summary of the Invention

[0007] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a method and system for measuring eyeball protrusion, so as to solve the above-mentioned technical problems in the prior art.

[0008] To achieve the above and other related objectives, the present invention provides a method for measuring eyeball protrusion. The method includes: converting a three-dimensional face file to be measured into a corresponding two-dimensional texture file; identifying the eye region based on the two-dimensional texture file to obtain eye region identification data; segmenting the eye contour based on the eye region identification data to obtain eye contour segmentation data; extracting the pupil region based on the eye contour segmentation data to obtain pupil region extraction data; obtaining two-dimensional UV coordinate data based on the eye contour segmentation data and the pupil region extraction data; wherein the two-dimensional UV coordinate data includes: two-dimensional coordinate data of the temporal orbital margin and two-dimensional coordinate data of the pupil margin; mapping the two-dimensional UV coordinate data to three-dimensional spatial coordinates to obtain corresponding three-dimensional coordinate data; and calculating the three-dimensional center coordinates and eyeball radius length based on the three-dimensional coordinate data to obtain eyeball protrusion.

[0009] In one embodiment of the present invention, the step of identifying the eye region based on the two-dimensional texture file to obtain eye region identification data includes: identifying the rectangular eye region in the two-dimensional texture file based on the YOLO network to obtain eye region identification data.

[0010] In one embodiment of the present invention, the step of segmenting the human eye contour based on the eye region recognition data to obtain human eye contour segmentation data includes: using the Sobel operator to segment the human eye contour on the eye region recognition data to obtain human eye contour segmentation data.

[0011] In one embodiment of the present invention, the step of extracting the pupil region based on the human eye contour segmentation data to obtain pupil region extraction data includes: using Hough transform to perform pupil region detection on the human eye contour segmentation data to obtain pupil region extraction data.

[0012] In one embodiment of the present invention, obtaining two-dimensional UV coordinate data based on the human eye contour segmentation data and the pupil region extraction data includes: traversing the human eye contour segmentation data based on a set edge threshold to obtain two-dimensional coordinate data of the temporal orbital edge; and obtaining two-dimensional coordinate data of the pupil edge based on the center coordinate data and radius coordinate data in the pupil region extraction data.

[0013] In one embodiment of the present invention, the step of performing three-dimensional spatial coordinate mapping on the two-dimensional UV coordinate data to obtain corresponding three-dimensional coordinate data includes: based on a three-dimensional model file, performing three-dimensional spatial coordinate mapping on the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupil edge respectively to obtain corresponding three-dimensional coordinate data of the temporal orbital edge and three-dimensional coordinate data of the pupil edge; wherein, the three-dimensional model file includes: vertex three-dimensional coordinate data, vertex texture coordinate data and vertex normal data corresponding to multiple vertices respectively.

[0014] In one embodiment of the present invention, the step of mapping the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupillary edge to three-dimensional spatial coordinates based on the three-dimensional model file to obtain the corresponding three-dimensional coordinate data of the temporal orbital edge and the pupillary edge includes: calculating the Euclidean distance between each vertex and the temporal orbital edge based on the vertex texture coordinate data of each vertex in the three-dimensional model file and the two-dimensional coordinate data of the temporal orbital edge to determine the three-dimensional coordinate data of the temporal orbital edge; determining one or more circumferential discrete points through the two-dimensional coordinate data of the pupillary edge, and obtaining the three-dimensional coordinate data of the pupillary edge according to the three-dimensional model file.

[0015] In one embodiment of the present invention, determining one or more discrete points on the circumference using two-dimensional coordinate data of the pupil edge and obtaining three-dimensional coordinate data of the pupil edge according to the three-dimensional model file includes: obtaining a circular analytical expression of the pupil outline based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the left and right vertices of the pupil from the two-dimensional coordinate data of the pupil edge; taking one or more discrete points on the circumference corresponding to the circular analytical expression; and obtaining three-dimensional coordinate data of each discrete point according to the three-dimensional model file; obtaining a circular analytical expression of the pupil interior outline based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the pupil interior near the left and right vertices of the pupil from the two-dimensional coordinate data of the pupil edge; taking one or more discrete points on the circumference corresponding to the circular analytical expression; and obtaining three-dimensional coordinate data of each discrete point according to the three-dimensional model file.

[0016] In one embodiment of the present invention, the step of calculating the three-dimensional spherical center coordinates and eyeball radius length based on the three-dimensional coordinate data to obtain eyeball protrusion includes: obtaining the spherical center coordinates and eyeball radius length by least squares fitting based on the matrixed three-dimensional spherical coordinate equation; calculating the vertical distance between the spherical center and the temporal orbital edge based on the spherical center coordinates and the three-dimensional coordinate data of the temporal orbital edge; and obtaining the eyeball protrusion based on the eyeball radius length and the vertical distance.

[0017] To achieve the above and other related objectives, the present invention provides an eyeball protrusion measurement system, the system comprising: a two-dimensional file conversion module for converting a three-dimensional face file to be measured into a corresponding two-dimensional texture file; an eye region recognition module connected to the two-dimensional file conversion module for recognizing the eye region based on the two-dimensional texture file to obtain eye region recognition data; an eye contour segmentation module connected to the eye region recognition module for segmenting the eye contour based on the eye region recognition data to obtain eye contour segmentation data; and a pupil region extraction module connected to the eye contour segmentation module for extracting the pupil region based on the eye contour segmentation data to obtain pupil region data. The system includes: a domain extraction data module; a two-dimensional coordinate acquisition module, connected to the human eye contour segmentation module and the pupil region extraction module, used to obtain two-dimensional UV coordinate data based on the human eye contour segmentation data and the pupil region extraction data; wherein, the two-dimensional UV coordinate data includes: two-dimensional coordinate data of the temporal orbital edge and two-dimensional coordinate data of the pupil edge; a three-dimensional coordinate acquisition module, connected to the two-dimensional coordinate acquisition module, used to perform three-dimensional spatial coordinate mapping on the two-dimensional UV coordinate data to obtain the corresponding three-dimensional coordinate data; and an eyeball protrusion calculation module, connected to the three-dimensional coordinate acquisition module, used to calculate the three-dimensional spherical center coordinates and eyeball radius length based on the three-dimensional coordinate data to obtain the eyeball protrusion.

[0018] As described above, this invention provides a method and system for measuring eye protrusion, offering the following advantages: First, the invention converts a 3D face file into a 2D texture, simplifying the complexity of computer vision processing. Then, the eye region is extracted, removing redundant parts of the image to improve accuracy. Next, the eye contour is extracted, and after detecting the eye socket, the pupil is detected, extracting the coordinates of the temporal orbital edge and pupil edge on the 2D texture. This is then mapped back to a 3D photograph, and the 3D spherical coordinate equation is matrixed and fitted using the least squares method to obtain the coordinates of the spherical center and the eyeball radius, allowing the calculation of the eye protrusion distance. This application uses a convenient, contactless 3D face image as input, and through the application of artificial intelligence, computer vision, and mathematical calculations based on 3D coordinates, achieves automated and accurate measurement of eye protrusion. Attached Figure Description

[0019] Figure 1 The diagram shown is a flowchart illustrating the eyeball protrusion measurement method according to an embodiment of the present invention.

[0020] Figure 2 The diagram shown is a structural schematic of a YOLO network according to an embodiment of the present invention.

[0021] Figure 3 The diagram shown is a flowchart illustrating the eyeball protrusion measurement method according to an embodiment of the present invention.

[0022] Figure 4 The diagram shown is a structural schematic of an ocular protrusion measurement system according to an embodiment of the present invention. Detailed Implementation

[0023] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0024] It should be noted that in the following description, reference is made to the accompanying drawings, which illustrate several embodiments of the invention. It should be understood that other embodiments may also be used, and changes in mechanical composition, structure, electrical system, and operation may be made without departing from the spirit and scope of the invention. The following detailed description should not be considered limiting, and the scope of the embodiments of the invention is defined only by the claims of the published patents. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. Spatially related terms, such as “upper,” “lower,” “left,” “right,” “below,” “below,” “lower part,” “above,” “upper part,” etc., may be used herein to illustrate the relationship between one element or feature shown in the figures and another element or feature.

[0025] Throughout this specification, when it is said that a part is "connected" to another part, this includes not only "direct connection" but also "indirect connection" by placing other elements in between. Furthermore, when it is said that a part "includes" a certain constituent element, unless otherwise stated otherwise, this does not exclude other constituent elements, but rather means that other constituent elements may also be included.

[0026] The terms "first," "second," and "third," etc., used herein are for the purpose of describing various parts, components, regions, layers, and / or segments, but are not limiting. These terms are used only to distinguish one part, component, region, layer, or segment from others. Therefore, the "first part," "component," "region," "layer," or "segment" described below may refer to a "second part," "component," "region," "layer," or "segment" without departing from the scope of this invention.

[0027] Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of the stated feature, operation, element, component, item, kind, and / or group, but do not preclude the presence, occurrence, or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition arise only when combinations of elements, functions, or operations are inherently mutually exclusive in some manner.

[0028] There are significant differences in current methods for measuring eyeball protrusion and even in the standards for eyeball protrusion. There is an urgent need for a reasonable, convenient, stable and consistent method to help clinicians better study and measure eyeball protrusion and obtain standard reference values ​​for eyeball protrusion suitable for different age groups of Chinese people, so as to better help in the clinical diagnosis or prediction of related diseases.

[0029] Therefore, this invention provides a method and system for measuring eye protrusion. First, a 3D face file is converted into a 2D texture, simplifying the complexity of computer vision processing. Then, the eye region is extracted, removing redundant parts of the image to improve accuracy. Next, the eye contour is extracted, and after detecting the eye socket, the pupil is detected, extracting the coordinates of the temporal orbital edge and pupil edge on the 2D texture. This is then mapped back to a 3D photograph, and the 3D spherical coordinate equation is matrixed and fitted using the least squares method to obtain the coordinates of the spherical center and the eyeball radius, allowing the calculation of the eye protrusion distance. This application uses a convenient, contactless 3D face image as input, and through the use of artificial intelligence, computer vision, and mathematical calculations based on 3D coordinates, achieves automated and accurate measurement of eye protrusion.

[0030] The present invention will now be described in detail with reference to the accompanying drawings, so that those skilled in the art can readily implement it. The present invention can be embodied in many different forms and is not limited to the embodiments described herein.

[0031] like Figure 1 A flowchart illustrating a method for measuring ocular protrusion according to an embodiment of the present invention is shown.

[0032] The method includes:

[0033] Step S11: Convert the 3D face file to be measured into the corresponding 2D texture file.

[0034] In measuring bulging eye, we need to use 3D faces as input. However, current mature face parsing and semantic segmentation algorithms are all based on 2D faces. Therefore, it is necessary to convert the 3D face file into a corresponding 2D texture file.

[0035] Optionally, we can use UV mapping to map 3D faces to 2D texture files. UV stands for coordinates in the 2D texture file. We only need to perform eye segmentation on the 2D texture file (JPG format, as shown in the image above) to find the corresponding eye contour coordinates in the original 3D face.

[0036] Step S12: Identify the eye region based on the two-dimensional texture file to obtain eye region identification data.

[0037] Optionally, since the texture contains a lot of redundant parts, directly performing edge detection will affect the accuracy. We first perform target detection and extraction on the rectangular area of ​​the eye, remove the parts outside the rectangular area, and then further recognize the eye contour.

[0038] Step S12 includes: based on the YOLO network, identifying the rectangular eye region in the two-dimensional texture file to obtain eye region recognition data.

[0039] Object detection is one of the most critical problems in computer vision. There are many machine learning algorithms available for this purpose, such as R-CNN, Retina-Net, and ResNet. We adopted the classic YOLO algorithm, which not only has high accuracy but also significant advantages in real-time performance, greatly improving the speed of practical applications.

[0040] Preferably, the basic structure of the YOLO network used is as follows: Figure 2 As shown, YOLO is divided into two parts: feature extraction (Darknet53 without FC layer) and prediction / classification. The CBL module in the diagram consists of a convolutional layer (conv), a normalization layer (BN), and an activation function layer (Leaky ReLU), forming a basic convolutional module. The Res unit, or residual block, performs a skip connection every two convolutions. ResX is a residual unit, composed of one CBL and x residual blocks. The feature extraction part borrows the residual skip connection structure from the ResNet network, using a large number of residual blocks (Res units) to avoid overfitting. Simultaneously, to reduce the negative gradient effect of pooling, YOLO directly uses the convolutional stride for downsampling. The prediction / classification borrows the upsampling and fusion algorithms from the FPN network, detecting on feature maps of three scales, enhancing the accuracy of small object detection, which is also suitable for detecting small objects like the human eye.

[0041] Step S13: Segment the human eye contour based on the eye region recognition data to obtain human eye contour segmentation data.

[0042] Optionally, after recognizing the rectangular region of the eye, we need to segment the outline of the human eye. Because image edges exhibit abrupt changes in information, such as color and texture, traditional vision methods often employ differentiation for edge recognition. Considering the relatively distinct eye socket outline and significant color difference between the inner and outer regions, directly using traditional vision methods can achieve good results. Most traditional edge detection methods are based on convolution using the gradient's directional derivative, with commonly used operators including the Sobel operator and the Canny operator.

[0043] Preferably, step S13 includes: using the Sobel operator to segment the eye region recognition data to obtain human eye contour segmentation data. Specifically, the Sobel operator is a discrete differential operator that combines Gaussian smoothing and differential differentiation. The Sobel operator detects edges based on the phenomenon that the weighted difference of the gray levels of the pixel's upper and lower, left and right neighboring points reaches an extreme value at the edge. It has a smoothing effect on noise and provides more accurate edge direction information. Because the Sobel operator combines Gaussian smoothing and differential differentiation (differentiation), the result has greater noise resistance. Due to the strong noise interference from eyelashes near the eye socket, the noise resistance of this operator is also of practical significance for eye socket recognition.

[0044] Compared to the Canny operator, the Sobel operator produces results with more texture details, while the Canny operator under the aforementioned parameter conditions is binarized, resulting in a more general outline. Since some texture detail is needed for subsequent pupil localization, the Sobel operator is used for processing.

[0045] Step S14: Extract the pupil region based on the human eye contour segmentation data to obtain pupil region extraction data.

[0046] Optionally, step S14 includes: using Hough Transform to detect the pupil region in the human eye contour segmentation data to obtain pupil region extraction data. Hough Transform is a feature extraction technique in image processing that uses a voting algorithm to detect objects with specific shapes. Hough Transform is one of the fundamental methods for identifying geometric shapes from images in image processing. The basic principle of Hough Transform is to utilize the duality of points and lines to transform a given curve in the original image space into a point in the parameter space. This transforms the problem of detecting a given curve in the original image into finding a peak value in the parameter space. In other words, it transforms the detection of global characteristics into the detection of local characteristics, such as straight lines, ellipses, circles, and arcs.

[0047] To detect the boundary region of the pupil, the contour image obtained using the Sobel operator is used to apply the Hough transform to detect circles, thus obtaining the result. The specific algorithm consists of two parts: estimating the circle center and estimating the radius. When estimating the circle center, the circle center parameter space N(a,b) is initialized first. All non-zero pixels are traversed, and N(a,b) is statistically sorted to obtain possible circle centers. To estimate the radius, a circle center (a,b) is selected first. The distances from all non-zero points to the circle center are calculated and sorted from smallest to largest. A suitable possible radius is selected based on a threshold. Then, using the parameter space method, the final radius value is obtained by statistically sorting the possible radii.

[0048] Step S15: Obtain two-dimensional UV coordinate data based on the human eye contour segmentation data and pupil region extraction data.

[0049] In detail, the extraction of two-dimensional UV coordinates (two-dimensional texture coordinates) includes the extraction of temporal orbital edge coordinates and the extraction of pupil edge coordinates. The two-dimensional UV coordinate data includes: temporal orbital edge two-dimensional coordinate data and pupil edge two-dimensional coordinate data.

[0050] Optionally, step S15 includes:

[0051] Based on a set edge threshold, the human eye contour segmentation data is traversed to obtain the two-dimensional coordinate data of the temporal orbital edge. Specifically, based on the matrix obtained by the Sobel operator (the essence of an image is a matrix), an edge threshold t is set. If the value of a point is greater than the threshold, it means that the point belongs to the eye contour. The matrix is ​​traversed from right to left and from left to right respectively. Among the points with coordinates greater than the threshold, the leftmost and rightmost points are the temporal orbital edges of the eye. That is, their corresponding coordinates are the two-dimensional coordinate data of the temporal orbital edge.

[0052] Based on the center coordinates and radius coordinates extracted from the pupil region data, two-dimensional coordinate data of the pupil edge is obtained. Specifically, the center coordinates and radius coordinates have already been obtained during the Hough transform process, and can be directly used as the two-dimensional coordinate representation of the pupil edge, i.e., the two-dimensional coordinate data of the pupil edge.

[0053] Step S16: Map the two-dimensional UV coordinate data to three-dimensional spatial coordinates to obtain the corresponding three-dimensional coordinate data.

[0054] Optionally, step S16 includes: based on the three-dimensional model file, performing three-dimensional spatial coordinate mapping on the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupil edge respectively to obtain the corresponding three-dimensional coordinate data of the temporal orbital edge and the three-dimensional coordinate data of the pupil edge; wherein, the three-dimensional model file includes: vertex three-dimensional coordinate data, vertex texture coordinate data and vertex normal data corresponding to multiple vertices respectively.

[0055] The 3D model file is an OBJ file, and the meanings of its parameters are as follows: v (vertices): 3D coordinates of the vertices. vt (vertex texture): Vertex texture coordinates, which are 2D coordinates. There is a 2D texture map that corresponds one-to-one with this OBJ file. Since v only stores spatial position information, vt is used to supplement and store texture information, and this correspondence is exactly what this paper needs to utilize. vn (vertex normal): Vertex normal. A 3D volume is composed of countless faces, and vn is the normal of these faces.

[0056] Optionally, the step of mapping the two-dimensional coordinate data of the temporal orbital margin and the two-dimensional coordinate data of the pupillary margin to three-dimensional spatial coordinates based on the three-dimensional model file to obtain the corresponding three-dimensional coordinate data of the temporal orbital margin and the three-dimensional coordinate data of the pupillary margin includes:

[0057] Based on the vertex texture coordinate data of each vertex in the 3D model file and the 2D coordinate data of the temporal orbital edge, the Euclidean distance between each vertex and the temporal orbital edge is calculated to determine the 3D coordinate data of the temporal orbital edge. Specifically, for the temporal orbital edge, all vt values ​​in the 3D model file are traversed to calculate their Euclidean distances to the temporal orbital edge. The point with the smallest Euclidean distance is found as the closest vt coordinate, and the corresponding v point coordinates are obtained through the OBJ file, which are the 3D coordinates corresponding to the temporal orbital edge.

[0058] One or more discrete points on the circumference are determined by the two-dimensional coordinate data of the pupil edge, and the three-dimensional coordinate data of the pupil edge is obtained according to the three-dimensional model file.

[0059] Optionally, for the pupil, to reduce the z-direction error generated during meshlab fitting, we do not directly use the three-dimensional coordinates corresponding to the pupil center point in the two-dimensional image, but instead perform the following optimization steps:

[0060] Based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the left and right vertices of the pupil, a circular analytical expression of the pupil outline is obtained. One or more discrete points are taken on the circumference corresponding to the circular analytical expression, and the three-dimensional coordinate data of each discrete point is obtained based on the three-dimensional model file.

[0061] Based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the pupil interior near the left and right vertices of the pupil, a circular analytical expression for the inner contour of the pupil is obtained. One or more discrete points are taken on the circumference corresponding to the circular analytical expression, and the three-dimensional coordinate data of each discrete point is obtained according to the three-dimensional model file.

[0062] Step S17: Calculate the three-dimensional center coordinates and eyeball radius length based on the three-dimensional coordinate data to obtain the eyeball protrusion.

[0063] Optionally, step S17 includes:

[0064] Based on the matrixed three-dimensional spherical coordinate equation, the coordinates of the sphere center and the length of the eyeball radius are obtained by least squares fitting based on the three-dimensional coordinate data.

[0065] The vertical distance between the center of the sphere and the edge of the temporal orbit is calculated based on the coordinates of the sphere's center and the three-dimensional coordinates of the temporal orbital edge.

[0066] The eyeball protrusion is obtained based on the eyeball radius and the vertical distance.

[0067] Preferably, we first consider the analytical expression of the sphere in three-dimensional space.

[0068] (xa) 2 +(yb) 2 +(zc) 2 =r 2 (1)

[0069] Expand

[0070] x 2 -2ax+a 2 +y 2 -2by+b 2 +z 2 -2cz+c 2 =r 2 (2)

[0071] Organized

[0072] (-2x)*a+(-2y)*b+(-2z)*c+1*(a 2 +b 2 +c 2 -r 2 )=-x 2 -y 2 -z 2 (3)

[0073] Let A be the name of the person in question. x =-2x; A y =-2y; A Z =-2z; A d =1; d=a 2 +b 2 +c 2 -r 2 e = -x 2 -y 2 -z 2 ;

[0074] The equation can then be transformed into

[0075] A x a+A y b+A z c+A d d = e; (4)

[0076] That is, the matrix-based three-dimensional spherical coordinate equation:

[0077] By obtaining the three-dimensional coordinate data, we can obtain matrix A = [A x A y A z A d and vector e;

[0078] The optimal vector can be fitted using the least squares method. The coordinates of the sphere's center (a, b, c) can be obtained. Substituting a, b, and c into d, the radius r of the sphere can be calculated, which is the length of the eyeball radius.

[0079] Let the coordinates of the temporal orbital margin be (x1, y1, z1), then the distance of eyeball protrusion is...

[0080] h = |(c+r)-z1|; (6)

[0081] Currently, according to the Chinese internet and related medical textbooks, the average normal exophthalmos range is 12-14mm. Generally, anything exceeding 2mm is considered abnormal, and the difference between the two eyes should not exceed 2mm. A range below 12mm is called enophthalmos, and a range above 12mm is called exophthalmos. Both enophthalmos and exophthalmos are considered abnormalities in eye position and are important clinical symptoms.

[0082] Multicenter studies have shown that the measurement of ocular protrusion has a large bias, so much so that the standard reference value varies in different literature or data sources. This further reflects the need for a method that is highly reproducible and convenient enough to accumulate a large amount of data to obtain reliable results.

[0083] For example, in the Chengdu area, the Hertel exophthalmos measurements for adult Han Chinese were as follows: males: left eye (15.3±2.2) mm, right eye (15.0±2.1) mm; females: left eye (14.1±2.0) mm, right eye (14.0±2.0) mm. In Jining City, a cluster random sampling of normal individuals over 50 years of age showed the following Hertel exophthalmos measurements: right eye: average 12.73±2.43 mm; left eye: average 12.62±2.42 mm; right eye: average 15.70±2.07 mm; left eye: average 15.83±2.22 mm. In Qingdao, measurements of exophthalmos in children and adolescents showed an average of 10.47 mm for both sexes in the 5-year-old group, gradually increasing to 13.91 mm in the 17-year-old group. International research indicates that measurements using the Hertel exophthalmometer within the range of 14-21 mm are considered normal, and exophthalmos measurements vary among different ethnic groups.

[0084] To better illustrate the above-mentioned method for measuring ocular protrusion, the present invention provides the following specific embodiments.

[0085] Example 1: A method for measuring ocular protrusion. (e.g.) Figure 3 The diagram shown illustrates the process for measuring ocular protrusion; the methods include:

[0086] Based on the 3D image of the face, we establish a mapping relationship between the 3D and 2D photos. We find the coordinates of the center and edge of the eyeball in the 2D face, and then map them back to 3D. The height difference between the two points is the eyeball protrusion.

[0087] First, UV mapping is used to convert 3D face files into 2D textures, simplifying the complexity of computer vision processing. A YOLO network is used to extract the eye region, removing redundant parts of the image and improving accuracy. To extract the eye contour, we compared the performance of common edge detection operators, Canny and Sobel, in orbital recognition, ultimately choosing the more effective Sobel operator. After detecting the eye orbit, we used Hough Transform to detect the pupil, specifically locating the pupil edge by detecting circles. At this point, we extracted the temporal orbital edge and pupil edge onto the 2D texture. Standard Then, by mapping back to the 3D photograph, matrixing the 3D spherical coordinate equation, and using the least squares method to fit the coordinates of the sphere center and the length of the eyeball radius, the eyeball protrusion distance can be calculated.

[0088] In summary, current methods for measuring eye protrusion, and even existing standards for eye protrusion, exhibit significant inconsistencies. There is an urgent need for a reasonable, convenient, stable, and consistent method to help clinicians better study and measure eye protrusion, and to obtain standard reference values ​​for eye protrusion suitable for different age groups in Chinese individuals, thereby better aiding in the clinical diagnosis and prediction of related diseases. This embodiment uses convenient, contactless 3D facial images as input, and through the application of artificial intelligence, computer vision, and mathematical calculations based on 3D coordinates, achieves automated and precise measurement of eye protrusion. Compared to traditional methods, this patent makes eye protrusion measurement accurate, objective, safe, convenient, low-cost, repeatable, and with high patient compliance. Furthermore, the changes in eye protrusion during disease progression have high reference value for clinical observation. The convenient and rapid image recognition enabled by artificial intelligence helps us accurately and quickly measure eye protrusion, thereby promoting the screening of orbital diseases and achieving early diagnosis and treatment.

[0089] Furthermore, this method possesses good scalability, data storability, and extensibility. First, the application of artificial intelligence technology allows this diagnostic process to be largely unrestricted by objective conditions, such as time, location, medical equipment, and hardware facilities. Second, compared to manually measuring exophthalmos, the algorithm directly derives the exophthalmos value, making it easy to aggregate the results into a database, eliminating the need for digitization and archiving. Finally, as the data becomes more comprehensive, the method will continuously improve its accuracy. If supplemented with other methods, it can go beyond simply measuring exophthalmos value and further assist in the precise diagnosis of orbital diseases.

[0090] Similar in principle to the above embodiments, the present invention provides an eyeball protrusion measurement system.

[0091] The following specific embodiments are provided in conjunction with the accompanying drawings:

[0092] like Figure 4 A schematic diagram of the structure of an ocular protrusion measurement system according to an embodiment of the present invention is shown.

[0093] The system includes:

[0094] The 2D file conversion module 41 is used to convert the 3D face file to be measured into the corresponding 2D texture file;

[0095] The eye region recognition module 42 is connected to the two-dimensional file conversion module 41 and is used to recognize the eye region based on the two-dimensional texture file to obtain eye region recognition data.

[0096] The human eye contour segmentation module 43 is connected to the eye region recognition module 42 and is used to segment the human eye contour based on the eye region recognition data to obtain human eye contour segmentation data.

[0097] The pupil region extraction module 44 is connected to the human eye contour segmentation module 43 and is used to extract the pupil region based on the human eye contour segmentation data to obtain pupil region extraction data.

[0098] The two-dimensional coordinate acquisition module 45 is connected to the human eye contour segmentation module 43 and the pupil region extraction module 44, and is used to obtain two-dimensional UV coordinate data based on the human eye contour segmentation data and the pupil region extraction data; wherein, the two-dimensional UV coordinate data includes: temporal orbital edge two-dimensional coordinate data and pupil edge two-dimensional coordinate data.

[0099] The three-dimensional coordinate acquisition module 46 is connected to the two-dimensional coordinate acquisition module 45 and is used to perform three-dimensional spatial coordinate mapping on the two-dimensional UV coordinate data to obtain the corresponding three-dimensional coordinate data.

[0100] The eyeball protrusion calculation module 47 is connected to the three-dimensional coordinate acquisition module 46 and is used to calculate the three-dimensional center coordinates and eyeball radius length based on the three-dimensional coordinate data to obtain the eyeball protrusion.

[0101] It should be noted that, as should be understood Figure 4 The division of modules in the system embodiment is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these units can be implemented entirely in software through processing element calls; they can be implemented entirely in hardware; or some units can be implemented by processing element calls to software, while others are implemented in hardware.

[0102] Since the implementation principle of this eyeball protrusion measurement system has been described in the foregoing embodiments, it will not be repeated here.

[0103] Optionally, the step of identifying the eye region based on the two-dimensional texture file to obtain eye region recognition data includes: identifying the rectangular eye region in the two-dimensional texture file based on the YOLO network to obtain eye region recognition data.

[0104] Optionally, the step of segmenting the human eye contour based on the eye region recognition data to obtain human eye contour segmentation data includes: using the Sobel operator to segment the human eye contour on the eye region recognition data to obtain human eye contour segmentation data.

[0105] Optionally, the step of extracting the pupil region based on the human eye contour segmentation data to obtain pupil region extraction data includes: using Hough transform to perform pupil region detection on the human eye contour segmentation data to obtain pupil region extraction data.

[0106] Optionally, obtaining two-dimensional UV coordinate data based on the human eye contour segmentation data and pupil region extraction data includes: traversing the human eye contour segmentation data based on a set edge threshold to obtain two-dimensional coordinate data of the temporal orbital edge; and obtaining two-dimensional coordinate data of the pupil edge based on the center coordinate data and radius coordinate data in the pupil region extraction data.

[0107] Optionally, the step of mapping the two-dimensional UV coordinate data to three-dimensional spatial coordinates to obtain corresponding three-dimensional coordinate data includes: based on the three-dimensional model file, mapping the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupil edge to three-dimensional spatial coordinates respectively to obtain corresponding three-dimensional coordinate data of the temporal orbital edge and three-dimensional coordinate data of the pupil edge; wherein, the three-dimensional model file includes: vertex three-dimensional coordinate data, vertex texture coordinate data and vertex normal data corresponding to multiple vertices respectively.

[0108] Optionally, the step of mapping the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupillary edge to three-dimensional spatial coordinates based on the three-dimensional model file to obtain the corresponding three-dimensional coordinate data of the temporal orbital edge and the pupillary edge includes: calculating the Euclidean distance between each vertex and the temporal orbital edge based on the vertex texture coordinate data of each vertex in the three-dimensional model file and the two-dimensional coordinate data of the temporal orbital edge to determine the three-dimensional coordinate data of the temporal orbital edge; determining one or more circumferential discrete points through the two-dimensional coordinate data of the pupillary edge, and obtaining the three-dimensional coordinate data of the pupillary edge according to the three-dimensional model file.

[0109] Optionally, determining one or more discrete points on the circumference using the two-dimensional coordinate data of the pupil edge, and obtaining the three-dimensional coordinate data of the pupil edge according to the three-dimensional model file includes: obtaining a circular analytical expression of the pupil outline based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the left and right vertices of the pupil from the two-dimensional coordinate data of the pupil edge, and taking one or more discrete points on the circumference corresponding to the circular analytical expression, and obtaining the three-dimensional coordinate data of each discrete point according to the three-dimensional model file; obtaining a circular analytical expression of the inner contour of the pupil based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the inner pupil near the left and right vertices of the pupil from the two-dimensional coordinate data of the pupil edge, and taking one or more discrete points on the circumference corresponding to the circular analytical expression, and obtaining the three-dimensional coordinate data of each discrete point according to the three-dimensional model file.

[0110] Optionally, the step of calculating the three-dimensional spherical center coordinates and eyeball radius length based on the three-dimensional coordinate data to obtain eyeball protrusion includes: obtaining the spherical center coordinates and eyeball radius length by least squares fitting based on the matrixed three-dimensional spherical coordinate equation; calculating the vertical distance between the spherical center and the temporal orbital edge based on the spherical center coordinates and the three-dimensional coordinate data of the temporal orbital edge; and obtaining the eyeball protrusion based on the eyeball radius length and the vertical distance.

[0111] In summary, the eye protrusion measurement system of the present invention first converts a 3D face file into a 2D texture, simplifying the complexity of computer vision processing. Then, the eye region is extracted, removing redundant parts of the image to improve accuracy. Next, the eye contour is extracted, and after detecting the eye socket, the pupil is detected, extracting the coordinates of the temporal orbital edge and pupil edge on the 2D texture. This is then mapped back to a 3D photograph, and the 3D spherical coordinate equation is matrixed and fitted using the least squares method to obtain the coordinates of the spherical center and the eyeball radius, thus calculating the eye protrusion distance. This application uses a convenient and contactless 3D face image as input, and through the use of artificial intelligence, computer vision, and mathematical calculations based on 3D coordinates, it achieves automated and accurate measurement of eye protrusion. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has high industrial applicability.

[0112] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for measuring ocular protrusion, characterized in that, The method includes: Convert the 3D face file to be measured into the corresponding 2D texture file; The eye region is identified based on the two-dimensional texture file to obtain eye region recognition data; The human eye contour is segmented based on the eye region recognition data to obtain human eye contour segmentation data; Extracting the pupil region based on the human eye contour segmentation data to obtain pupil region extraction data includes: using Hough transform to detect the pupil region on the human eye contour segmentation data to obtain pupil region extraction data. The process of obtaining two-dimensional UV coordinate data based on the human eye contour segmentation data and pupil region extraction data includes: traversing the human eye contour segmentation data based on a set edge threshold to obtain two-dimensional coordinate data of the temporal orbital edge; and obtaining two-dimensional coordinate data of the pupil edge based on the center coordinate data and radius coordinate data in the pupil region extraction data. The two-dimensional UV coordinate data includes two-dimensional coordinate data of the temporal orbital edge and two-dimensional coordinate data of the pupil edge. Mapping the two-dimensional UV coordinate data to three-dimensional spatial coordinates to obtain corresponding three-dimensional coordinate data includes: based on a three-dimensional model file, mapping the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupillary edge to three-dimensional spatial coordinates respectively to obtain corresponding three-dimensional coordinate data of the temporal orbital edge and the pupillary edge; wherein, the three-dimensional model file includes: vertex three-dimensional coordinate data, vertex texture coordinate data, and vertex normal data corresponding to multiple vertices respectively; wherein, the step of mapping the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupillary edge to three-dimensional spatial coordinates respectively based on the three-dimensional model file to obtain corresponding three-dimensional coordinate data of the temporal orbital edge and the pupillary edge includes: Based on the vertex texture coordinate data of each vertex in the 3D model file and the 2D coordinate data of the temporal orbital edge, the Euclidean distance between each vertex and the temporal orbital edge is calculated to determine the 3D coordinate data of the temporal orbital edge; one or more circumferential discrete points are determined by the 2D coordinate data of the pupil edge, and the 3D coordinate data of the pupil edge is obtained according to the 3D model file. The three-dimensional coordinates of the spherical center and the radius of the eyeball are calculated based on the three-dimensional coordinate data to obtain the eyeball protrusion. This includes: obtaining the spherical center coordinates and the radius of the eyeball by least squares fitting based on the matrixed three-dimensional spherical coordinate equation; calculating the vertical distance between the spherical center and the temporal orbital edge based on the spherical center coordinates and the three-dimensional coordinate data of the temporal orbital edge; and obtaining the eyeball protrusion based on the eyeball radius and the vertical distance.

2. The method for measuring ocular protrusion according to claim 1, characterized in that, The step of identifying the eye region based on the two-dimensional texture file to obtain eye region recognition data includes: Based on the YOLO network, the rectangular region of the eye in the two-dimensional texture file is identified to obtain eye region recognition data.

3. The method for measuring ocular protrusion according to claim 1, characterized in that, The step of segmenting the human eye contour based on the eye region recognition data to obtain human eye contour segmentation data includes: The Sobel operator is used to segment the human eye contour in the eye region recognition data to obtain human eye contour segmentation data.

4. The method for measuring ocular protrusion according to claim 1, characterized in that, The step of determining one or more discrete points on the circumference using two-dimensional coordinate data of the pupil edge, and obtaining three-dimensional coordinate data of the pupil edge based on the three-dimensional model file includes: Based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the left and right vertices of the pupil, a circular analytical expression of the pupil outline is obtained. One or more discrete points are taken on the circumference corresponding to the circular analytical expression, and the three-dimensional coordinate data of each discrete point is obtained based on the three-dimensional model file. Based on the two-dimensional coordinates of the pupil center and the two-dimensional coordinates of the pupil interior near the left and right vertices of the pupil, a circular analytical expression for the inner contour of the pupil is obtained. One or more discrete points are taken on the circumference corresponding to the circular analytical expression, and the three-dimensional coordinate data of each discrete point is obtained according to the three-dimensional model file.

5. A system for measuring eyeball protrusion, characterized in that, The system includes: The 2D file conversion module is used to convert the 3D face file to be measured into the corresponding 2D texture file; An eye region recognition module, connected to the two-dimensional file conversion module, is used to recognize the eye region based on the two-dimensional texture file to obtain eye region recognition data; The human eye contour segmentation module is connected to the eye region recognition module and is used to segment the human eye contour based on the eye region recognition data to obtain human eye contour segmentation data. The pupil region extraction module is connected to the human eye contour segmentation module and is used to extract the pupil region based on the human eye contour segmentation data to obtain pupil region extraction data. The pupil region extraction module is also used to perform pupil region detection on the human eye contour segmentation data using Hough transform to obtain pupil region extraction data. A two-dimensional coordinate acquisition module, connected to the human eye contour segmentation module and the pupil region extraction module, is used to obtain two-dimensional UV coordinate data based on the human eye contour segmentation data and the pupil region extraction data; wherein, the two-dimensional UV coordinate data includes: two-dimensional coordinate data of the temporal orbital edge and two-dimensional coordinate data of the pupil edge; The two-dimensional coordinate acquisition module is also used to traverse the human eye contour segmentation data based on a set edge threshold to obtain two-dimensional coordinate data of the temporal orbital edge; and to obtain two-dimensional coordinate data of the pupil edge based on the center coordinate data and radius coordinate data extracted from the pupil region data. A three-dimensional coordinate acquisition module, connected to the two-dimensional coordinate acquisition module, is used to perform three-dimensional spatial coordinate mapping on the two-dimensional UV coordinate data to obtain the corresponding three-dimensional coordinate data. The three-dimensional coordinate acquisition module is further configured to perform three-dimensional spatial coordinate mapping on the two-dimensional coordinate data of the temporal orbital edge and the two-dimensional coordinate data of the pupil edge based on the three-dimensional model file, to obtain the corresponding three-dimensional coordinate data of the temporal orbital edge and the three-dimensional coordinate data of the pupil edge; wherein, the three-dimensional model file includes: vertex three-dimensional coordinate data, vertex texture coordinate data and vertex normal data corresponding to multiple vertices respectively; The three-dimensional coordinate acquisition module is also used to calculate the Euclidean distance between each vertex and the temporal orbital edge based on the vertex texture coordinate data of each vertex in the three-dimensional model file and the two-dimensional coordinate data of the temporal orbital edge, so as to determine the three-dimensional coordinate data of the temporal orbital edge; determine one or more circumferential discrete points through the two-dimensional coordinate data of the pupil edge, and obtain the three-dimensional coordinate data of the pupil edge according to the three-dimensional model file; An eyeball protrusion calculation module, connected to the three-dimensional coordinate acquisition module, is used to calculate the three-dimensional center coordinates and eyeball radius length based on the three-dimensional coordinate data, so as to obtain the eyeball protrusion. The eyeball protrusion calculation module is also used to obtain the coordinates of the center of the eyeball and the radius of the eyeball by least squares fitting based on the matrixed three-dimensional spherical coordinate equation; to calculate the vertical distance between the center of the eyeball and the temporal orbital edge based on the coordinates of the center of the eyeball and the three-dimensional coordinate data of the temporal orbital edge; and to obtain the eyeball protrusion based on the radius of the eyeball and the vertical distance.

Images