External auricle key point detection method

Abstract

The invention provides an external auricle key point detection method, which comprises the following steps: preprocessing a to-be-detected external auricle image to obtain a target external auricle image; inputting the target external auricle image into a pre-trained target detection model to obtain a prediction key point on the external auricle; the target detection model comprises a shared encoder, a region decoder for outputting an external auricle contour region and a key point decoder for outputting key points on external auricles; in the training process of the target detection model, a target loss function is utilized to adjust model parameters of the target detection model, the target loss function comprises collinear loss items, and the collinear loss items are determined according to position information of key points with collinear relations on the outer auricles; and performing fitting optimization processing on the predicted key points according to a collinear relationship between the predicted key points to obtain an optimal key point. The method can improve the positioning precision of the target detection model, and further improves the measurement accuracy of the key points.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and more specifically, to a method for detecting key points on the auricle. Background Technology

[0002] Currently, in the field of cosmetic surgery, auricular deformity surgery and reconstruction surgery are important means to improve the appearance of patients' auricles and restore normal physiological function. The accuracy of the detection of key morphological points of the auricle directly affects the surgical outcome. Among them, the auricle mainly includes 10 key points, namely, the supraauricular point, the infraauricular point, the antitragus point, the intersection of the line connecting the supraauricular point and the infraauricular point and the lower crus of the antihelix, the outermost point of the auricle, the anterior auricular point, the intersection of the lower crus of the antihelix and the helix, the supratragus notch, the tragus point, and the intertragus notch, which contain key morphological information of the auricle.

[0003] Existing methods for detecting key points on the outer ear mostly rely on manual operation or simple image processing algorithms. Manual operation is not only inefficient, but also greatly affected by subjective factors, making it difficult to guarantee the accuracy and consistency of measurement results. Simple image processing algorithms have poor detection accuracy and robustness when faced with complex and varied outer ear morphology, individual differences, and problems such as image noise, lighting, and angle. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method for detecting key points of the auricle, which can improve the positioning accuracy of the target detection model and thus improve the measurement accuracy of key points.

[0005] In a first aspect, embodiments of this application provide a method for detecting key points of the auricle, the method comprising: The image of the external auricle to be detected is preprocessed to obtain the target external auricle image; The target external auricle image is input into a pre-trained target detection model to obtain predicted key points on the external auricle. The target detection model includes a shared encoder, a region decoder for outputting the external auricle contour region, and a key point decoder for outputting key points on the external auricle. During the training of the target detection model, the model parameters are adjusted using a target loss function, which includes a collinearity loss term determined based on the location information of key points on the external auricle that have collinear relationships. Based on the collinear relationship between the predicted key points, the predicted key points are fitted and optimized to obtain the optimal key points.

[0006] In one optional embodiment, the target detection model is trained through the following steps: Obtain an image sample of the outer ear, wherein the image sample of the outer ear is labeled with actual key points; The external auricle image sample is preprocessed to obtain the target external auricle image sample; Using the outer ear region as the cropping area, the target outer ear image sample is cropped to obtain a cropped outer ear image sample. Construct a target detection model that includes a shared encoder, a region decoder, and a keypoint decoder; The target external auricle image sample and the external auricle cropped image sample are input into the shared encoder to extract general features; the extracted general features are input into the region decoder and the keypoint decoder respectively to train the region decoder and the keypoint decoder simultaneously. The model parameters of the target detection model are continuously updated and iterated to gradually reduce the target loss function until the convergence condition is met, thus obtaining a trained target detection model.

[0007] In one optional embodiment, the target loss function further includes a distance loss term, which is determined based on the predicted location information of the predicted key points and the actual location information of the actual key points.

[0008] In one optional embodiment, the target loss function is calculated using the following formula:

[0009] in, Represents the target loss function. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the collinearity loss term. This represents the distance loss term.

[0010] In one optional embodiment, the collinearity loss term is calculated using the following formula:

[0011] in, The term denotes the collinearity loss term, and det represents the modulus of the determinant. The first key point among the key points indicating a collinear relationship. The second key point among the key points indicating a collinear relationship. The third key point among the key points indicating a collinear relationship. The fourth key point among the key points indicating a collinear relationship.

[0012] In one optional embodiment, the preprocessing of the external auricle image sample to obtain the target external auricle image sample includes: Data augmentation is performed on the external auricle image sample to obtain the target external auricle image sample; The data augmentation includes at least one of the following: random color perturbation, random contrast perturbation, random brightness perturbation, random motion blur, random rotation, random flipping, and simulated surgical occlusion; Adjust the target external auricle image sample to a first fixed size; The target external auricle image sample is cropped using the external auricle region as the cropping region to obtain a cropped external auricle image sample, including: The target external auricle image sample is located based on an image processing algorithm; The located auricle region is cropped to obtain a cropped auricle image sample; The image sample of the outer ear is adjusted to a second fixed size; the first fixed size is larger than the second fixed size.

[0013] In one optional embodiment, the actual key points on the auricle include: supraauricular point, infraauricular point, antitragus point, the intersection of the line connecting the supraauricular point and the infraauricular point and the lower crus of the antihelix, the outermost point of the auricle, the anterior auricular point, the intersection of the lower crus of the antihelix and the helix, supratragus notch, tragus point, and intertragus notch. Among them, the points of the upper ear, lower ear, antitragus, and the intersection of the line connecting the upper ear and lower ear with the lower crus of the antihelix are collinear; the intersection of the lower crus of the antihelix with the helix, the supratragus notch, the tragus point, and the intertragus notch are collinear.

[0014] In an optional embodiment, the method further includes: Based on the optimal key points, target feature parameters for use in auricular surgery are determined; the target feature parameters include at least one of the following: the ratio of the helix to the antihelix angle, the proportion of the tragus notch spacing, and the golden ratio of the length and width of the auricle.

[0015] In one optional embodiment, the preprocessing of the image of the outer ear to be detected to obtain the target outer ear image includes: Data augmentation is performed on the image of the external auricle to be detected to obtain the target external auricle image; The data augmentation includes at least one of the following: random color perturbation, random contrast perturbation, random brightness perturbation, random motion blur, random rotation, random flipping, and simulated surgical occlusion; The image size of the target external auricle image is adjusted to a first fixed size.

[0016] In one optional embodiment, the step of performing fitting optimization processing on the predicted key points based on the collinear relationship between the predicted key points to obtain the optimal key points includes: Predicted key points that are collinear are input into a collinear decoder, which performs collinear fitting on the input predicted key points to obtain the optimal key points.

[0017] Secondly, embodiments of this application also provide a device for detecting key points of the auricle, the device comprising: The preprocessing module is used to preprocess the image of the external auricle to be detected, so as to obtain the target external auricle image; A keypoint detection module is used to input the target auricle image into a pre-trained target detection model to obtain predicted keypoints on the auricle. The target detection model includes a shared encoder, a region decoder for outputting the auricle contour region, and a keypoint decoder for outputting keypoints on the auricle. During the training process of the target detection model, the model parameters are adjusted using a target loss function, which includes a collinearity loss term determined based on the collinearity of keypoint positions on the auricle. The optimization processing module is used to perform fitting and optimization processing on the predicted key points based on the collinear relationship between them, so as to obtain the optimal key points.

[0018] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the external auricle key point detection method described above are performed.

[0019] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the external auricle key point detection method described above.

[0020] The method for detecting key points of the auricle provided in this application has at least the following technical effects: This study utilizes a target detection model incorporating a shared encoder and dual decoders for keypoint measurement. A collinearity loss term is introduced to constrain the collinearity of keypoints. Combined with fitting optimization, this improves the accuracy and robustness of keypoint detection in the auricle, addressing the issues of traditional algorithms' sensitivity to lighting and angle, as well as insufficient positioning accuracy. This provides accurate morphological data support for medical plastic surgery. Furthermore, the process requires no manual intervention, resulting in high efficiency and ensuring the accuracy and consistency of measurement results.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a method for detecting key points of the auricle provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of a target detection model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the location of key points on the auricle provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a key point detection device for the auricle provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0025] Research has found that in the field of cosmetic surgery, auricular deformity surgery and reconstruction surgery are important means to improve the appearance of patients' auricles and restore normal physiological function. The accuracy of auricular morphological key point detection is directly related to the surgical outcome. The auricle mainly includes 10 key points: the supraauricular point, infraauricular point, antitragus point, the intersection of the line connecting the supraauricular point and infraauricular point and the lower crus of the antihelix, the outermost point of the auricle, the anterior auricular point, the intersection of the lower crus of the antihelix and the helix, the supratragus notch, the tragus point, and the intertragus notch, which contain key morphological information of the auricle. Existing auricular key point detection methods mostly rely on manual operation or simple image processing algorithms. Manual operation is not only inefficient but also greatly affected by subjective factors, making it difficult to guarantee the accuracy and consistency of measurement results. Simple image processing algorithms have poor detection accuracy and robustness when faced with complex and varied auricular morphology, individual differences, and problems such as image noise, lighting, and angle, failing to meet the high precision and high reliability requirements of cosmetic surgery.

[0026] Based on this, embodiments of this application provide a method for detecting key points of the auricle to improve the positioning accuracy of the target detection model, thereby improving the measurement accuracy of key points.

[0027] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting key points of the auricle provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the method includes: S101. Preprocess the image of the external auricle to be detected to obtain the target external auricle image; S102. Input the target auricle image into a pre-trained target detection model to obtain predicted key points on the auricle; wherein, the target detection model includes a shared encoder, a region decoder for outputting the auricle contour region, and a key point decoder for outputting key points on the auricle; during the training process of the target detection model, the model parameters of the target detection model are adjusted using a target loss function, which includes a collinearity loss term, which is determined based on the location information of key points on the auricle that have collinear relationships; S103. Based on the collinear relationship between the predicted key points, the predicted key points are fitted and optimized to obtain the optimal key points.

[0028] Steps S101 to S103 above utilize a target detection model incorporating a shared encoder and dual decoders to measure key points. A collinearity loss term is introduced to constrain the collinearity of key points. Combined with fitting optimization, this improves the accuracy and robustness of key point detection in the auricle, resolving the issues of sensitivity to illumination and angle, and insufficient positioning accuracy in traditional algorithms. This provides accurate morphological data support for medical plastic surgery. Furthermore, no manual operation is required, resulting in high efficiency and ensuring the accuracy and consistency of measurement results.

[0029] The above steps are illustrated by specific embodiments below: In step S101, the image of the external auricle to be detected is preprocessed to obtain the target external auricle image.

[0030] Here, the image of the external auricle to be detected refers to the original image of the external auricle acquired from the patient. This image may contain issues such as uneven lighting, angular deviations, and background interference, which could affect the accuracy of subsequent keypoint detection. Therefore, preprocessing is necessary to improve image quality and make it more suitable for model processing. Preprocessing may include format conversion, resizing, and enhancement of the input image to improve image quality, enhance image features, and adapt to the model's input requirements. The target external auricle image refers to the preprocessed image that meets the input requirements of the target detection model. Its image quality and data features have been optimized, enabling better recognition and processing by the model.

[0031] Optionally, step S101 specifically includes: performing data enhancement on the image of the external auricle to be detected to obtain a target external auricle image; the data enhancement includes at least one of the following: random color perturbation, random contrast perturbation, random brightness perturbation, random motion blur, random rotation, random flipping, and simulated surgical occlusion; and adjusting the image size of the target external auricle image to a first fixed size.

[0032] In the above steps, preprocessing includes data augmentation and size adjustment. For example, random color perturbation and random contrast perturbation are performed on the image of the external auricle to be detected to simulate different lighting conditions; random rotation (±45°) and random flipping are used to enhance the model's adaptability to images at different angles.

[0033] Adjusting the image size of the target auricle image to a first fixed size is to standardize the image input specifications. For example, adjusting the image size to 512×512 pixels ensures that each image has the same size and resolution when input into the model, guaranteeing consistency and stability in model processing. Furthermore, if the input image size is 1024×1024 pixels, it can be adjusted to the first fixed size of 512×512 pixels, reducing computational load while preserving key features.

[0034] In addition, preprocessing can also include noise filtering, for example, using Gaussian filtering to remove random noise generated during image acquisition, ensuring the image quality of the input model.

[0035] In step S102, the target auricle image is input into a pre-trained target detection model to obtain predicted key points on the auricle. The target detection model includes a shared encoder, a region decoder for outputting the auricle contour region, and a key point decoder for outputting key points on the auricle. During the training of the target detection model, the model parameters of the target detection model are adjusted using a target loss function, which includes a collinearity loss term. The collinearity loss term is determined based on the location information of key points on the auricle that have collinear relationships.

[0036] In the above steps, the object detection model is built based on deep learning algorithms to realize the function of key point detection on the outer ear. It consists of a shared encoder, a region decoder, and a key point decoder. By learning from a large amount of image data, it can accurately identify key points and contour regions in the outer ear image. The shared encoder is one of the core components of the object detection model. Its function is to extract features from the input target outer ear image, transforming the image into feature vectors that the model can process. These feature vectors contain various information in the image, such as edges, textures, and shapes, providing a foundation for subsequent region decoding and key point detection. The region decoder receives the feature vectors extracted by the shared encoder and, through a series of calculations and processing, outputs the contour region of the outer ear, that is, it locates the approximate range of the outer ear in the outer ear image, providing a more accurate region range for key point detection. The key point decoder, based on the feature vectors extracted by the shared encoder and combined with the collinearity constraint module, outputs the key points on the outer ear. It is the key part of realizing key point detection, determining the position of each key point on the outer ear through the analysis and processing of the feature vectors.

[0037] The objective loss function is used to evaluate the difference between the predicted and actual results of the object detection model. By continuously adjusting the model parameters, the value of the objective loss function is minimized, thereby improving the detection accuracy of the model. The collinearity loss term is an important component of the objective loss function. It is calculated based on the location information of key points on the outer ear that have collinear relationships. Its purpose is to constrain the collinearity relationship between these key points and improve the accuracy of key point detection.

[0038] Specifically, the collinearity loss term calculates the collinearity loss for keypoint groups with collinear relationships, such as group 1 (comprising the supraauricular point, infraauricular point, antitragus point, the intersection of the line connecting the supraauricular point and infraauricular point and the lower crura of the antihelix) and group 2 (comprising the intersection of the lower crura of the antihelix and the helix, the supratragus notch, the tragus point, and the intertragus notch), based on their positional information. For example, for keypoints in group 1, the collinearity loss value for that group is obtained by calculating the modulus of the determinant of any three keypoints and averaging the results of all combinations. This constrains the collinearity relationship between these keypoints, making the keypoints output by the model more consistent with the actual anatomical structure.

[0039] For example, such as Figure 2 As shown, the object detection model includes a shared encoder backbone (using the HRNet network architecture) and is equipped with two decoders: a region decoder DetDecoder (using the RTMDetSepBNHead module) and a keypoint decoder PointDecoder (using the HeatmapHead module). The shared encoder is initialized by transferring pre-trained weights from face keypoint detection, which allows the network to significantly reduce the amount of training data required during training, thereby improving the model's training efficiency and generalization ability.

[0040] For example, after the target auricle image (512×512 pixels) is input into HRNet, the shared encoder extracts general features, the region decoder outputs the bounding box (bbox) of the auricle contour based on the general features, and the keypoint decoder outputs the coordinate prediction values ​​of multiple keypoints. In this embodiment, the coordinate prediction values ​​of 10 keypoints are output. In the actual training process, a large number of auricle image samples labeled with actual keypoints are prepared, and these samples are divided into training set, validation set, and test set. During training, the target auricle image samples and auricle cropped image samples are input into the shared encoder. After the shared encoder extracts general features, they are input into the region decoder and keypoint decoder respectively for synchronous training. During the training process, the difference between the model prediction result and the actual result is calculated according to the target loss function, and the model parameters are continuously updated and iterated, such as adjusting the weights and biases in the neural network, so that the target loss function gradually decreases. For example, when the validation set loss no longer decreases after 5 consecutive epochs, an adaptive learning rate adjustment strategy is adopted, multiplying the learning rate by 0.1, and training continues until the convergence condition is met, resulting in a trained target detection model. In practical applications, the target external auricle image is input into the trained model, the region decoder outputs the external auricle contour region, and the key point decoder outputs the predicted key points on the external auricle.

[0041] Specifically, step S102 trains the object detection model through the following steps: Step 1021: Obtain an image sample of the outer ear. The image sample of the outer ear is marked with actual key points.

[0042] Here, professional image acquisition equipment, such as high-resolution cameras, can be used to collect a large amount of external auricle image data from different individuals, including images of normal external auricles and images of various types of external auricle deformities. The actual key points are 10 manually labeled morphological key points, such as... Figure 3 As shown, the actual key points on the outer ear include: supraauricular point 30, subauricular point 31, antitragus point 32, the intersection of the line connecting the supraauricular point and the subauricular point and the lower crura of the antihelix 33, the outermost point of the auricle 34, the preauricular point 35, the intersection of the lower crura of the antihelix and the helix 36, the supratragus notch 37, the tragus point 38, and the intertragus notch 39; among them, the supraauricular point 30, the subauricular point 31, the antitragus point 32, the intersection of the line connecting the supraauricular point and the subauricular point and the lower crura of the antihelix 33 are collinear, and the intersection of the lower crura of the antihelix and the helix 36, the supratragus notch 37, the tragus point 38, and the intertragus notch 39 are collinear.

[0043] Among them, point 30 on the upper part of the ear refers to the highest point of the upper edge of the auricle, which is the top of the auricle in the vertical direction; point 31 on the lower part of the ear refers to the lowest point of the lower edge of the auricle (if there is an earlobe, it is usually the midpoint of the bottom of the earlobe; if there is no earlobe, it is the bottom of the lower edge of the auricle); point 32 on the antitragus refers to the most prominent point of the antitragus, which is a small bulge in front of the auricle, located on the opposite side of the tragus (the protrusion in front of the ear canal), and the two are separated by the intertragic notch; point 34 on the outermost part of the auricle refers to the point on the auricle that is furthest from the midline of the head in the horizontal direction, that is, the point where the auricle protrudes to the side. The highest point; the preauricular point 35 refers to the junction between the front of the auricle and the facial skin, located in the depression or skin fold in front of the tragus, usually corresponding to the skin point in front of the ear canal opening; the supratragus notch 37 refers to the depression above the tragus and antitragus, located between the upper edges of the tragus and antitragus; the tragus point 38 refers to the most anterior point of the tragus, that is, the highest point of the tragus protruding forward towards the face; the intertragus notch 39 refers to the depression below the tragus and antitragus, located between the bottom of the tragus and antitragus, and is a distinct groove above the earlobe.

[0044] For the intersection point 33 between the line connecting the upper ear point and the lower ear point and the lower leg of the antihelix, first determine the upper ear point and the lower ear point, and connect the two points to form a straight line perpendicular to the side of the head; the lower leg of the antihelix is ​​a branch extending downward from the antihelix (the arc-shaped ridge in the middle of the auricle), connecting the antihelix and the upper part of the earlobe; the intersection point of this line and the lower leg of the antihelix is ​​the intersection point.

[0045] Regarding the intersection point 36 between the lower crus of the antihelix and the helix, the lower crus of the antihelix is ​​a downward branch of the antihelix, and the helix is ​​an arc-shaped ridge on the edge of the auricle (encircling the outer side of the auricle); the point where the lower crus of the antihelix extends upward to the helix is ​​the intersection point.

[0046] For example, high-resolution cameras can be used to acquire images of the external auricle of 204 different individuals. Clinicians can then use professional annotation tools to annotate 10 key points in each image, ensuring sub-pixel accuracy. For instance, when annotating points on the upper ear, the most prominent point at the top of the auricle is used as a reference, and its coordinates are recorded using mouse clicks or coordinate input.

[0047] In addition, the external auricle image samples can also include images under different lighting conditions (such as natural light, surgical shadowless lamp) and different shooting angles (anteroposterior and lateral ±30°) to enhance the model's generalization ability.

[0048] Step 1022: Preprocess the external auricle image sample to obtain the target external auricle image sample.

[0049] Specifically, step 1022 includes: performing data enhancement on the external auricle image sample to obtain a target external auricle image sample; the data enhancement includes at least one of the following: random color perturbation, random contrast perturbation, random brightness perturbation, random motion blur, random rotation, random flipping, and simulated surgical occlusion; and adjusting the target external auricle image sample to a first fixed size.

[0050] In this step, the implementation method of preprocessing can be referred to the explanation of preprocessing in step S101, and will not be repeated here.

[0051] For example, random brightness perturbations (brightness variation range ±20%) and random motion blur (kernel size 3×3) are applied to external auricle image samples to simulate lighting fluctuations and hand tremors during clinical photography; the image size is uniformly adjusted to 512×512 pixels for easy model input. Surgical occlusion can also be simulated by using rectangular masks to cover 10%-30% of the image area to train the model's robustness to partial occlusion.

[0052] It should be noted that the target auricle image samples can be divided into training set, validation set, and test set, with the training set accounting for 70%, the validation set accounting for 15%, and the test set accounting for 15%. The training set provides learning materials for the target detection model, the validation set assists in optimizing the training of the target detection model, and the test set evaluates the final performance of the target detection model, together ensuring that the auricle target detection model can operate accurately and reliably.

[0053] Step 1023: Using the outer ear region as the cropping area, crop the target outer ear image sample to obtain the cropped outer ear image sample.

[0054] Here, step 1023 specifically includes: locating the auricular region of the target auricular image sample based on an image processing algorithm; cropping the located auricular region to obtain a cropped auricular image sample; adjusting the cropped auricular image sample to a second fixed size; and the first fixed size being larger than the second fixed size.

[0055] The localization of the outer ear region can be based on traditional image processing algorithms, such as edge detection combined with morphological operations. The cropped image size can be 256×256 pixels, smaller than the original image size of 512×512 pixels. For example, a traditional image processing algorithm uses Canny edge detection to extract the outer ear contour. Specifically, the outer ear contour is first extracted using Canny edge detection, and then morphological operations such as dilation and erosion are used to fill the gaps to obtain a complete contour region. Based on this, a 256×256 pixel cropped image is obtained. For example, the located outer ear region in the original image is (150, 180, 350, 380), and after cropping, it is adjusted to 256×256 pixels, focusing on the details of the ear. Optionally, the cropping region can be dynamically determined based on the coordinates of key points, for example, using the smallest bounding rectangle of all key points as the cropping box to ensure that the cropped image includes all key points.

[0056] Step 1024: Construct a target detection model that includes a shared encoder, a region decoder, and a keypoint decoder.

[0057] In this step, the shared encoder can adopt the HRNet architecture, transfer the pre-trained weights of face landmark detection, the region decoder can be the RTMDetSepBNHead module, and the landmark decoder can be the HeatmapHead module.

[0058] Specifically, the pre-trained weights of HRNet come from the keypoint detection task of a large-scale face dataset and can be directly used for external ear feature extraction; the region decoder receives the general features of the encoder and outputs the bounding box coordinates of the external ear contour, and the keypoint decoder outputs the keypoint heatmap.

[0059] Step 1025: Input the target auricle image sample and the auricle cropped image sample into the shared encoder to extract general features; input the extracted general features into the region decoder and the keypoint decoder respectively to train the region decoder and the keypoint decoder simultaneously.

[0060] Here, the target auricle image sample is used as the original image (size 512×512) as input for coarse localization of the auricle region. The auricle cropped image sample (size 256×256) is input into the same network, and the sub-pixel accuracy of key points is improved through feature fusion.

[0061] In this implementation, both the target auricle image sample (512×512) and the cropped auricle image sample (256×256) are simultaneously input into a shared encoder to achieve dual-scale input. The shared encoder extracts general features and then feeds them into dual decoders. In an optional implementation, each batch during training contains 50% target auricle image samples and 50% cropped auricle image samples. The target auricle image samples are used for coarse localization of the auricle region, while the cropped auricle image samples are used for fine-tuning keypoint locations. For example, after the target auricle image samples are input, the region decoder learns to predict the bounding box of the complete auricle; after the cropped auricle image samples are input, the keypoint decoder learns more accurate keypoint coordinates within the local region. The two complement each other through the shared encoder.

[0062] Step 1026: Continuously update and iterate the model parameters of the object detection model to gradually reduce the target loss function until the convergence condition is met, and obtain the trained object detection model.

[0063] Specifically, during the continuous updating and iteration of the object detection model parameters, after multiple adjustments to the learning rate, the validation set loss no longer shows a significant decreasing trend (e.g., fluctuating within a certain range, with the fluctuation amplitude less than a set threshold, such as 0.001). At the same time, the number of training rounds reaches the set total number of rounds of 100, or if the number of training rounds has not reached 100, the validation set loss no longer decreases regardless of how the learning rate is adjusted and training continues. At this point, the convergence condition is met, and the object detection model is considered to have completed training.

[0064] For example, using an adaptive learning rate adjustment strategy, when the validation set loss does not decrease for five consecutive epochs, the learning rate decays to 0.1 times its current value, with a total of 100 training epochs. For instance, if the validation set loss stops decreasing in epoch 30, the learning rate is adjusted from 0.001 to 0.0001, and training continues until convergence. Additionally, the batch size can be set to 32, and the total number of training epochs can be set to 100, ultimately resulting in a model with a keypoint error of less than 0.04 mm on the test set.

[0065] Optionally, the target loss function includes a collinearity loss term, which is determined based on the location information of key points on the outer ear that have a collinear relationship.

[0066] Specifically, for four key points that are collinear, the modulus of any combination formed by three of these collinear key points is calculated, and the modulus of all calculated combinations is then used. The average of the results yields the value corresponding to the collinearity loss term.

[0067] Furthermore, the target loss function also includes a distance loss term, which is determined based on the predicted location information of the key points and the actual location information of the actual key points.

[0068] Here, the distance loss term is the mean square error (MSE) between the predicted keypoints and the actual keypoints.

[0069] Furthermore, the target loss function comprises a weighted sum of collinearity loss terms and distance loss terms. Specifically, the target loss function is calculated using the following formula:

[0070] in, Represents the target loss function. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the collinearity loss term. This represents the distance loss term. For example, the first weighting coefficient... =0.4, second weighting coefficient =0.6.

[0071] The collinearity loss term is calculated using the following formula:

[0072] in, The term denotes the collinearity loss term, and det represents the modulus of the determinant. The first key point among the key points indicating a collinear relationship. The second key point among the key points indicating a collinear relationship. The third key point among the key points indicating a collinear relationship. The fourth key point among the key points indicating a collinear relationship.

[0073] Here, if the three points are strictly collinear, the determinant value is 0; if a key point deviates from the straight line, the determinant modulus increases.

[0074] The collinearity loss term constrains the positional relationships between collinear keypoints, enabling the collinear points predicted by the object detection model during training to better satisfy the collinearity condition, thus achieving convergence in keypoint prediction due to collinearity constraints. Furthermore, combining the collinearity loss term with the distance loss term to form the objective loss function can comprehensively optimize the model's prediction accuracy for keypoints.

[0075] In step S103, based on the collinear relationship between the predicted key points, the predicted key points are fitted and optimized to obtain the optimal key points.

[0076] Here, collinearity refers to the geometric constraints that a specific set of key points on the outer ear must satisfy, such as the collinear set formed by the supraauricular point, the subauricular point, and the antitragus point.

[0077] Specifically, step S103 includes: inputting the predicted key points that have a collinear relationship into the collinear decoder, and performing collinear fitting processing on the input predicted key points through the collinear decoder to obtain the optimal key points.

[0078] Specifically, the predicted key points that are collinear are input into the collinear decoder, which can use techniques such as least squares line fitting to perform collinear fitting on the input predicted key points.

[0079] For example, for four prediction points that are collinear, the collinear decoder calculates an optimal straight line that minimizes the sum of the squares of the distances from each point to the line, and projects each point onto the line to obtain the optimized coordinates, thereby adjusting and optimizing the position of the key points.

[0080] That is, after the target detection model outputs keypoints, the collinear decoder performs collinear fitting on several sets of keypoints that have a collinear relationship, and then outputs the optimized keypoint positions. In this way, the accuracy of keypoint detection can be further improved, making the detection results more consistent with the actual morphological characteristics of the auricle.

[0081] Furthermore, the accuracy of key points can be continuously improved through multiple fitting optimization processes. For example, after the first fitting, a set of optimized key points is obtained. These points are then input into the collinear decoder for a second fitting, further adjusting the positions of the key points until their positions no longer change significantly or meet certain accuracy requirements, thus obtaining the final optimal key points.

[0082] When the target detection model provided in this application measures key points on an image of the outer ear, the measured key points have an error of 0.04 mm, a collinearity deviation of 0.08°, and a parameter calculation time of 0.5 s. Compared with the key point error of 2.15 mm, collinearity deviation of 1.2°, and parameter calculation time of 3 s for manually labeled key points, this model achieves better results.

[0083] In one optional embodiment, the present application further includes: determining target feature parameters for use in auricular surgery based on optimal key points; the target feature parameters include at least one of the following: the ratio of the helix to the antihelix angle, the proportion of the tragus notch spacing, and the golden ratio of the length and width of the auricle.

[0084] Here, the helix-antihelix angle ratio refers to the ratio of the angle between two curves calculated using the coordinates of the key points of the helix and antihelix; the golden ratio of auricle length and width refers to the ratio of the longest distance of the auricle (such as from the upper ear point to the lower ear point) to the widest distance (from the outermost point of the auricle to the front ear point), with an ideal value of approximately 0.618; the tragus notch spacing ratio refers to the proportion of the distance between the upper tragus notch and the intertragus notch to the total height of the auricle.

[0085] Furthermore, determining the target characteristic parameters for auricular surgery based on optimal key points provides strong support for surgical planning and outcome evaluation. For example, the helix-antihelix angle ratio reflects the morphological relationship of important structures in the auricle. By accurately calculating this angle ratio using optimal key points, doctors can more clearly determine the degree and type of auricular deformity. For instance, in patients with cup ear deformity, the helix-antihelix angle ratio will deviate from the normal range. Accurately obtaining this parameter helps doctors develop targeted correction plans, determine the angles and ranges that need adjustment, and make the surgical operation more targeted and scientific. The golden ratio of auricular length and width embodies the aesthetic standard of the overall shape of the auricle. Accurately calculating this ratio based on optimal key points helps doctors balance functional recovery and aesthetic appearance during surgery, making the postoperative auricular shape closer to normal aesthetic standards and improving patient satisfaction.

[0086] The method provided in this application uses a target detection model including a shared encoder and dual decoders to measure key points. It introduces a collinearity loss term to constrain the collinearity of key points and combines this with fitting optimization to improve the accuracy and robustness of key point detection in the auricle. This solves the problems of traditional algorithms being sensitive to illumination and angle, and having insufficient positioning accuracy, providing accurate morphological data support for medical plastic surgery. Furthermore, it requires no manual operation, has high efficiency, and ensures the accuracy and consistency of measurement results.

[0087] Based on the same inventive concept, this application also provides an external auricle key point detection device corresponding to the external auricle key point detection method. Since the principle of the device in this application is similar to the external auricle key point detection method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0088] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a key point detection device for the auricle provided in an embodiment of this application. Figure 4 As shown, the device 400 includes: Preprocessing module 401 is used to preprocess the image of the external auricle to be detected to obtain the target external auricle image; The keypoint detection module 402 is used to input the target auricle image into a pre-trained target detection model to obtain predicted keypoints on the auricle; wherein, the target detection model includes a shared encoder, a region decoder for outputting the auricle contour region, and a keypoint decoder for outputting keypoints on the auricle; during the training process of the target detection model, the model parameters of the target detection model are adjusted using a target loss function, the target loss function including a collinearity loss term, which is determined based on the location information of keypoints on the auricle that have collinear relationships; The optimization processing module 403 is used to perform fitting optimization processing on the predicted key points based on the collinear relationship between the predicted key points to obtain the optimal key points.

[0089] The device provided in this application embodiment measures key points using a target detection model comprising a shared encoder and dual decoders. It introduces a collinearity loss term to constrain the collinearity of key points and, combined with fitting optimization, improves the accuracy and robustness of key point detection in the auricle. This solves the problems of traditional algorithms being sensitive to illumination and angle, and lacking sufficient positioning accuracy, providing accurate morphological data support for medical plastic surgery. Furthermore, it requires no manual operation, has high operational efficiency, and ensures the accuracy and consistency of measurement results.

[0090] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.

[0091] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 The steps of the external auricle key point detection method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0092] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the external auricle key point detection method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0093] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0095] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0096] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0097] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting key points of an external auditory meatus, characterized in that, The method comprises: preprocessing a to-be-detected external auditory meatus image to obtain a target external auditory meatus image; inputting the target external auditory meatus image into a pre-trained target detection model to obtain predicted key points on the external auditory meatus; wherein the target detection model comprises a shared encoder, a region decoder for outputting an external auditory meatus contour region, and a key point decoder for outputting key points on the external auditory meatus; in the training process of the target detection model, the model parameters of the target detection model are adjusted by using a target loss function, and the target loss function comprises a collinearity loss term determined according to the position information of key points in a collinear relationship on the external auditory meatus; fitting and optimizing the predicted key points according to the collinear relationship between the predicted key points to obtain optimal key points.

2. The method of claim 1, wherein, The target detection model is trained by the following steps: obtaining external auditory meatus image samples, which are labeled with actual key points; preprocessing the external auditory meatus image samples to obtain target external auditory meatus image samples; cropping the target external auditory meatus image samples to obtain external auditory meatus cropped image samples with the external auditory meatus region as the cropping region; constructing a target detection model comprising a shared encoder, a region decoder, and a key point decoder; inputting the target external auditory meatus image samples and the external auditory meatus cropped image samples into the shared encoder to extract general features; inputting the extracted general features into the region decoder and the key point decoder respectively to synchronously train the region decoder and the key point decoder; constantly updating and iterating the model parameters of the target detection model to gradually reduce the target loss function until the convergence condition is met, thereby obtaining the trained target detection model.

3. The method of claim 2, wherein, The target loss function further comprises a distance loss term determined according to the predicted position information of the predicted key points and the actual position information of the actual key points.

4. The method of claim 3, wherein, The target loss function is calculated by the following formula: wherein, represents a target loss function, represents a first weight coefficient, represents a second weight coefficient, represents a collinear loss term, represents a distance loss term.

5. The method of claim 4, wherein, The collinearity loss term is calculated by the following formula: wherein, represents a collinearity loss term, det represents a modulus of a determinant, represents a first key point among key points in which a collinearity exists, represents a second key point among key points in which a collinearity exists, represents a third key point among key points in which a collinearity exists, represents a fourth key point among key points in which a collinearity exists.

6. The method of claim 2, wherein, The preprocessing of the external auditory meatus image samples to obtain target external auditory meatus image samples comprises: performing data enhancement on the external auditory meatus image samples to obtain target external auditory meatus image samples; the data enhancement comprises at least one of the following: random color disturbance, random contrast disturbance, random brightness disturbance, random motion blur, random rotation, random flipping, and simulated surgical occlusion; adjusting the target external auditory meatus image samples to a first fixed size; cropping the target external auditory meatus image samples to obtain external auditory meatus cropped image samples with the external auditory meatus region as the cropping region comprises: positioning the external auditory meatus region of the target external auditory meatus image samples based on an image processing algorithm; cropping the positioned external auditory meatus region to obtain external auditory meatus cropped image samples; adjusting the external auditory meatus cropped image samples to a second fixed size; the first fixed size is greater than the second fixed size.

7. The method of claim 2, wherein, Actual key points on the external ear include: the supratragic point, the infratragic point, the antitragal point, the intersection of the line between the supratragic point and the infratragic point and the antihelical foot, the most external point of the auricle, the preauricular point, the intersection of the antihelical foot and the helix, the superior incisura of the tragus, the tragus point, the intertragal incisure; Wherein, the supratragic point, the infratragic point, the antitragal point, the intersection of the line between the supratragic point and the infratragic point and the antihelical foot have a collinear relationship, and the intersection of the antihelical foot and the helix, the superior incisura of the tragus, the tragus point and the intertragal incisure have a collinear relationship.

8. The method of claim 1, wherein, The method further comprises: Based on the optimal key points, determining a target feature parameter applied to the external ear surgery; the target feature parameter includes at least one of the following: helix-antihelix angle ratio, tragus incisure interval ratio, and auricle length-width golden section ratio.

9. The method of claim 1, wherein, The pre-processing of the to-be-detected external ear image to obtain a target external ear image comprises: Data augmentation is performed on the to-be-detected external ear image to obtain a target external ear image; The data augmentation includes at least one of the following: random color disturbance, random contrast disturbance, random brightness disturbance, random motion blur, random rotation, random flip, and simulated surgical occlusion; The image size of the target external ear image is adjusted to a first fixed size.

10. The method of claim 1, wherein, The fitting optimization processing of the predicted key points according to the collinear relationship between the predicted key points comprises: The predicted key points with the collinear relationship are input into a collinear decoder, and the collinear fitting processing of the input predicted key points is performed by the collinear decoder to obtain the optimal key points.

Images