Construction method of multi-resolution-channel attention network capable of detecting key points of high heterogeneity X-ray lateral head image

Abstract

The application discloses a kind of multi-resolution-channel attention network construction methods for detecting high heterogeneity X-ray head lateral position image key point.First, the feature extraction network consisting of lightweight fusion convolution and channel attention mechanism is constructed, to reduce the computational complexity and inhibit non-important channel features.Then construct multi-resolution progressive feature extraction subnetwork to extract multi-resolution features, enhance the feature multi-scale expression ability.Then construct multi-scale attention guided feature fusion network and the above-mentioned subnetwork multi-point connection, so that the extracted multi-resolution features are fully fused.The prediction network constructed by upsampling is used to align and splice multi-scale features to fuse different levels of information, use independent detection head to process different levels of features, and automatically output the prediction result of key point position in image, and then use multi-level aggregation loss function to improve the model generalization ability.The application can realize the automatic detection of key points in high heterogeneity X-ray head lateral position image in oral clinical.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision prediction, in particular to a multi-resolution-channel attention network construction method capable of detecting key points of high-heterogeneity X-ray lateral cephalogram images. BACKGROUND

[0002] Measurement and analysis of lateral cephalogram images are key steps in modern orthodontic research on craniofacial growth and development, malformation diagnosis, and development of correction programs. The analysis results will directly affect the judgment of the overall situation of the patient by the doctor. When analyzing lateral cephalogram images, the doctor needs to mark the key points of anatomical structures such as teeth and jaws, craniofacial, etc. on the image, and quantitatively and qualitatively analyze the distance and angle relationship between these key points. These analysis results will provide an important basis for developing diagnosis and surgery programs. Therefore, the accuracy of key point coordinate positioning is directly related to the effect of diagnosis and treatment.

[0003] At present, most doctors in the clinic still adopt the manual positioning of key points. The speed and accuracy of this method depend largely on the professional level and physical condition of the doctor, and may lead to diagnosis errors or even surgery failure due to positioning deviation. Therefore, it is particularly urgent to develop a key point automatic detection method for high-heterogeneity X-ray lateral cephalogram images to assist doctors in accurately positioning the key point position. This not only helps to improve the positioning accuracy, but also significantly improves the clinical diagnosis efficiency.

[0004] In recent years, deep learning has shown excellent performance in image target detection and recognition, and some scholars have tried to introduce deep learning models into the field of orthodontics to promote the development of key point automatic detection technology.

[0005] Yang et al. proposed a U-shaped network CETransNet based on convolution enhanced Transformer for positioning key points of lateral cephalogram images. The network retains the ability of convolutional neural network to obtain local information while establishing global context connection, and introduces an exponential weighted loss function to further improve the positioning accuracy by suppressing the loss of distant pixels. Experimental results show that this method has certain clinical application value.

[0006] Wang et al. proposed a high-resolution network HRNet, which gradually introduces multi-scale feature fusion while maintaining high-resolution features, and performs well in key point detection and human pose estimation tasks. Its superior multi-scale processing capability makes it particularly suitable for clinical application scenarios that require high-precision key point positioning.

[0007] In summary, in the field of medical image analysis and facial feature key point detection, high-precision key point positioning is of great significance. Although traditional HRNet performs well in maintaining high-resolution features, it still has the following problems:

[0008] 1. Low feature extraction efficiency and high computational resource consumption;

[0009] 2. Lack of effective attention mechanism, making it difficult to highlight key features;

[0010] 3. Single feature fusion method, not fully utilizing multi-scale information;

[0011] 4. All images processed are of high consistency, and high-heterogeneous image detection has not been achieved.

[0012] To address these shortcomings, a multi-resolution-channel attention network construction method for detecting key points of high-heterogeneous X-ray skull lateral images is proposed. By introducing a channel attention mechanism module, the features of important channels are highlighted, and the interference of irrelevant channels is suppressed. The design of lightweight fusion convolution reduces the number of model parameters and the computational complexity while maintaining the effectiveness of feature extraction. In the multi-scale attention guided feature fusion network, the features extracted by the lightweight fusion feature extraction network and the multi-resolution progressive feature extraction subnetwork are repeatedly fused to achieve more effective multi-scale feature expression. The introduction of a multi-level aggregation loss function in the prediction network improves the generalization ability of the model. The optimized network structure and efficient feature extraction method effectively improve the processing speed of the model, enabling it to complete the real-time key point detection task of skull lateral images more quickly, thereby providing accurate and efficient assistance for doctors in oral and maxillofacial surgery to accurately locate the key point positions of high-heterogeneous skull lateral images. SUMMARY

[0013] To solve the above technical problems, the present application provides a multi-resolution-channel attention network construction method for detecting key points of high-heterogeneous X-ray skull lateral images, comprising the following steps:

[0014] Step 1: Construct a lightweight fusion feature extraction network: the lightweight fusion feature extraction network is composed of a number of lightweight fusion convolution-batch normalization-ReLU activation function combination layers, a number of bottleneck units connected in order, and finally a channel attention mechanism module.

[0015] Step 2: Construct a multi-resolution progressive feature extraction subnetwork: the multi-resolution progressive feature extraction subnetwork is based on a simplified feature pyramid network as the basic structure, which extracts multi-resolution features from the original input image to construct features of different resolutions, and fuses them through a skip connection in the main part of the network to achieve more effective multi-scale feature expression.

[0016] Step 3: Constructing a multi-scale attention-guided feature fusion network: the multi-scale attention-guided feature fusion network is composed of a plurality of convolution-batch normalization-ReLU activation function combination layers and a plurality of multi-scale attention-guided feature fusion sub-networks connected in sequence;

[0017] Step 4: Constructing a prediction network: the prediction network aligns the multi-scale features obtained by the multi-scale attention-guided feature fusion network to the same scale through upsampling, and then splices them to fuse information at different levels; then the features before splicing and the features after splicing are processed through independent detection heads, respectively, to process features at different levels and generate prediction results, and the generalization ability of the model is supervised and improved through a multi-level aggregation loss function;

[0018] Step 5: Network training and prediction: using the constructed multi-resolution-channel attention key point detection network to train the labeled head lateral image; input the newly collected head lateral image to be detected into the trained network, automatically predict the key point position in the image, and output the error between the prediction result and the manual annotation.

[0019] The further defined technical solutions of the application are:

[0020] The multi-resolution-channel attention key point detection network construction method described above, in step 1, the lightweight fusion feature extraction network includes a lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR), a bottleneck unit, and a channel attention mechanism module:

[0021] First, the input image is subjected to a lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR) to complete preliminary feature extraction, and the extracted feature size is HxW; then subjected to a second lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR) to perform feature extraction while completing downsampling operation, and the feature size becomes H / 2xW / 2;

[0022] The obtained feature is subjected to four cascaded bottleneck units and then subjected to a channel attention mechanism module.

[0023] The lightweight fusion feature extraction network described above, in the lightweight fusion convolution, the input feature is subjected to a lightweight convolution and then subjected to a standard convolution to perform splicing operation, and the splicing result is subjected to ReLU activation function to obtain the output of the lightweight fusion convolution.

[0024] In the aforementioned lightweight fusion feature extraction module, the bottleneck unit sequentially passes the input features through two convolution-batch normalization-activation function combination layers (CBR), one convolution, and one batch normalization combination. Then, it performs an element-by-element addition operation with the input features, and the result of the addition is passed through the ReLU activation function to obtain the output of the bottleneck unit.

[0025] In the aforementioned lightweight fusion feature extraction module, the channel attention mechanism module captures global information at the channel level, multiplies the obtained weights with the original features, and achieves adaptive feature recalibration at the channel level. This highlights the features of important channels, suppresses interference from irrelevant channels, and significantly improves the model's ability to capture detailed features in skull X-ray images. The specific process is as follows:

[0026] First, for input features of size B×C×H×W, the spatial dimension H×W is compressed into a global channel feature vector using global average pooling, resulting in a compressed size of B×C. Then, a two-layer fully connected network is used to generate weight values ​​for each channel: the first layer reduces the number of channels from C to C / 16, reducing dimensionality to decrease the number of parameters and computational cost while learning compressed features between channels, and then uses the ReLU activation function to introduce non-linearity; the second layer restores the original number of channels C, while generating a weight value for each channel, which is then normalized to [0,1] using the Sigmoid activation function; the generated channel weights are multiplied with the input features channel by channel to enhance or suppress the channel features.

[0027] In the aforementioned multi-resolution-channel attention keypoint detection network construction method, step 2 includes a multi-resolution progressive feature extraction sub-network comprising a convolutional-batch normalization-ReLU activation function combination layer (CBR), a convolutional-bilinear interpolation upsampling combination layer, and a weighted fusion module.

[0028] First, the input image has dimensions H×W. After passing through a CBR module, feature X1 is obtained, with a feature size of H / 2×W / 2. X1 passes through a CBR module to obtain feature X2, with a feature size of H / 4×W / 4. X2 passes through a CBR module to obtain feature X3, with a feature size of H / 8×W / 8. X3 passes through a CBR module to obtain feature X4, with a feature size of H / 16×W / 16.

[0029] Feature X2 is weighted and fused with feature X1 after passing through an upsampling combination layer to obtain feature Y1; feature X3 is weighted and fused with feature X2 after passing through an upsampling combination layer to obtain feature Y2; feature X4 is weighted and fused with feature X3 after passing through an upsampling combination layer to obtain feature Y3; feature Y4 is directly obtained from feature X4.

[0030] The finally output features are Y1, Y2, Y3 and Y4, wherein the feature size of Y1 is H / 2×W / 2, the feature size of Y2 is H / 4×W / 4, the feature size of Y3 is H / 8×W / 8, and the feature size of Y4 is H / 16×W / 16.

[0031] The method for constructing the multi-resolution-channel attention key point detection network is as follows: in step 3, the multi-scale attention guided feature fusion network comprises a convolution-batch normalization-rectified linear unit (CBR) combination layer, a first multi-scale attention guided feature fusion subnetwork, a second multi-scale attention guided feature fusion subnetwork and a third multi-scale attention guided feature fusion subnetwork.

[0032] Firstly, the input feature passes through a CBR module to maintain the original resolution branch and change the current channel number to obtain feature A1; the input feature passes through a CBR module to downsample to generate a new resolution branch to obtain feature A2; A1 and A2 pass through the first multi-scale attention guided feature fusion subnetwork to obtain output features D1 and D2; wherein the feature sizes of the input feature, feature A1 and feature D1 are H / 2×W / 2; the feature sizes of feature A2 and feature D2 are H / 4×W / 4.

[0033] The output features D1 and D2 pass through a CBR module respectively to obtain features A3 and A4; D2 passes through a CBR module to downsample to generate a new resolution branch to obtain feature A5; A3, A4 and A5 pass through the second multi-scale attention guided feature fusion subnetwork to obtain output features D3, D4 and D5; wherein the feature sizes of feature A3 and feature D3 are H / 2×W / 2; the feature sizes of feature A4 and feature D4 are H / 4×W / 4; the feature sizes of feature A5 and feature D5 are H / 8×W / 8.

[0034] The output features D3, D4 and D5 pass through a CBR module respectively to obtain features A6, A7 and A8; D5 passes through a CBR module to downsample to generate a new resolution branch to obtain feature A9; A6, A7, A8 and A9 pass through the third multi-scale attention guided feature fusion subnetwork to obtain output features D6, D7, D8 and D9; wherein the feature sizes of feature A6 and feature D6 are H / 2×W / 2, the feature sizes of feature A7 and feature D7 are H / 4×W / 4, the feature sizes of feature A8 and feature D8 are H / 8×W / 8, and the feature sizes of feature A9 and feature D9 are H / 16×W / 16.

[0035] In the multi-scale attention guided feature fusion network, the multi-scale attention guided feature fusion sub-network one comprises a basic unit, a convolution-bilinear interpolation upsampling combination layer, a convolution-batch normalization combination layer, an element-wise addition module, a weighted fusion module, and a channel attention mechanism module.

[0036] In the multi-scale attention guided feature fusion sub-network one, the input features are A1 and A2; the feature A1 passes through four cascaded basic units to obtain a feature B1; the feature A2 passes through four cascaded basic units to obtain a feature B2; wherein the feature size of the feature B1 is H / 2×W / 2, and the feature size of the feature B2 is H / 4×W / 4;

[0037] The feature B1 is added element-wise with the B2 passing through the upsampling combination layer to obtain a feature C1; the feature B2 is added element-wise with the B1 passing through the downsampling combination layer 1 to obtain a feature C2; the feature size of the feature C1 is H / 2×W / 2, and the feature size of the feature C2 is H / 4×W / 4;

[0038] The feature C1 passes through a ReLU activation function, is weightedly fused with the Y1 output in step 2, and then passes through a channel attention mechanism to obtain a feature D1; the feature C2 passes through a ReLU activation function, is weightedly fused with the Y2 output in step 2, and then passes through a channel attention mechanism to obtain a feature D2.

[0039] In the multi-scale attention guided feature fusion network, the multi-scale attention guided feature fusion sub-network two comprises a convolution-bilinear interpolation upsampling combination layer, a convolution-batch normalization combination layer, a CBR-convolution-batch normalization combination layer, an element-wise addition module, a weighted fusion module, and a channel attention mechanism module.

[0040] In the multi-scale attention guided feature fusion sub-network two, the input features are A3, A4, and A5; the feature A3 passes through eight cascaded basic units to obtain a feature B3; the feature A4 passes through eight cascaded basic units to obtain a feature B4; the feature A5 passes through eight cascaded basic units to obtain a feature B5; the feature size of the feature B3 is H / 2×W / 2, the feature size of the feature B4 is H / 4×W / 4, and the feature size of the feature B5 is H / 8×W / 8;

[0041] The feature B3 is added element-wise with the B4 and B5 passing through the upsampling combination layer in sequence to obtain a feature C3; the feature B4 is added element-wise with the B3 passing through the downsampling combination layer 1 and the B5 passing through the upsampling combination layer in sequence to obtain a feature C4; the feature B5 is added element-wise with the B3 passing through the downsampling combination layer 2 and the B4 passing through the downsampling combination layer 1 in sequence to obtain a feature C5; the feature size of the feature C3 is H / 2×W / 2, the feature size of the feature C4 is H / 4×W / 4, and the feature size of the feature C5 is H / 8×W / 8;

[0042] The feature C3 is fused with Y1 output by step 2 after a ReLU activation function, and then a channel attention mechanism is performed to obtain a feature D3; the feature C4 is fused with Y2 output by step 2 after a ReLU activation function, and then a channel attention mechanism is performed to obtain a feature D4; and the feature C5 is fused with Y3 output by step 2 after a ReLU activation function, and then a channel attention mechanism is performed to obtain a feature D5.

[0043] In the multi-scale attention guided feature fusion network, the multi-scale attention guided feature fusion sub-network three includes a convolution-bilinear interpolation upsampling combination layer, a convolution-batch normalization combination layer, a CBR-convolution-batch normalization combination layer, a CBR-CBR-convolution-batch normalization combination layer, an element-wise addition module, a weighted fusion module, and a channel attention mechanism module.

[0044] In the multi-scale attention guided feature fusion sub-network three, the input features are A6, A7, A8, and A9; the feature A6 is input into eight cascaded basic units to obtain a feature B6; the feature A7 is input into eight cascaded basic units to obtain a feature B7; the feature A8 is input into eight cascaded basic units to obtain a feature B8; and the feature A9 is input into eight cascaded basic units to obtain a feature B9; the feature size of the feature B6 is H / 2×W / 2, the feature size of the feature B7 is H / 4×W / 4, the feature size of the feature B8 is H / 8×W / 8, and the feature size of the feature B9 is H / 16×W / 16.

[0045] The feature B6 is element-wise added with B7, B8, and B9 that have passed through the upsampling combination layer to obtain a feature C6; the feature B7 is element-wise added with B6 that has passed through the downsampling combination layer 1, B8 that has passed through the upsampling combination layer, and B9 to obtain a feature C7; the feature B8 is element-wise added with B6 that has passed through the downsampling combination layer 2, B7 that has passed through the downsampling combination layer 1, and B9 that has passed through the upsampling combination layer to obtain a feature C8; and the feature B9 is element-wise added with B6 that has passed through the downsampling combination layer 3, B7 that has passed through the downsampling combination layer 2, and B8 that has passed through the downsampling combination layer 1 to obtain a feature C9; the feature size of the feature C6 is H / 2×W / 2, the feature size of the feature C7 is H / 4×W / 4, the feature size of the feature C8 is H / 8×W / 8, and the feature size of the feature C9 is H / 16×W / 16.

[0046] Feature C6 is fused with Y1 output in step 2 after passing through a ReLU activation function, and then passes through a channel attention mechanism to obtain feature D6; feature C7 is fused with Y2 output in step 2 after passing through a ReLU activation function, and then passes through a channel attention mechanism to obtain feature D7; feature C8 is fused with Y3 output in step 2 after passing through a ReLU activation function, and then passes through a channel attention mechanism to obtain feature D8; feature C9 is fused with Y4 output in step 2 after passing through a ReLU activation function, and then passes through a channel attention mechanism to obtain feature D9.

[0047] In the aforementioned basic unit, the input feature sequentially passes through a combination of a CBR, a convolution, and a batch normalization, and then is subjected to an element-by-element addition operation with the input feature, and the addition result is subjected to a ReLU activation function to obtain the output of the basic unit.

[0048] In the aforementioned multi-resolution-channel attention key point detection network construction method, in step 4, the prediction network includes a bilinear interpolation upsampling module, a splicing module, and a detection head module.

[0049] First, the multi-scale attention guided feature fusion network obtains multi-scale features aligned to the same scale H / 2xW / 2 through bilinear interpolation upsampling, and the obtained features are Z1, Z2, Z3, and Z4, which are subjected to four independent detection heads to obtain four prediction results; the features Z1, Z2, Z3, and Z4 are spliced along the channel dimension to obtain a feature Z0, which is subjected to a detection head to obtain a prediction result.

[0050] The five prediction results are subjected to loss calculation, and the five losses are fused through a multi-level aggregation loss function, and the model's feature learning ability is improved through multi-scale supervision, thereby providing a stronger supervision signal, optimizing the feature learning process, and enhancing the stability of the detection result.

[0051] In the aforementioned prediction network, in the detection head module, the input feature sequentially passes through a combination of a CBR and a convolution to obtain the output of the detection head.

[0052] In the aforementioned multi-resolution-channel attention key point detection network construction method, in step 5, the network prediction part outputs the error between the predicted result and the manually labeled result while predicting the key point coordinate position in the output image, thereby obtaining the qualified rate of the test data under different evaluation indicators, so as to better observe the accuracy of the prediction result.

[0053] Compared with the prior art, the technical effects of the present application are: first, the present application inputs the newly collected head lateral image into the trained multi-resolution-channel attention key point detection network, and in the test process, only one scanning of the whole head lateral image is needed, that is, the key point position in the image can be quickly and automatically detected, and the error and qualified rate evaluation index between the prediction result and the artificial annotation are given. Secondly, in the present application, the channel attention mechanism module and the lightweight fusion convolution are used to adaptively adjust the weight of each channel to highlight the features of important channels while reducing the network complexity and parameter amount. Thirdly, in the present application, the multi-scale attention guided feature fusion network is used to repeatedly fuse the features extracted by the lightweight fusion feature extraction network and the multi-resolution progressive feature extraction subnetwork, so as to realize more effective multi-scale feature expression. In addition, in the prediction network in the present application, a multi-level aggregation loss function is introduced to supervise and improve the generalization ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 The flowchart of the embodiment steps of the present application is shown in the figure;

[0055] Figure 2 The overall structure diagram of the network model of the present application is shown in the figure;

[0056] Figure 3 The partial basic structure diagram of the network model of the present application is shown in the figure;

[0057] Figure 4 The multi-resolution progressive feature extraction subnetwork diagram of the network model of the present application is shown in the figure;

[0058] Figure 5 The multi-scale attention guided feature fusion subnetwork diagram of the network model of the present application is shown in the figure;

[0059] Figure 6 The result diagram of detecting the key points of the head lateral image in the embodiment of the present application is shown in the figure; DETAILED DESCRIPTION

[0060] The present embodiment provides a multi-resolution-channel attention network construction method capable of detecting key points of X-ray head lateral images with high heterogeneity, as shown in the figure, comprising the following steps: Figure 1

[0061] Step 1: data acquisition and processing. Collect the head lateral image, and mark the position coordinate information of 12 key points in the image from the head lateral image. Perform data enhancement processing on the head lateral image to generate diversified training data to improve the generalization ability of the model. Integrate the processed image data to form a training set for model training.

[0062] Data acquisition and processing includes the following two parts:​

[0063] (1) Label the position coordinate information of 12 key feature points from the lateral cephalogram image. The specific steps are as follows: first, label in the physical coordinate system, and set the measurement unit as millimeter; then, convert the position coordinates labeled in the physical coordinate system into corresponding coordinates in the pixel coordinate system by using a predetermined conversion algorithm, so as to complete the mapping from the physical space to the pixel space.

[0064] (2) Use data enhancement technology for the lateral cephalogram image used for training. The specific steps are as follows: through the diversified processing of random scaling, rotation and the like on the lateral cephalogram image, the model can better adapt to different angles and background changes, at the same time, the defects of less image data are made up, and the robustness and performance of the detection method are improved.

[0065] Step 2: Construct a multi-resolution-channel attention key point detection network model. As shown in Figure 2 , the model comprises a lightweight fusion feature extraction network, a multi-resolution progressive feature extraction subnetwork, a multi-scale attention guided feature fusion network and a prediction network.

[0066] The lightweight fusion feature extraction network is composed of a plurality of lightweight fusion convolution-batch normalization-activation function combination layers and a plurality of bottleneck units connected in order, and finally composed of a channel attention mechanism module; the multi-resolution progressive feature extraction subnetwork extracts multi-resolution features, and fuses through the way of skip connection in the main part of the network, so as to realize more effective multi-scale feature expression; the multi-scale attention guided feature fusion network is composed of a plurality of convolution-batch normalization-activation function combination layers and a plurality of multi-scale attention guided feature fusion subnetworks connected in order, the convolution combination layer transmits the features of the original branch and generates a new resolution branch, and the multi-scale attention guided feature fusion subnetwork repeatedly fuses the information of all branches; the prediction network aligns the multi-scale features to the same scale through up-sampling and performs splicing to fuse the information of different levels, and the features before splicing and the features after splicing are processed through independent detection heads, respectively, to process different levels of features and generate prediction results, and the generalization ability of the model is supervised and improved through a multi-level aggregation loss function.

[0067] As shown in Figure 2 , the lightweight fusion feature extraction network comprises a lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR), a bottleneck unit and a channel attention mechanism module:

[0068] Firstly, the input image is processed by a light-weight fusion convolution-batch normalization-ReLU activation function combination layer (LBR) to complete preliminary feature extraction, and the extracted feature size is HxW; then the feature is processed by a second light-weight fusion convolution-batch normalization-ReLU activation function combination layer (LBR) to complete feature extraction and down-sampling operation, and the feature size becomes H / 2xW / 2;

[0069] The obtained feature is processed by four cascaded bottleneck units, and then processed by a channel attention mechanism module.

[0070] As shown in Figure 3 , in the light-weight fusion convolution of the light-weight fusion feature extraction network, the input feature is processed by a light-weight convolution, and then spliced with the feature processed by a standard convolution. The spliced result is processed by a ReLU activation function to obtain the output of the light-weight fusion convolution.

[0071] As shown in Figure 3 , in the bottleneck unit of the light-weight fusion feature extraction network, the input feature is sequentially processed by two convolution-batch normalization-activation function combination layers (CBRs), a convolution, and a batch normalization combination, and then element-wise added with the input feature. The added result is processed by a ReLU activation function to obtain the output of the bottleneck unit.

[0072] The channel attention mechanism module captures global information at the channel level, multiplies the obtained weight with the original feature, realizes adaptive feature re-labeling at the channel level, highlights the features of important channels, suppresses the interference of irrelevant channels, and significantly improves the model's ability to capture detailed features in the head X-ray image. The specific process is as follows:

[0073] Firstly, for the input feature with a size of BxCxHxW, the spatial dimension HxW information is compressed into a global channel feature vector by global average pooling, and the size after compression is BxC; then a two-layer fully connected network is used to generate the weight value of each channel: the first layer reduces the channel number from C to C / 16 to reduce the parameter quantity and calculation quantity and learn the compressed features between channels, and then uses a ReLU activation function to introduce non-linear ability; the second layer restores to the original channel number C, and generates a weight value for each channel, and then normalizes to [0, 1] through a Sigmoid activation function; the generated channel weight is multiplied with the input feature channel by channel to complete the enhancement or suppression of the channel features.

[0074] As shown in Figure 4 , the multi-resolution progressive feature extraction sub-network includes a convolution-batch normalization-ReLU activation function combination layer (CBR), a convolution-bilinear interpolation up-sampling combination layer, and a weighted fusion module:

[0075] Firstly, the input image dimension is HxW, and the feature X1 is obtained after a CBR module, and the feature size is H / 2xW / 2; the feature X2 is obtained after a CBR module, and the feature size is H / 4xW / 4; the feature X3 is obtained after a CBR module, and the feature size is H / 8xW / 8; the feature X4 is obtained after a CBR module, and the feature size is H / 16xW / 16;

[0076] The feature X2 is fused with the feature X1 after the up-sampling combination layer to obtain the feature Y1; the feature X3 is fused with the feature X2 after the up-sampling combination layer to obtain the feature Y2; the feature X4 is fused with the feature X3 after the up-sampling combination layer to obtain the feature Y3; and the feature Y4 is directly obtained from the feature X4;

[0077] Finally, the output features are Y1, Y2, Y3 and Y4, wherein the feature size of Y1 is H / 2xW / 2, the feature size of Y2 is H / 4xW / 4, the feature size of Y3 is H / 8xW / 8, and the feature size of Y4 is H / 16xW / 16.

[0078] As shown in Figure 2 , the multi-scale attention guided feature fusion network includes a convolution-batch normalization-ReLU activation function combination layer (CBR), a multi-scale attention guided feature fusion sub-network one, a multi-scale attention guided feature fusion sub-network two, and a multi-scale attention guided feature fusion sub-network three:

[0079] Firstly, the input feature passes through a CBR module, maintains the original resolution branch, and changes the current channel number to obtain the feature A1; the input feature passes through a CBR module for down-sampling to generate a new resolution branch to obtain the feature A2; A1 and A2 pass through the multi-scale attention guided feature fusion sub-network one to obtain the output features D1 and D2; wherein the feature sizes of the input feature, the feature A1 and the feature D1 are H / 2xW / 2; the feature sizes of the feature A2 and the feature D2 are H / 4xW / 4;

[0080] The output features D1 and D2 pass through a CBR module respectively to obtain the features A3 and A4; D2 passes through a CBR module for down-sampling to generate a new resolution branch to obtain the feature A5; A3, A4 and A5 pass through four cascaded multi-scale attention guided feature fusion sub-network two to obtain the output features D3, D4 and D5; wherein the feature sizes of the feature A3 and the feature D3 are H / 2xW / 2; the feature sizes of the feature A4 and the feature D4 are H / 4xW / 4; and the feature sizes of the feature A5 and the feature D5 are H / 8xW / 8;

[0081] The output features D3, D4 and D5 pass through a CBR module respectively to obtain features A6, A7 and A8; D5 passes through a CBR module to down-sample to generate a new resolution branch to obtain feature A9; A6, A7, A8 and A9 pass through three cascaded multi-scale attention guided feature fusion sub-networks three to obtain output features D6, D7, D8 and D9; wherein the feature size of feature A6 and feature D6 is H / 2×W / 2, the feature size of feature A7 and feature D7 is H / 4×W / 4, the feature size of feature A8 and feature D8 is H / 8×W / 8, and the feature size of feature A9 and feature D9 is H / 16×W / 16.

[0082] As shown in Figure 5 , the multi-scale attention guided feature fusion sub-network one includes a basic unit, a convolution-bilinear interpolation up-sampling combination layer, a convolution-batch normalization combination layer, an element-wise addition module, a weighted fusion module and a channel attention mechanism module:

[0083] In the multi-scale attention guided feature fusion sub-network one, the input features are A1 and A2; A1 passes through four cascaded basic units to obtain feature B1; A2 passes through four cascaded basic units to obtain feature B2; wherein the feature size of feature B1 is H / 2×W / 2, and the feature size of feature B2 is H / 4×W / 4.

[0084] Feature B1 and B2 passing through the up-sampling combination layer are added element by element to obtain feature C1; feature B2 and B1 passing through the down-sampling combination layer 1 are added element by element to obtain feature C2; the feature size of feature C1 is H / 2×W / 2, and the feature size of feature C2 is H / 4×W / 4.

[0085] After feature C1 passes through the ReLU activation function, it is weightedly fused with Y1 output in step 2, and then passes through the channel attention mechanism to obtain feature D1; after feature C2 passes through the ReLU activation function, it is weightedly fused with Y2 output in step 2, and then passes through the channel attention mechanism to obtain feature D2.

[0086] As shown in Figure 5 , the multi-scale attention guided feature fusion sub-network two includes a convolution-bilinear interpolation up-sampling combination layer, a convolution-batch normalization combination layer, a CBR-convolution-batch normalization combination layer, an element-wise addition module, a weighted fusion module and a channel attention mechanism module:

[0087] In the second multi-scale attention-guided feature fusion subnetwork, the input features are A3, A4, and A5, respectively. Feature A3 is processed through eight cascaded basic units to obtain feature B3; feature A4 is processed through eight cascaded basic units to obtain feature B4; feature A5 is processed through eight cascaded basic units to obtain feature B5; the feature size of feature B4 is H / 4×W / 4, and the feature size of feature B5 is H / 8×W / 8.

[0088] Feature B3 is added element-wise to B4 and B5 after the upsampling combination layer to obtain feature C3; feature B4 is added element-wise to B3 after the downsampling combination layer 1 and B5 after the upsampling combination layer to obtain feature C4; feature B5 is added element-wise to B3 after the downsampling combination layer 2 and B4 after the downsampling combination layer 1 to obtain feature C5; the feature size of feature C3 is H / 2×W / 2, the feature size of features B4 and C4 is H / 4×W / 4, and the feature size of features B5 and C5 is H / 8×W / 8.

[0089] Feature C3 is activated by ReLU and then weighted and fused with Y1, the output of step 2, and then processed through a channel attention mechanism to obtain feature D3; feature C4 is activated by ReLU and then weighted and fused with Y2, the output of step 2, and then processed through a channel attention mechanism to obtain feature D4; feature C5 is activated by ReLU and then weighted and fused with Y3, the output of step 2, and then processed through a channel attention mechanism to obtain feature D5.

[0090] like Figure 5 As shown, the multi-scale attention-guided feature fusion sub-network three includes a convolution-bilinear interpolation upsampling combination layer, a convolution-batch normalization combination layer, a CBR-convolution-batch normalization combination layer, a CBR-CBR-convolution-batch normalization combination layer, an element-by-element addition module, a weighted fusion module, and a channel attention mechanism module:

[0091] In the third multi-scale attention-guided feature fusion subnetwork, the input features are A6, A7, A8, and A9, respectively. Feature A6 is processed through eight cascaded basic units to obtain feature B6; feature A7 is processed through eight cascaded basic units to obtain feature B7; feature A8 is processed through eight cascaded basic units to obtain feature B8; and feature A9 is processed through eight cascaded basic units to obtain feature B9. The feature size of feature B6 is H / 2×W / 2, the feature size of feature B7 is H / 4×W / 4, the feature size of feature B8 is H / 8×W / 8, and the feature size of feature B9 is H / 16×W / 16.

[0092] Feature B6 is added element by element with B7, B8 and B9 of the up-sampling combination layer in turn to obtain feature C6; feature B7 is added element by element with B6 of the down-sampling combination layer 1, B8 and B9 of the up-sampling combination layer in turn to obtain feature C7; feature B8 is added element by element with B6 of the down-sampling combination layer 2, B7 of the down-sampling combination layer 1 and B9 of the up-sampling combination layer in turn to obtain feature C8; feature B9 is added element by element with B6 of the down-sampling combination layer 3, B7 of the down-sampling combination layer 2 and B8 of the down-sampling combination layer 1 in turn to obtain feature C9; feature size of feature C6 is H / 2×W / 2, feature size of feature C7 is H / 4×W / 4, feature size of feature C8 is H / 8×W / 8, and feature size of feature C9 is H / 16×W / 16;

[0093] Feature C6 is subjected to a ReLU activation function, then weighted fused with Y1 output by step 2, and then subjected to a channel attention mechanism to obtain feature D6; feature C7 is subjected to a ReLU activation function, then weighted fused with Y2 output by step 2, and then subjected to a channel attention mechanism to obtain feature D7; feature C8 is subjected to a ReLU activation function, then weighted fused with Y3 output by step 2, and then subjected to a channel attention mechanism to obtain feature D8; feature C9 is subjected to a ReLU activation function, then weighted fused with Y4 output by step 2, and then subjected to a channel attention mechanism to obtain feature D9.

[0094] As shown in FIG. 1, the basic unit comprises a CBR, a convolution, a batch normalization combination, and an element-wise addition operation. Figure 3 As shown in FIG. 1, the basic unit comprises a CBR, a convolution, a batch normalization combination, and an element-wise addition operation.

[0095] As shown in FIG. 1, the basic unit comprises a CBR, a convolution, a batch normalization combination, and an element-wise addition operation. Figure 2 As shown in FIG. 1, the basic unit comprises a CBR, a convolution, a batch normalization combination, and an element-wise addition operation.

[0096] First, the multi-scale attention guided feature fusion network is used to align the multi-scale features obtained by the multi-scale attention guided feature fusion network to the same scale H / 2×W / 2, and the obtained features are Z1, Z2, Z3 and Z4. Four prediction results are obtained by passing the features through four independent detection heads; the features Z1, Z2, Z3 and Z4 are spliced along the channel dimension to obtain a feature Z0, and one prediction result is obtained by passing the feature Z0 through a detection head.

[0097] The five prediction results are subjected to loss calculation, and the five losses are fused by a multi-level aggregation loss function. The model's feature learning ability is improved in a multi-scale supervision manner, so as to provide a stronger supervision signal, optimize the feature learning process, and enhance the stability of the detection result.

[0098] As shown in FIG. 1, the basic unit comprises a CBR, a convolution, a batch normalization combination, and an element-wise addition operation. Figure 3As shown, in the detection head module of the prediction network, the input features are sequentially passed through a CBR and a convolution combination to obtain the output of the detection head.

[0099] Step 3: Network training and prediction. The constructed multi-resolution-channel attention key point detection network is used to train the labeled head lateral position image. The newly collected head lateral position image to be detected is input into the trained network to automatically predict the coordinates of 12 key points. At the same time, the error between the prediction result and the manual annotation is output to better observe the accuracy of the prediction data. Thus, the qualified rate of the test data under different evaluation indicators is obtained to better observe the accuracy of the prediction data.

[0100] The loss function of each output head uses the face key point detection evaluation index NME, and the NME of each image is defined as: the Euclidean distance between all predicted points and labeled points, divided by the number of key points. The above loss function expression is as follows:

[0101]

[0102] Where P and G are the predicted value and true value of the key point coordinates of each image, p i is the predicted value of the i-th key point, g i is the true value of the i-th key point, M is the number of key points, and d is a normalization factor used to eliminate the influence of scale, which is set to 1 here.

[0103] When the multi-level aggregation loss function weights and fuses the loss of each detection head: the loss of the prediction result obtained after feature concatenation is recorded as loss0, accounting for 0.6; the loss of the four prediction results obtained before feature concatenation is recorded as loss1, loss2, loss3 and loss4, each accounting for 0.1.

[0104] The network prediction part can output the error between the prediction result and the manual annotation result while outputting the coordinates of 12 key points, so as to obtain the qualified rate of the test data under different evaluation indicators, thereby better observing the accuracy of the prediction result.

[0105] For an X-ray lateral position image to be detected, the actual application specifically includes the following steps:

[0106] (1) The training set includes 336 high-heterogeneity X-ray lateral position images of research cases.

[0107] (2) Label the coordinate positions of 12 key points on 336 X-ray lateral head images, the specific steps are: first, label in the physical coordinate system, and the measurement unit is set as millimeter; subsequently, convert the position coordinates labeled in the physical coordinate system into corresponding coordinates in the pixel coordinate system by using a predetermined conversion algorithm, so as to complete the mapping from the physical space to the pixel space.

[0108] (3) Use data enhancement technology on the 336 training data, including random scaling, rotation and other diversified processing, to obtain 1680 data, and divide them into training set and validation set according to the ratio of 0.85:0.15, finally obtain 1344 training data and 336 validation data.

[0109] (4) Set the initial value of the key point detection network model hyperparameter, set the training round to 50 rounds, set the initial learning rate to 0.0002, and perform learning rate decay at 10, 20 and 30 rounds, respectively, and use Adam optimizer.

[0110] (5) Repeat steps (3) to (4) as the training stage of the multi-resolution-channel attention key point detection network model, and obtain the final model weight.

[0111] (6) Use the multi-resolution-channel attention key point detection network model trained in step (5) to test the newly collected X-ray lateral head images, and finally obtain the predicted coordinate positions of 12 key points in the newly collected X-ray lateral head images. At the same time, output the error between the prediction result and the manual labeling, so as to better observe the accuracy of the prediction data.

[0112] As shown in Figure 6 , the final 12 key point coordinate positions output by the present application are shown in the figure, and the numbers 1-12 in the figure indicate the serial numbers of the points.

[0113] The lightweight fusion feature extraction network constructed in the application carries out feature extraction and enhancement operation, through lightweight design, the model parameter quantity is reduced, the calculation complexity is significantly reduced, and the effectiveness of feature extraction is maintained. The channel attention mechanism module highlights the features of important channels and suppresses the interference of irrelevant channels by adaptively adjusting the weight of each channel. The multi-resolution progressive feature extraction sub-network extracts multi-resolution features, which are fused in the main part of the network through the way of skip connection, realizing more effective multi-scale feature expression. The multi-scale attention guided feature fusion network generates a branch of new resolution, and repeatedly fuses the information of all current branches. The prediction network aligns the multi-scale features obtained by the multi-scale attention guided feature fusion network to the same scale through upsampling, and performs splicing to fuse the information of different levels. The features before splicing and the features after splicing are processed by independent detection heads respectively, different levels of features are generated, and the generalization ability of the model is supervised and improved through a multi-level aggregation loss function. In the test stage, the newly collected head lateral image is input into the multi-resolution-channel attention key point detection network which has been trained, only one scanning of the whole head lateral image is needed, and the positions of 12 key feature points in the image can be quickly and automatically located.

[0114] In addition to the above embodiments, the application can have other implementation manners. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the application.

Claims

1. A method for constructing a multi-resolution-channel attention network capable of detecting key points in highly heterogeneous X-ray lateral cephalometric images, characterized in that... Includes the following steps: Step 1: Construct a lightweight fusion feature extraction network: The lightweight fusion feature extraction network consists of a lightweight fusion convolution-batch normalization-ReLU activation function combination layer, a bottleneck unit connected in sequence, and finally a channel attention mechanism module; Step 2: Construct a multi-resolution progressive feature extraction sub-network: The multi-resolution progressive feature extraction sub-network is based on a feature pyramid network, which performs multi-resolution feature extraction on the original input image and fuses them in the main part of the network through skip connections. Step 3: Construct a multi-scale attention-guided feature fusion network: The multi-scale attention-guided feature fusion network consists of a convolutional-batch normalization-ReLU activation function combination layer and a multi-scale attention-guided feature fusion sub-network connected sequentially; the multi-scale attention-guided feature fusion network includes a convolutional-batch normalization-ReLU activation function combination layer (CBR), multi-scale attention-guided feature fusion sub-network one, multi-scale attention-guided feature fusion sub-network two, and multi-scale attention-guided feature fusion sub-network three; The multi-scale attention-guided feature fusion subnetwork includes basic units, a convolution-bilinear interpolation upsampling combined layer, a convolution-batch normalization combined layer, an element-by-element addition module, a weighted fusion module, and a channel attention mechanism module. The multi-scale attention-guided feature fusion subnetwork 2 includes a convolution-bilinear interpolation upsampling combination layer, a convolution-batch normalization combination layer, a CBR-convolution-batch normalization combination layer, an element-by-element addition module, a weighted fusion module, and a channel attention mechanism module; The multi-scale attention-guided feature fusion sub-network includes a convolution-bilinear interpolation upsampling combination layer, a convolution-batch normalization combination layer, a CBR-convolution-batch normalization combination layer, a CBR-CBR-convolution-batch normalization combination layer, an element-by-element addition module, a weighted fusion module, and a channel attention mechanism module. In the basic unit, the input features are sequentially processed by a combination of a CBR, a convolution, and a batch normalization, and then added element-wise with the input features. The result of the addition is then passed through the ReLU activation function to obtain the output of the basic unit. Step 4: Construct the prediction network: The prediction network aligns the multi-scale features obtained from the multi-scale attention-guided feature fusion network to the same scale through upsampling, and then concatenates them to fuse information at different levels; then the features before and after concatenation are processed by independent detection heads to generate prediction results, and the generalization ability of the model is improved by supervising the multi-level aggregation loss function. Step 5: Network Training and Prediction: The constructed multi-resolution-channel attention keypoint detection network is used to train the labeled lateral head images; the newly acquired lateral head images to be detected are input into the trained network to automatically predict the keypoint positions in the images and output the error between the prediction results and the manual annotations.

2. The method according to claim 1, characterized in that: In step 1, the constructed lightweight fusion feature extraction network includes a lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR), a bottleneck unit, and a channel attention mechanism module: First, the input image passes through a lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR) to complete the initial feature extraction, with the extracted feature size being H×W; then it passes through a second lightweight fusion convolution-batch normalization-ReLU activation function combination layer (LBR) to perform feature extraction and downsampling, changing the feature size to H / 2×W / 2. The obtained features are passed through four cascaded bottleneck units, and then through a channel attention mechanism module.

3. The method according to claim 2, characterized in that: In the lightweight fusion convolution, the input features are processed by a lightweight convolution and then concatenated with the features processed by a standard convolution. The concatenated result is then processed by the ReLU activation function to obtain the output of the lightweight fusion convolution.

4. The method according to claim 2, characterized in that: In the bottleneck unit, the input features are sequentially passed through two convolution-batch normalization-activation function combination layers (CBR), one convolution, and one batch normalization layer. Then, they are added element-by-element to the input features. The result of the addition is then passed through the ReLU activation function to obtain the output of the bottleneck unit.

5. The method according to claim 2, characterized in that: The channel attention mechanism module captures global information at the channel level, multiplies the obtained weights with the original features, and achieves adaptive feature recalibration at the channel level, highlighting the features of important channels and suppressing interference from irrelevant channels; the specific process is as follows: First, for the input features of size B×C×H×W, the information of spatial dimension H×W is compressed into a global channel feature vector through global average pooling, and the compressed size is B×C. Then a two-layer fully connected network is used to generate the weight values ​​for each channel: the first layer reduces the number of channels from C to C / 16, dimensionality reduction to reduce the number of parameters and computation and learn the compressed features between channels, and then the ReLU activation function is used to introduce nonlinearity. The second layer restores the original number of channels C, and generates a weight value for each channel. Then, it is normalized to [0,1] by the Sigmoid activation function. The generated channel weights are multiplied with the input features channel by channel to enhance or suppress the channel features.

6. The method according to claim 1, characterized in that: In step 2, the multi-resolution progressive feature extraction subnetwork includes a convolution-batch normalization-ReLU activation function combination layer (CBR), a convolution-bilinear interpolation upsampling combination layer, and a weighted fusion module. First, the input image has dimensions H×W. After passing through a CBR module, feature X1 is obtained, with a feature size of H / 2×W / 2. X1 passes through a CBR module to obtain feature X2, with a feature size of H / 4×W / 4. X2 passes through a CBR module to obtain feature X3, with a feature size of H / 8×W / 8. X3 passes through a CBR module to obtain feature X4, with a feature size of H / 16×W / 16. Feature X2 is weighted and fused with feature X1 after passing through an upsampling combination layer to obtain feature Y1; feature X3 is weighted and fused with feature X2 after passing through an upsampling combination layer to obtain feature Y2; feature X4 is weighted and fused with feature X3 after passing through an upsampling combination layer to obtain feature Y3; feature Y4 is directly obtained from feature X4. The final output features are Y1, Y2, Y3 and Y4, where the feature size of Y1 is H / 2×W / 2, the feature size of Y2 is H / 4×W / 4, the feature size of Y3 is H / 8×W / 8, and the feature size of Y4 is H / 16×W / 16.

7. The method according to claim 1, characterized in that: In the first multi-scale attention-guided feature fusion subnetwork, the input features are A1 and A2, respectively; feature A1 is processed through four cascaded basic units to obtain feature B1; Feature A2 is passed through four cascaded basic units to obtain feature B2; wherein the feature size of feature B1 is H / 2×W / 2, and the feature size of feature B2 is H / 4×W / 4; Feature B1 is added element-wise to B2 after the upsampling combination layer to obtain feature C1; feature B2 is added element-wise to B1 after the downsampling combination layer 1 to obtain feature C2; ​​the feature size of feature C1 is H / 2×W / 2, and the feature size of feature C2 is H / 4×W / 4. Feature C1 is activated by ReLU and then weighted and fused with Y1, the output of step 2, and then processed by channel attention mechanism to obtain feature D1; feature C2 is activated by ReLU and then weighted and fused with Y2, the output of step 2, and then processed by channel attention mechanism to obtain feature D2. In the second multi-scale attention-guided feature fusion subnetwork, the input features are A3, A4, and A5, respectively. Feature A3 is processed through eight cascaded basic units to obtain feature B3; feature A4 is processed through eight cascaded basic units to obtain feature B4; feature A5 is processed through eight cascaded basic units to obtain feature B5; the feature size of feature B3 is H / 2×W / 2, the feature size of feature B4 is H / 4×W / 4, and the feature size of feature B5 is H / 8×W / 8. Feature B3 is obtained by adding element-wise to B4 and B5 after the upsampling combination layer; feature B4 is obtained by adding element-wise to B3 after the downsampling combination layer 1 and B5 after the upsampling combination layer; feature B5 is obtained by adding element-wise to B3 after the downsampling combination layer 2 and B4 after the downsampling combination layer 1; the feature size of feature C3 is H / 2×W / 2, the feature size of feature C4 is H / 4×W / 4, and the feature size of feature C5 is H / 8×W / 8. Feature C3 is activated by ReLU and then weighted and fused with Y1, the output of step 2, and then processed through a channel attention mechanism to obtain feature D3; feature C4 is activated by ReLU and then weighted and fused with Y2, the output of step 2, and then processed through a channel attention mechanism to obtain feature D4; feature C5 is activated by ReLU and then weighted and fused with Y3, the output of step 2, and then processed through a channel attention mechanism to obtain feature D5. In the third multi-scale attention-guided feature fusion subnetwork, the input features are A6, A7, A8, and A9, respectively. Feature A6 is processed through eight cascaded basic units to obtain feature B6; feature A7 is processed through eight cascaded basic units to obtain feature B7; feature A8 is processed through eight cascaded basic units to obtain feature B8; and feature A9 is processed through eight cascaded basic units to obtain feature B9. The feature size of feature B6 is H / 2×W / 2, the feature size of feature B7 is H / 4×W / 4, the feature size of feature B8 is H / 8×W / 8, and the feature size of feature B9 is H / 16×W / 16. Feature B6 is added element-wise to B7, B8, and B9 after the upsampling combination layer to obtain feature C6; feature B7 is added element-wise to B6 after the downsampling combination layer 1, B8 after the upsampling combination layer, and B9 after the upsampling combination layer to obtain feature C7; feature B8 is added element-wise to B6 after the downsampling combination layer 2, B7 after the downsampling combination layer 1, and B9 after the upsampling combination layer to obtain feature C8; feature B9 is added element-wise to B6 after the downsampling combination layer 3, B7 after the downsampling combination layer 2, and B8 after the downsampling combination layer 1 to obtain feature C9. The feature size of feature C6 is H / 2×W / 2, the feature size of feature C7 is H / 4×W / 4, the feature size of feature C8 is H / 8×W / 8, and the feature size of feature C9 is H / 16×W / 16. Feature C6 is activated by ReLU and then weighted and fused with Y1, the output of step 2, and then processed through a channel attention mechanism to obtain feature D6; feature C7 is activated by ReLU and then weighted and fused with Y2, the output of step 2, and then processed through a channel attention mechanism to obtain feature D7; feature C8 is activated by ReLU and then weighted and fused with Y3, the output of step 2, and then processed through a channel attention mechanism to obtain feature D8; feature C9 is activated by ReLU and then weighted and fused with Y4, the output of step 2, and then processed through a channel attention mechanism to obtain feature D9.

8. The method according to claim 1, characterized in that: In step 4, the prediction network includes a bilinear interpolation upsampling module, a splicing module, and a detection head module: First, the multi-scale features obtained from the multi-scale attention-guided feature fusion network are aligned to the same scale H / 2×W / 2 through bilinear interpolation upsampling, resulting in features Z1, Z2, Z3, and Z4. These features are then passed through four independent detection heads to obtain four prediction results. Finally, features Z1, Z2, Z3, and Z4 are concatenated along the channel dimension to obtain feature Z0, which is then passed through one detection head to obtain one prediction result. The five predicted results are used to calculate the loss. The five losses are then fused using a multi-level aggregation loss function. Multi-scale supervision is used to improve the model's feature learning ability, thereby providing stronger supervision signals, optimizing the feature learning process, and enhancing the stability of the detection results. In the detection head module, the input features are sequentially processed by a combination of a CBR and a convolution to obtain the output of the detection head.

9. The method according to claim 1, characterized in that: In step 5, the network prediction part compares the prediction results with the manually labeled results while outputting the coordinates of key points in the image, and outputs the error between them, thereby obtaining the pass rate of the test data under different evaluation indicators.

10. The method according to claim 1, characterized in that: The specific structure of the multi-scale attention-guided feature fusion network is as follows: First, the input features are passed through a CBR module, which maintains the original resolution branch while changing the current number of channels to obtain feature A1; The input features are downsampled through a CBR module to generate a new resolution branch, resulting in feature A2; A1 and A2 are then fused through a multi-scale attention-guided feature fusion subnetwork to obtain output features D1 and D2. in, The feature size of input feature, feature A1, and feature D1 is H / 2×W / 2; the feature size of feature A2 and feature D2 is H / 4×W / 4; Output features D1 and D2 are each passed through a CBR module to obtain features A3 and A4; D2 is downsampled through a CBR module to generate a new resolution branch, resulting in feature A5; A3, A4, and A5 are then passed through four cascaded multi-scale attention-guided feature fusion subnetworks to obtain output features D3, D4, and D5; where the feature size of features A3 and D3 is H / 2 × W / 2; the feature size of features A4 and D4 is H / 4 × W / 4; and the feature size of features A5 and D5 is H / 8 × W / 8. Output features D3, D4, and D5 are passed through a CBR module to obtain features A6, A7, and A8, respectively. D5 is downsampled through a CBR module to generate a new resolution branch, resulting in feature A9. A6, A7, A8, and A9 are then passed through three cascaded multi-scale attention-guided feature fusion subnetworks to obtain output features D6, D7, D8, and D9. The feature size of features A6 and D6 is H / 2 × W / 2, the feature size of features A7 and D7 is H / 4 × W / 4, the feature size of features A8 and D8 is H / 8 × W / 8, and the feature size of features A9 and D9 is H / 16 × W / 16.

Images