Multi-category keypoint detection method based on fusion of high-resolution features

Abstract

The present invention discloses a multi-category key point detection method based on the fusion of high-resolution features, which relates to the field of computer vision technology. The method includes labeling a bounding box and key point coordinates of a target to be detected; performing the longest alignment on the key points of the image and converting them into a Gaussian distribution map or directly converting them into a Gaussian distribution map; using bounding box data to train a target detector; passing the Gaussian distribution map as a label into an improved deep learning key point detection model for training; inputting the image to be detected into the trained target detector to obtain target position information, and passing it into the model after preprocessing to obtain key point coordinates; calculating the image gradient of the key point and its adjacent pixels, and considering the point with the largest gradient change as the final key point. The present invention can fuse high-resolution feature maps and secondary-resolution feature maps, more accurately capture effective position information, and predict more accurate key point positions. It can also be extended to multiple categories of key point detection without increasing too much computational overhead.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, in particular to the field of target key point detection technology, and specifically to a multi-category key point detection method based on the fusion of high-resolution features. Background Art

[0002] In recent years, keypoint detection has become a significant research area in image processing and computer vision. This technology aims to determine the locations of keypoints on a target, including human pose estimation, facial keypoint estimation, and aircraft keypoint estimation. The accuracy of this technology directly impacts subsequent data-driven decision-making, such as quantifying human motion (for rehabilitation assessments), object motion (for robotic gripping), and attacking vulnerable areas on aircraft. With the rapid development of deep learning, many neural network-based methods for object keypoint detection have achieved remarkable results. These methods have, in their own ways, advanced keypoint technology. However, the following challenges remain:

[0003] First, existing methods continuously downsample the original image through convolutional neural networks. During this process, although the network model can learn high-level features of the image, as the number of convolutions increases, the network may find it difficult to extract useful features. For key point detection tasks, since this task is extremely sensitive to position information, if incorrect position information is extracted in the network, the accuracy of the final prediction result will be seriously affected.

[0004] In addition, the current key point detection is based on a single category, while in actual application needs, there are many different categories of targets. For example, in the aircraft key point detection task, it may include fixed-wing aircraft, rotorcraft, etc. If the data needs to be re-labeled and the network needs to be trained for each type of aircraft, it will not only take a lot of time, but also make the system bloated.

[0005] It can be seen that in the key point detection method, there are problems such as the accuracy of key point detection being affected by incorrect position information, and the time-consuming and bloated system required to implement multiple categories of key point detection. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-category key point detection method based on the fusion of high-resolution features, which is used to solve the problems in the prior art that the accuracy of key point detection is affected by erroneous position information and that it takes a long time to realize key point detection of multiple categories and the system is bloated.

[0007] The present invention solves the above problems through the following technical solutions:

[0008] A multi-category keypoint detection method based on fusion of high-resolution features, including:

[0009] Step S100: collecting image data of the target to be detected to obtain image data samples;

[0010] Step S200: annotating the bounding box and key point coordinates of the target to be detected in the image data sample to obtain bounding box data and key point data;

[0011] Step S300: Determine whether the key point data requires longest alignment. If so, perform longest alignment and then convert the key point data after the longest alignment into a Gaussian distribution map. Otherwise, directly convert the key point data into a Gaussian distribution map.

[0012] Step S400: using bounding box data to train an object detector, the object detector is used to predict the position information of the object to be detected by key points in the image to be detected;

[0013] Step S500: passing the Gaussian distribution map as a label into the improved deep learning key point detection model for training, and obtaining a trained key point detection model;

[0014] Step S600: Input the image to be detected into the trained object detector to obtain the position information of all objects in the image to be detected. The position of the object in the image to be detected is cut out to obtain a sub-image. If the size of the sub-image is not equal to the preset size, the sub-image is stretched or compressed to the preset size and then input into the trained key point detection model to obtain the key point coordinate information.

[0015] In step S700 , based on the key point coordinate information predicted by the model, the image gradient of the key point and its adjacent pixels is calculated, and the point with the largest gradient change is regarded as the final key point.

[0016] By constructing an improved deep learning keypoint detection model, this paper continuously fuses high-resolution feature maps from the original size with secondary-resolution feature maps from upper convolution layers during feature extraction, thereby more accurately capturing effective location information and predicting more precise keypoint locations. Through longest alignment, a network model can be expanded from detecting keypoints in a single category to detecting keypoints in multiple categories without significantly increasing computational overhead.

[0017] Furthermore, the method for determining whether the key point data requires longest alignment in step S300 is: when the number of key points of a certain category in the target to be detected is equal to the maximum number of key points in all categories, longest alignment is not required; otherwise, longest alignment is required.

[0018] For example, when detecting fixed-wing aircraft and quadcopters, the categories are fixed-wing aircraft (with 5 keypoints) and quadcopters (with 7 keypoints). The maximum number of keypoints between the two categories is the number of keypoints for the quadcopter. In real applications, the objects to be detected may not always be aircraft; they may also be other categories, such as keypoints of the human skeleton. Furthermore, the maximum number of keypoints is determined based on the actual application and the target to be detected. For example, if three categories, A, B, and C, are detected, and the corresponding numbers of keypoints are 6, 9, and 9, respectively, the maximum number of keypoints is 9.

[0019] Furthermore, the longest alignment method in step S300 is as follows:

[0020] Step S310: If key points of the same category are marked, the order of marking key points of each target must be consistent;

[0021] In step S320, if key points of different categories are marked and the number of key points between categories is inconsistent, then after the target with fewer key points is marked in order, it is necessary to perform a zero-filling operation at the end until the number is filled to the maximum number of key points among all categories, thereby completing the longest alignment.

[0022] Furthermore, the method of converting the key point data into a Gaussian distribution map is:

[0023] Step A1: convert the coordinates of the key points into scale factors relative to the width and height of the image;

[0024] Step A2: pre-fix the width and height of the target to be measured. If the obtained image is inconsistent with the preset width and height, stretch or compress the image to the preset value;

[0025] Step A3: Generate an image with a channel of 1, a size of the preset value, and all pixel values ​​of 0;

[0026] Step A4: Generate a two-dimensional Gaussian distribution map at the corresponding position of the image generated in step A3 using the scale factor. The corresponding function is:

[0027] ;

[0028] Where, Indicates the generation of a two-dimensional Gaussian distribution graph in The value at the coordinate, represents the amplitude, and Represents the center of the two-dimensional Gaussian distribution map, that is, the position coordinates of the key point in the image, and Represents the manually preset variance on the x-axis and y-axis in the Gaussian function respectively;

[0029] If a key point is obtained through the longest alignment, then the amplitude of the Gaussian function corresponding to the key point is 0.

[0030] Furthermore, the improved deep learning key point detection model includes two parts: an encoder and a decoder. The encoder is responsible for extracting semantic features in the image, and the decoder is responsible for translating the extracted features.

[0031] The encoder has four layers, each of which is composed of a first module, which is composed of a resolution-maintaining component, a downsampling component and a first feature fusion component; the resolution-maintaining component is composed of a convolution layer, a BN layer, and an activation function layer. After the data passes through the resolution-maintaining component, the size of the feature map does not change, thereby obtaining a high-resolution feature map; the downsampling component is composed of a convolution layer, a BN layer, an activation function layer, and a pooling layer. After the data passes through the downsampling component, the size of the feature map is reduced, but more advanced semantic features are obtained; the first feature fusion component is composed of a convolution layer, a BN layer, an activation function layer, a pooling layer, and a channel splicing layer. The feature fusion component fuses features of different sizes together to ensure the stability of feature extraction during the encoding process; in the first module of the first layer of the encoder, the first feature fusion component only needs to fuse the output of the resolution-maintaining component in the first module of the layer and the output features of the downsampling component. In the first modules of the subsequent three layers of the encoder, in addition to fusing the output of the resolution-maintaining component in the first module of the corresponding layer and the output features of the downsampling component, the first feature fusion component also fuses the output of the resolution-maintaining component in the first module of the first layer;

[0032] The decoder consists of four layers, each layer consists of a second module, and each second module is divided into two parts: an upsampling component and a second feature fusion component; the upsampling component consists of a transposed convolution layer, and after passing through the transposed convolution layer, the size of the output feature map will be enlarged to the size of the feature map output by the resolution preservation component corresponding to the same layer in the encoder; the second feature fusion component includes a feature channel splicing layer, a convolution layer, a BN layer, and an activation function. The second feature fusion layer will fuse the feature map output by the resolution preservation component in the first module of the same layer of the encoder with the feature map after the upsampling component in the second module of the same layer of the decoder to ensure that the decoder decodes in the correct direction.

[0033] Furthermore, the loss function of the improved deep learning key point detection model is defined as:

[0034] ;

[0035] Where m represents the maximum number of key points, and n represents the number of pixels output for each feature map. Indicates the true The Gaussian distribution map corresponding to the key point pixel values, The predicted The Gaussian distribution map corresponding to the key point pixel values.

[0036] Furthermore, the specific method for calculating the image gradient of the key point and its adjacent pixels in step S700 is:

[0037] Gradient value calculation template in the x direction:

[0038] ;

[0039] Gradient value calculation template in the y direction:

[0040] ;

[0041] The gradient value corresponding to the pixel point:

[0042] ;

[0043] in, represents the x-direction template, I represents the corresponding image area, Indicates the y-direction template; 、 is the absolute value of the gradient in the x and y directions;

[0044] Finally, the pixel with the largest gradient is regarded as the location of the final key point.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] (1) By constructing a network model, the present invention can continuously fuse high-resolution feature maps from the original size and secondary-resolution feature maps from the upper convolution layer in the convolution extraction feature, thereby more accurately capturing effective position information, thereby being able to predict more accurate key point positions in practical applications and to a certain extent suppressing the effects of background interference and changes in object posture.

[0047] (2) The present invention performs the longest alignment of key points during the model training stage, so that the network model has the ability to detect key points of multiple categories, thereby eliminating the need to train the model separately for each category, thereby adapting to multi-target key point detection in actual engineering and increasing flexibility in engineering applications.

[0048] (3) The key points designed in the present invention are aligned for the longest time. Since the pixel values ​​of the feature map finally generated by the aligned key points are all 0, not too many resources are consumed in both the model training stage and the prediction stage, thus reducing the computational cost.

[0049] (4) The present invention adds image gradient correction based on the prediction results of the model. By calculating the gradients of the predicted pixel points and the adjacent pixel points, the final prediction results are updated to make the prediction results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a network structure diagram of the improved deep learning key point detection model in the present invention;

[0051] Figure 2 The following are the detection results of five key points of a fixed-wing aircraft, where (a) is the detection result under a normal viewing angle; (b) is the detection result under a partially occluded viewing angle; and (c) is the detection result under a deformed viewing angle caused by the aircraft flipping.

[0052] Figure 3 The following are the detection results of seven key points of the quadrotor aircraft, where (a) is the detection result under a blurred perspective; (b) is the detection result under an occluded perspective; and (c) is the detection result under a good perspective. DETAILED DESCRIPTION

[0053] The present invention will be further described in detail below with reference to the examples, but the embodiments of the present invention are not limited thereto.

[0054] Example 1:

[0055] Combined with attachment Figure 1 As shown in FIG, a multi-category key point detection method based on the fusion of high-resolution features includes:

[0056] Step 1: Based on actual needs, a certain number of target images to be detected are collected by the detector. For example, in this embodiment, the fixed-wing aircraft and the quadrotor aircraft each collect one thousand images with a width and height of 1024 pixels and a channel number of 1 for subsequent model training and testing.

[0057] Step 2: Use the labelme annotation tool to annotate the collected images. Since the longest alignment is required later, the number of longest key points needs to be determined before annotation.

[0058] Specifically, in this embodiment, the longest keypoint count is 7. The specific format for labeling an object is as follows: horizontal and vertical coordinates of the upper left corner of the object's bounding box, horizontal and vertical coordinates of the lower right corner of the object's bounding box, coordinates of keypoint 1, coordinates of keypoint 2, coordinates of keypoint 3, coordinates of keypoint 4, coordinates of keypoint 5, coordinates of keypoint 6, coordinates of keypoint 7. It is important to note that: 1. The order of labeling keypoints for the same type of object must be consistent, and the specific order can be customized; 2. Insufficient keypoints on an object must be padded with zeros for alignment. In this embodiment, the labeling results of a quadrotor aircraft with seven key points are as follows: the horizontal and vertical coordinates of the upper left corner of the aircraft enclosing box, the horizontal and vertical coordinates of the lower right corner of the aircraft enclosing box, the nose coordinates, the fuselage coordinates, the tail coordinates, the rotor 1 coordinates, the rotor 2 coordinates, the rotor 3 coordinates, and the rotor 4 coordinates; the labeling results of a fixed-wing aircraft with five key points after the longest alignment are as follows: the horizontal and vertical coordinates of the upper left corner of the aircraft enclosing box, the horizontal and vertical coordinates of the lower right corner of the aircraft enclosing box, the nose coordinates, the fuselage coordinates, the tail coordinates, the left wing coordinates, the right wing coordinates, (0, 0), (0, 0).

[0059] Step 3: Divide the labeled data into training and test sets according to the ratio. Specifically, in this embodiment, the training and test sets are divided in a ratio of 7:3, that is, 700 images of fixed-wing aircraft and 300 images of quadrotor aircraft are used for training and 300 images of each are used for testing.

[0060] In step 4, a target detection model is first trained using the labeled data. Since keypoint detection only requires the location of the target in the image and does not require the specific category, the number of categories in the target detection model is set to 1, and the category names are customizable. Specifically, in this embodiment, YOLOv5 is used as the target detection model to detect the position of the aircraft in the image. The model outputs this location information as input for the next step of keypoint detection.

[0061] Step 5: In the training phase, the data is preprocessed, and then the preprocessed data is trained using the improved deep learning key point detection model. Specifically, in this embodiment, the data preprocessing method is as follows: First, the size of the target is calculated based on the coordinates of the bounding box, and the calculation method is as follows:

[0062] , ;

[0063] Among them, w represents the target width, Indicates the horizontal coordinate of the lower right corner of the bounding box, Indicates the horizontal coordinate of the upper left corner of the bounding box; h indicates the target height, Indicates the vertical coordinate of the lower right corner of the bounding box, Indicates the vertical coordinate of the upper left corner of the bounding box.

[0064] Secondly, the width, height and key point position information of the target to be detected are obtained based on the original annotation information, and based on this, the scale factor of each key point relative to the width and height of the target is obtained. The calculation method is as follows:

[0065] , ;

[0066] in, represents the scale factor for width, Indicates the distance between the horizontal coordinate of the key point and the left side of the bounding box, and w indicates the width of the bounding box; represents the scale factor for height, Indicates the distance between the vertical coordinate of the key point and the top of the bounding box, and h indicates the height of the bounding box;

[0067] According to the above calculation, if the width or height of the target is not equal to 255, the target will be transformed to the original image with a width and height of 255 by stretching or shrinking.

[0068] Then, based on the transformed image size and the aspect ratio factor, the position of the transformed coordinate point is calculated as follows:

[0069] , ;

[0070] in Represents the horizontal coordinate of the key point after transformation, Indicates the vertical coordinate of the key point after transformation; is the width scaling factor obtained above, is the height scaling factor obtained above.

[0071] After obtaining the coordinates of the key points after transformation, n Gaussian distribution heat maps can be generated, where n is the maximum number of key points customized in the previous text. In this implementation case, n is 7. Specifically, first generate a picture with a width and height of 255, all values ​​​​are 0, and the number of channels is 1. Then, according to the transformed coordinates obtained in the previous text, the corresponding position of the picture is set to 255, and then a Gaussian convolution kernel is used for Gaussian filtering. Finally, a Gaussian distribution heat map corresponding to the key point is generated. In this implementation case, the size of the Gaussian kernel is 7. It should be noted that the label map values ​​generated by the key points obtained by the longest alignment should all be 0. In response to the problem that the key point detection task is sensitive to position information, the present invention makes corresponding improvements on the basis of the general convolutional neural network, integrating high-resolution feature maps, so that it can more effectively encode and decode position information. The specific improvements are as follows:

[0072] The model is divided into an encoding part and a decoding part. In the encoding part, there are a total of four layers, each of which contains a resolution-preserving component, a downsampling component, and a feature fusion component. The resolution-preserving component consists of a convolutional layer with a convolution kernel size of 1, a batch normalization layer, and an activation function layer. When the data passes through this component, the size of the feature map will not change, thereby preserving the high-resolution feature map. The downsampling component consists of a convolutional layer with a convolution kernel size of 3 and a padding number of 1, a batch normalization layer, an activation function layer, and a maximum pooling layer of size 2. When the data passes through this component, the size of the feature map will be halved, thereby obtaining more advanced semantic features. The feature fusion component consists of a convolutional layer with a convolution kernel size of 3 and a padding number of 1, a batch normalization layer, an activation function layer, a size-adaptive maximum pooling layer, and a data splicing layer. The data passing through this component can transform the size of the feature map to the same size as the fused feature map, and then the two data are spliced ​​together according to the number of channels through the data splicing layer. In the encoder, each module first accepts the feature map output by the previous module as its input, with the first module receiving the original image as its input. Each module then uses the resolution-preserving component to obtain a feature map with the same resolution as the module's input. Based on the module's input, the downsampling component then uses the downsampling component to obtain a map containing high-level features but with half the resolution. In the first module, the downsampled feature map only needs to be fused with the feature map output by the resolution-preserving component in that module through the feature fusion component. In the second, third, and fourth modules, the downsampled feature maps need to be fused with two sets of data: the feature map output by the resolution-preserving component of the current module and the feature map output by the resolution-preserving component of the first module. This continuous fusion of high-resolution features from the top layer and sub-high-resolution features from the adjacent upper layer enables the network to more accurately capture positional information features during feature extraction, thereby improving the accuracy and stability of network predictions.

[0073] The decoding layer consists of four layers, each of which includes an upsampling component and a feature concatenation component. The upsampling component consists of a transposed convolution with a kernel size of 4, a stride of 2, and a padding of 1. After the data passes through the upsampling component, the feature map size is doubled. The feature concatenation component concatenates the feature maps of the same size from the encoder with the feature maps after the upsampling component, ensuring more stable and accurate decoding.

[0074] After encoding and decoding, a feature map of the same size as the original input image is obtained. Finally, a convolution with a kernel size of 1 is used to convert the final feature map into n channels, where n represents the maximum number of key points to be detected, that is, one feature map is responsible for predicting one key point. In this embodiment, the target has the largest number of key points, which is 7, so n is also 7. Finally, the predicted value output by the network is compared with the true value of the annotation using the following loss function to calculate the loss:

[0075] ;

[0076] Where m represents the number of key points, Indicates the true The Gaussian distribution map corresponding to the key point pixel prediction values, The predicted The Gaussian distribution map corresponding to the key point The purpose of this loss function is to match an output feature map with a key point, calculate the difference between each pixel value of the feature map and the Gaussian distribution heat map corresponding to the key point, and finally take the average as the final loss for subsequent optimizer parameter updates.

[0077] Step 6: In the prediction phase, the target detector described above is first used to detect the location of the target in the image. Then, according to the image preprocessing method described above, the target is passed to the key point detection network to predict the Gaussian distribution heat map of each key point. The non-zero value in the heat map indicates that the key point predicted by the network is near this area, and the point with the largest value is the network predicted point. The scale factor of the predicted coordinate point is then calculated:

[0078] , ;

[0079] in represents the scale factor in the x direction, represents the scale factor in the y direction, represents the predicted x coordinate, Represents the predicted y coordinate, and 255 corresponds to the image size output to the key point detection network. Based on the above scale factor, the key point position in the original image is calculated:

[0080] , ;

[0081] in Represents the horizontal coordinate of the key point in the original image, Represents the vertical coordinate of the key point in the original image, Corresponding to the width of the original image, Corresponds to the height of the original image.

[0082] Step 7: After obtaining the key point coordinates predicted by the model, calculate the image gradient at the coordinates and the adjacent pixels. In this embodiment, the gradient values ​​at the predicted coordinates and the adjacent 8 pixels are calculated. The gradient calculation template is as follows:

[0083] , ;

[0084] in represents the x-direction template, I represents the corresponding image area, Represents the y-direction template, and the calculated gradient is as follows:

[0085] ;

[0086] Compare the gradient values ​​of the nine points and output the point with the largest gradient value as the final key point prediction point. Figure 2 、 Figure 3 As shown in the figure, according to the above steps, the key point positions of the fixed-wing aircraft and the quadcopter are predicted and visualized in the image. Figure 2 (a), (b), and (c) are the detection results of fixed-wing aircraft under normal viewing angle, partially blocked viewing angle, and flipped viewing angle, respectively. The number of key points is 5. Figure 3 (a), (b), and (c) are the detection results of the quadcopter under blurred perspective, partially blocked perspective, and normal good perspective, respectively, and the number of key points is 7. The present invention adds a resolution preservation component and a feature fusion component to the general convolutional network, ensuring that the network does not lose high-resolution features during feature extraction, thereby extracting more effective position codes. Furthermore, after the model completes key point prediction and calculates the image gradient of the predicted point and its surrounding adjacent pixels, the point where the gradient maximum is located is output as the final key point. Finally, through the longest alignment method, it is possible to complete multiple categories of key point detection in one network model, greatly improving the flexibility of the network.

[0087] Although the present invention is described herein with reference to illustrative embodiments of the present invention, the above embodiments are merely preferred embodiments of the present invention, and the embodiments of the present invention are not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and implementations, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A multi-category key point detection method based on fusion of high-resolution features, characterized in that: include: Step S100: collecting image data of the target to be detected to obtain image data samples; Step S200: annotating the bounding box and key point coordinates of the target to be detected in the image data sample to obtain bounding box data and key point data; Step S300: Determine whether the key point data requires longest alignment. If so, perform longest alignment and then convert the key point data after longest alignment into a Gaussian distribution map. Otherwise, directly convert the key point data into a Gaussian distribution map. The longest alignment method is as follows: Step S310: If key points of the same category are marked, the order of marking key points of each target must be consistent; Step S320: If key points of different categories are marked and the number of key points between categories is inconsistent, then the target with fewer key points needs to be marked in order and then padded with zeros at the end until the number of key points is padded to the target with the largest number of key points among all categories, thus completing the longest alignment; Step S400: using bounding box data to train an object detector, the object detector is used to predict the position information of the object to be detected by key points in the image to be detected; Step S500: passing the Gaussian distribution map as a label into the deep learning key point detection model for training, and obtaining the trained key point detection model; The deep learning key point detection model includes an encoder and a decoder. The encoder has four layers, each layer consists of a first module, the first module consists of a resolution preservation component, a downsampling component and a first feature fusion component; the resolution preservation component consists of a convolution layer, a BN layer, and an activation function layer; the downsampling component consists of a convolution layer, a BN layer, an activation function layer, and a pooling layer; the first feature fusion component consists of a convolution layer, a BN layer, an activation function layer, a pooling layer, and a channel splicing layer; in the first module of the first layer of the encoder, the first feature fusion component only needs to fuse the output of the resolution preservation component in the first module of the layer and the output features of the downsampling component. In the first modules of the subsequent three layers of the encoder, in addition to fusing the output of the resolution preservation component in the first module of the corresponding layer and the output features of the downsampling component, the first feature fusion component also fuses the output of the resolution preservation component in the first module of the first layer; The decoder consists of four layers, each layer consists of a second module, and each second module is divided into two parts: an upsampling component and a second feature fusion component; the upsampling component consists of a transposed convolution layer; the second feature fusion component includes a feature channel splicing layer, a convolution layer, a batch normalization layer, and an activation function. The second feature fusion component fuses the feature map output by the resolution preservation component in the first module of the same layer of the encoder with the feature map after the upsampling component in the second module of the same layer of the decoder to ensure that the decoder decodes in the correct direction; Step S600: Input the image to be detected into the trained object detector to obtain the position information of all objects in the image to be detected, cut out the position of the object in the image to be detected to obtain a sub-image, and if the size of the sub-image is not equal to the preset size, stretch or compress the sub-image to the preset size, and then input it into the trained key point detection model to obtain the key point coordinate information; Step S700: Based on the key point coordinate information predicted by the model, the image gradient of the key point and its adjacent pixels is calculated, and the point with the largest gradient change is regarded as the final key point.

2. The multi-category key point detection method based on fusion of high-resolution features according to claim 1, characterized in that: The method for determining whether the key point data requires longest alignment in step S300 is: when the number of key points of a certain category in the target to be detected is equal to the maximum number of key points in all categories, longest alignment is not required; otherwise, longest alignment is required.

3. The multi-category key point detection method based on fusion of high-resolution features according to claim 1, characterized in that: The method for converting key point data into a Gaussian distribution map is: Step A1: convert the coordinates of the key points into scale factors relative to the width and height of the image; Step A2: pre-fix the width and height of the target to be measured. If the obtained image is inconsistent with the preset width and height, stretch or compress the image to the preset value; Step A3: Generate an image with a channel of 1, a size of the preset value, and all pixel values ​​of 0; Step A4: Generate a two-dimensional Gaussian distribution map at the corresponding position of the image generated in step A3 using the scale factor. The corresponding function is: ； Where, Indicates the generation of a two-dimensional Gaussian distribution graph in The value at the coordinate, represents the amplitude, and Represents the center of the two-dimensional Gaussian distribution map, that is, the coordinates of the key points, and Represents the manually preset variance in the Gaussian function on the x-axis and y-axis respectively.

4. The multi-category key point detection method based on fusion of high-resolution features according to claim 1, characterized in that: The loss function of the deep learning key point detection model is defined as: ； Where m represents the maximum number of key points, and n represents the number of pixels output for each feature map; Indicates the true The Gaussian distribution map corresponding to the key point pixel values, The predicted The Gaussian distribution map corresponding to the key point pixel values.

5. The multi-category key point detection method based on fusion of high-resolution features according to claim 4 is characterized in that: The specific method for calculating the image gradient of the key point and its adjacent pixels in step S700 is: Gradient value calculation template in the x direction: ； Gradient value calculation template in the y direction: ； The gradient value corresponding to the pixel point: ； in, represents the x-direction template, I represents the corresponding image area, Indicates the y-direction template; 、 is the absolute value of the gradient in the x and y directions; Finally, the pixel with the largest gradient is regarded as the location of the final key point.

Images