Long jump action key point detection method and system based on multi-scale module

Abstract

The invention discloses a long jump action key point detection method and system based on a multi-scale module, and the method comprises the following steps: S1, obtaining a long jump image, and carrying out the preprocessing of the long jump image; s2, constructing an SRMpose network structure based on a multi-scale module; and S3, inputting the preprocessed long jump image into the SRMpose network structure, and outputting key point information through a non-maximum suppression algorithm. The SRMpose network structure adopts a PAN feature pyramid model structure, information of each dimension feature layer can be fully utilized, high-level features are fused to low-level features through fusion of information of different dimensions, the low-level feature expression ability is improved, the performance is improved, targets of different scales can be allocated to different layers for prediction, and the prediction efficiency is improved. Therefore, the image feature information is fully extracted, and the method can accurately detect the key points of the long jump action.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for detecting key points of a long jump action based on a multi-scale module. Background Art

[0002] Long jump is an item in daily training and physical examination for school students. Therefore, it is very important to obtain long jump results accurately.

[0003] Prior art CN218392171U discloses a standing long jump detection device, including a first side bar, a second side bar and a base pad, the first and second side bars are parallel, the two sides of the base pad are fixed on the first side bar and the second side bar, the long jump distance is detected by the memory foam on the pad, and the two bars are respectively provided with a laser transmitter and a laser receiver. The laser receiver is connected to the controller and the speaker, and whether the line is stepped on or not is determined by whether the laser is blocked by the students and the connected speaker, and the imprint of the memory foam is used to measure the long jump results.

[0004] The disadvantages of this technical solution are: (1) a large number of physical equipment such as the first side bar, the second side bar, the base pad and the laser transmitting and receiving device are used, which is costly, poor in portability and requires limited space for application; (2) the long jump distance is determined by memory foam, which may increase the test error or make the test impossible when the tester falls or makes other movements. (3) the staff checking the marks for scoring may lead to certain subjective errors and low detection efficiency.

[0005] Prior art CN111860107A discloses a standing long jump evaluation method based on deep learning posture estimation, including the following steps: using a convolutional neural network to build the overall module, completing the limb connection by identifying key point information, mainly divided into a front-end network and a back-end network, the front-end network is responsible for feature extraction using a VGG16 network, and the back-end network is responsible for identifying and connecting key points, only using convolutional layers and pooling layers for image feature extraction and training, and the overall network structure is built using a series connection. After obtaining information through the network, the initial position and the last standing position are used as the final score of the long jump.

[0006] The main disadvantages of this technical solution are: (1) The network structure adopts a serial structure, which lacks the use of multi-level information, and only uses convolution and pooling layers to build modules, lacking the relational extraction of feature information, resulting in a high number of parameters and calculations; (2) The score is calculated by the final location after landing, which will cause the tester to have an unstable standing position, and the final score error will be very large. (3) The VGG16 network has too many parameters and calculations, resulting in excessive weight parameters of subsequent convolution modules, which will cause difficulties in mobile migration and real-time detection projects. Summary of the invention

[0007] In view of the above problems, the present invention aims to provide a method and system for detecting key points of long jump action based on a multi-scale module.

[0008] The technical solution of the present invention is as follows:

[0009] On the one hand, a method for detecting key points of a long jump action based on a multi-scale module is provided, comprising the following steps:

[0010] S1: Acquire a long jump image and preprocess the long jump image;

[0011] S2: Construct SRMpose network structure based on multi-scale modules;

[0012] S3: Input the preprocessed long jump image into the SRMpose network structure, and output key point information through the non-maximum suppression algorithm.

[0013] Preferably, in step S1, the preprocessing includes data enhancement processing and data augmentation processing.

[0014] Preferably, the data augmentation processing includes any one or more of random rotation, shearing, scaling, color change, random movement and mirroring, and the data augmentation processing is stitching multiple images.

[0015] Preferably, in step S2, the SRMpose network structure includes a Conv module, a StarBlocks module, a SPPF module, a Seg_Attention module, a Concat module, an MRB module, a SRMhead module, an MRB_EMA module, a Dysample module, and an ADown module, five StarBlocks modules are provided, four Concat modules are provided, two MRB modules, two MRB_EMA modules, two Dysample modules, and two ADown modules are provided, and three SRMhead modules are provided;

[0016] The Conv module is sequentially connected to the first StarBlocks module, the second StarBlocks module, the third StarBlocks module, the fourth StarBlocks module, the fifth StarBlocks module, the SPPF module, the Seg_Attention module, the first Concat module, the first MRB module, and the first SRMhead module;

[0017] The second StarBlocks module is connected to the second Concat module, the first MRB_EMA module, and the second SRMhead module in sequence;

[0018] The third StarBlocks module is sequentially connected to the third Concat module, the second MRB_EMA module, the fourth Concat module, and the third SRMhead module;

[0019] The output end of the Seg_Attention module is also connected to the third Concat module through the first Dysample module, and the output end of the second MRB_EMA module is also connected to the second Concat module through the second Dysample module;

[0020] The output end of the first MRB_EMA module is also connected to the fourth Concat module through the first ADown module, and the output end of the fourth Concat module is also connected to the first Concat module through the second MRB module and the second ADown module connected in sequence.

[0021] Preferably, the StarBlocks module includes an input module, a DW-Conv module, a FC module and an output module, two DW-Conv modules are provided, and three FC modules are provided;

[0022] The output end of the input module is connected to the first DW-Conv module, the output end of the first DW-Conv module is connected to the parallel first FC module and the second FC module, the output feature matrices of the two parallel FC modules are added or matrix multiplied and then connected to the third FC module, the output end of the third FC module is connected to the second DW-Conv module, and the output end of the second DW-Conv module is added to the output feature matrix of the input module and then connected to the output module.

[0023] Preferably, the MRB module includes an input module, a Conv module with a convolution kernel of 1, a Conv module with a convolution kernel of 3, a Concat module and an output module, and four Conv modules with a convolution kernel of 3 are provided;

[0024] The output end of the input module inputs the input feature map of half the number of channels into two parallel residual structures, wherein one residual structure is composed of a Conv module with a first convolution kernel of 3, and the other residual structure is composed of a connected Conv module with a convolution kernel of 1 and a Conv module with a second convolution kernel of 3, and the output feature matrices of the two parallel residual structures are added and connected to the Concat module;

[0025] The output end of the input module inputs the input feature map with the other half of the number of channels to the third Conv module with a convolution kernel of 3, and the output end of the Conv module with a third convolution kernel of 3 is connected to the Concat module;

[0026] The output end of the input module is connected to the Conv module with a fourth convolution kernel of 3, and the output end of the Conv module with a fourth convolution kernel of 3, the output end of the input module, and the output feature matrix of the Concat module are added and connected to the output module.

[0027] Preferably, the SRMhead module includes a Detecthead module and a Posehead module;

[0028] The Detecthead module and the Posehead module both adjust features and reduce dimensions through the MulBlock module, and then use a layer of kernel=3 convolution to adjust the number of feature channels and the size of the feature map. The Detecthead module detects targets from top to bottom, and the Posehead module detects key points.

[0029] Preferably, the MulBlock module includes six Conv modules and one output module;

[0030] The six Conv modules are Conv module 1 with kernel=3, Conv module 2 with kernel=3 and padding=1, Conv module 3 with kernel=3 and padding=3, Conv module 4 with kernel=3 and padding=5, Conv module 5, and Conv module 6;

[0031] The output end of the Conv module one is connected to the Conv module two, Conv module three, and Conv module four respectively. The output features of the Conv module four are input into the Conv module five together with the output features of the Conv module three after BI bilinear interpolation. The output features of the Conv module five are input into the Conv module six together with the output features of the Conv module two after BI bilinear interpolation. The output feature matrices of the Conv module one, Conv module five, and Conv module six are added and input into the output module.

[0032] Preferably, in step S3, the non-maximum suppression algorithm adopts any one of the basic NMS algorithm, the Soft-NMS algorithm, the Softer-NMS algorithm, and the IoU-Net.

[0033] On the other hand, a long jump action key point detection system based on a multi-scale module is also provided, comprising:

[0034] A data acquisition and preprocessing module, used for acquiring long jump images and preprocessing the long jump images;

[0035] SRMpose network structure construction module, used to build SRMpose network structure based on multi-scale modules;

[0036] The key point detection module is used to input the preprocessed long jump image into the SRMpose network structure and output the key point information through the non-maximum suppression algorithm.

[0037] The beneficial effects of the present invention are:

[0038] The SRMpose network structure described in the present invention adopts the PAN feature pyramid model structure, which can make full use of the information of each dimensional feature layer, and through the fusion of different dimensional information, the high-level features are integrated into the low-level features, thereby increasing the expression ability of the low-level features and improving the performance. Targets of different scales can be assigned to different layers for prediction, thereby achieving full extraction of image feature information, so that the present invention can accurately detect the key points of the long jump action. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0040] Figure 1 It is a structural diagram of the SRMpose network structure of the present invention;

[0041] Figure 2 It is a schematic diagram of the structure of the StarBlocks module of the present invention;

[0042] Figure 3 It is a structural schematic diagram of the MRB module of the present invention;

[0043] Figure 4 It is a structural schematic diagram of the SRMhead module of the present invention;

[0044] Figure 5 It is a structural schematic diagram of the MulBlock module of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, in the absence of conflict, the embodiments in this application and the technical features in the embodiments can be combined with each other. It should be noted that, unless otherwise specified, all technical and scientific terms used in this application have the same meanings as those commonly understood by those of ordinary skill in the art to which this application belongs. The words "including" or "comprising" and the like used in the disclosure of the present invention mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0046] like Figure 1 As shown, the present invention provides a method and system for detecting key points of long jump action based on a multi-scale module, comprising the following steps:

[0047] S1: Acquire a long jump image, and preprocess the long jump image.

[0048] In a specific embodiment, the preprocessing includes data enhancement processing and data augmentation processing. Optionally, the data enhancement processing includes any one or more of random rotation, shearing, scaling, color change, random movement and mirroring, and the data augmentation processing is splicing multiple images.

[0049] S2: Construct the SRMpose network structure based on the multi-scale module.

[0050] In a specific embodiment, Figure 1 As shown, the SRMpose network structure includes Conv module, StarBlocks module, SPPF module, Seg_Attention module, Concat module, MRB module, SRMhead module, MRB_EMA module, Dysample module, and ADown module. Five StarBlocks modules are set, four Concat modules are set, two MRB modules, MRB_EMA modules, Dysample modules, and ADown modules are set, and three SRMhead modules are set;

[0051] The Conv module is sequentially connected to the first StarBlocks module, the second StarBlocks module, the third StarBlocks module, the fourth StarBlocks module, the fifth StarBlocks module, the SPPF module, the Seg_Attention module, the first Concat module, the first MRB module, and the first SRMhead module;

[0052] The second StarBlocks module is connected to the second Concat module, the first MRB_EMA module, and the second SRMhead module in sequence;

[0053] The third StarBlocks module is sequentially connected to the third Concat module, the second MRB_EMA module, the fourth Concat module, and the third SRMhead module;

[0054] The output end of the Seg_Attention module is also connected to the third Concat module through the first Dysample module, and the output end of the second MRB_EMA module is also connected to the second Concat module through the second Dysample module;

[0055] The output end of the first MRB_EMA module is also connected to the fourth Concat module through the first ADown module, and the output end of the fourth Concat module is also connected to the first Concat module through the second MRB module and the second ADown module connected in sequence.

[0056] In a specific embodiment, Figure 2 As shown, the StarBlocks module includes an input module, a DW-Conv module, a FC module and an output module, two DW-Conv modules are provided, and three FC modules are provided;

[0057] The output end of the input module is connected to the first DW-Conv module, the output end of the first DW-Conv module is connected to the parallel first FC module and the second FC module, the output feature matrices of the two parallel FC modules are added or matrix multiplied and then connected to the third FC module, the output end of the third FC module is connected to the second DW-Conv module, and the output end of the second DW-Conv module is added to the output feature matrix of the input module and then connected to the output module.

[0058] In a specific embodiment, Figure 3 As shown, the MRB module includes an input module, a Conv module with a convolution kernel of 1, a Conv module with a convolution kernel of 3, a Concat module, and an output module, and four Conv modules with a convolution kernel of 3 are provided;

[0059] The output end of the input module inputs the input feature map of half the number of channels into two parallel residual structures, wherein one residual structure is composed of a Conv module with a first convolution kernel of 3, and the other residual structure is composed of a connected Conv module with a convolution kernel of 1 and a Conv module with a second convolution kernel of 3, and the output feature matrices of the two parallel residual structures are added and connected to the Concat module;

[0060] The output end of the input module inputs the input feature map with the other half of the number of channels to the third Conv module with a convolution kernel of 3, and the output end of the Conv module with a third convolution kernel of 3 is connected to the Concat module;

[0061] The output end of the input module is connected to the Conv module with a fourth convolution kernel of 3, and the output end of the Conv module with a fourth convolution kernel of 3, the output end of the input module, and the output feature matrix of the Concat module are added and connected to the output module.

[0062] In a specific embodiment, Figure 4 As shown, the SRMhead module includes a Detecthead module and a Posehead module;

[0063] The Detecthead module and the Posehead module both adjust features and reduce dimensions through the MulBlock module, and then use a layer of kernel=3 convolution to adjust the number of feature channels and the size of the feature map. The Detecthead module detects targets from top to bottom, and the Posehead module detects key points.

[0064] In a specific embodiment, Figure 5 As shown, the MulBlock module includes six Conv modules and one output module;

[0065] The six Conv modules are Conv module 1 with kernel=3, Conv module 2 with kernel=3 and padding=1, Conv module 3 with kernel=3 and padding=3, Conv module 4 with kernel=3 and padding=5, Conv module 5, and Conv module 6;

[0066] The output end of the Conv module one is connected to the Conv module two, Conv module three, and Conv module four respectively. The output features of the Conv module four are input into the Conv module five together with the output features of the Conv module three after BI bilinear interpolation. The output features of the Conv module five are input into the Conv module six together with the output features of the Conv module two after BI bilinear interpolation. The output feature matrices of the Conv module one, Conv module five, and Conv module six are added and input into the output module.

[0067] In a specific embodiment, the SRMpose network structure described in the above embodiment is established by the following steps:

[0068] (1) StarBlock is used to build a backbone network structure, which can extract high-dimensional feature information while keeping it lightweight and improving efficiency.

[0069] To build a branch backbone network of the PAN structure, the module structure is built in series, where the Star operation is as follows:

[0070] Star(X)=(W1X)*(W2X) (1)

[0071] Where: X represents the input feature image; W represents the weight information that maps the feature information to high-dimensional information; subscripts 1 and 2 represent two channels.

[0072] The input of the Star operation is the two channels of the network. Each channel initially inputs X, but each channel has a weight W to be learned (used to adjust the dimension of the feature map). The subscripts 1 and 2 are used to distinguish the weights on the two channels.

[0073] The backbone network requires multiple layers of feature maps for feature fusion of the PAN structure. The present invention is mainly divided into five stages. The specific structure is shown in Figure 2 , using stage2, stage3, stage4, and stage5 as the subsequent feature map output. DW-Conv stands for depthwise separable convolution, FC stands for linear layer, and sum or star means that the operation can choose whether to multiply or add elements. Finally, the StarBlocks module is established as follows Figure 2 shown.

[0074] (2) Design the RMB module structure of the feature extraction network of the intermediate network to combine multi-scale feature information and feature channel information to improve the network's comprehensive sensitivity to feature information.

[0075] In order to further adjust and fuse features, the MRB (Multi-channel Residual Block) module is built to extract multi-scale features, and combined with channel segmentation, the feature extraction of different parts of the channel is improved. The overall residual structure is fine-tuned to improve the model's expressiveness. The module mainly uses convolution with a kernel of 3 to extract features and convolution with a kernel of 1 to adjust the number of channels. Finally, the channel adjustment structure and the kernel residual structure are combined to output the results.

[0076] like Figure 3 As shown in the figure, the upper part is half the number of channels of the input feature map entering the two residual structures above the convolution, extracting feature information through the convolution kernel of kernel=3, adjusting the number of channels through the convolution of kernel=1 and kernel=3, and finally outputting the final feature R1 of the upper part through the residual network.

[0077]

[0078] R1(X1)=C(X1,k=3)+C(C(X1,k=1),k=3) (3)

[0079] The middle part is the other half of the channels. After extracting features through the convolution kernel of kernel=3, R2 is directly output and concatenated with the feature map above by Concat channel number.

[0080] R2(X2)=C(X2,k=3) (4)

[0081] The lower part is the output feature map passed through the residual network with kernel=3, and the feature map spliced ​​in the middle part is output element by element as the sum O, which improves the generality of the network model.

[0082] R3(X)=Concat((R1(X1),R2(X2)),dim=1) (5)

[0083] O(X)=X+C(X,k=3)+R3(X) (6)

[0084] Assume that the input tensor is X and the output is O(X), where C(X,k) is the convolution operation on X, k is the convolution kernel scale, and Concat((X,Y), dim) means combining the X and Y tensors in the dim dimension.

[0085] (3) Design the SRMhead module structure of the back-end network detection head, improve the network operation efficiency by designing a semi-decoupled head, optimize the overall network structure and combine the designed MulBlock to extract the final multi-scale information.

[0086] The detection head adopts a top-down detection method and is divided into target detection Detecthead and key point detection head Posehead. For the feature adjustment part of the detection head, a semi-coupled head Detecthead is used. The features are adjusted by designing the MulBlock (Multi-scale Block) module, and the features are adjusted by 3*3 convolution (the default convolution kernel k=1 in the figure). For features at different levels, feature information of different receptive fields is further extracted to make full use of multi-scale feature information. In order to lighten the network and improve the extraction of multi-scale information, a semi-decoupled head SRMhead is designed in combination with MulBlock.

[0087] like Figure 5 As shown in the figure, the MulBlock module first uses kernel=3 convolution to perform preliminary feature adjustment and extraction, and then passes through four layers including three transformed feature maps, through kernel=3, padding 1, 3, and 5 convolution layers, the resulting feature map sizes are ×1, ×0.5, and ×0.25 respectively. The three adjusted scale information is converted from low-resolution feature maps to ×1 through BI (BilinearInterpol) bilinear interpolation, which not only improves the generalization of the network, but also requires lower model parameters and computational complexity.

[0088] After that, the first and second layers adjust the number of feature map channels through the convolution layer with kernel=1, and then output the final features by summing up each element.

[0089] The SRMhead module can output target position probability information, key point position and probability information, and use deep learning to obtain key weight information in convolution. The formulas involved in target position information, probability information, and coupled key point probability information and probability information are as follows:

[0090] X=C(X0,k=1,p=0) (7)

[0091] f1(X,k1,p1)=BI(C(X,k1,p1)) (8)

[0092] f2(X,k2,p2)=BI(C(C(X,k2,p2)+f1(X,k1,p1),k=1)) (9)

[0093] M1(X,k3,p3)=X+C(C(X,k3,p3)+f2(X,k2,p2),k=1) (10)

[0094] b(X,k3,p3)=C1(C(M1(X,k3,p3),k=3),k=1) (11)

[0095] c(X,k3,p3)=C2(C(M1(X,k3,p3),k=3),k=1) (12)

[0096] X1=C(M1(X,k3,p3),k=3) (13)

[0097] k(X1)=C(M2(X1,k4,p4),k=3) (14)

[0098] Assume that the input is X0, BI(X) performs bilinear interpolation on X, and C(X,k,p) performs convolution on X, where the convolution kernel is k and p indicates that the step size of the convolution is p.

[0099] (4) Build the overall PAN network structure, combine the backbone network and the intermediate network with the back-end network, and reasonably adjust the structure of information interaction to improve performance while ensuring efficiency, and finally build the SRMpose network structure.

[0100] The Path Aggregation Network (PAN) is used as the overall model structure design. The network's full-text information feature extraction for images is improved, context information is integrated, and the StarNet network is used as the backbone network to extract image features. The MRB module is designed to extract features, and the MRB_EMA module is combined with the attention mechanism EMA to combine context information. The upsampling operator Dysample and the downsampling operator ADown module are introduced. A semi-coupled head SRMhead is built, which only decouples the probability information and coordinate information, effectively reducing the number of parameters and the amount of calculation. The MulBlock module is designed for the output head part. The final SRMpose network structure is as follows: Figure 1 shown.

[0101] Figure 1 The SRMpose network structure shown is a deep learning network structure, which is mainly built with the convolution structure Conv. The SPPF (Serial Parallel Pooling Fusion) module is used to realize the fusion of local features and global features. The Seg_Attention module is an attention mechanism module that can improve the model's attention to feature extraction. Dysample and ADown are upsampling and downsampling modules, respectively, which mainly adjust the width of the feature map. Concat is a connection module that splices the second dimension of the two feature map tensors. MRB_EMA is to connect the EMA attention mechanism after the MRB structure.

[0102] In the above embodiment, the overall network of the SRMpose network structure described in the present invention adopts the PAN feature pyramid model structure, which makes full use of the information of each dimensional feature layer, and integrates high-level features into low-level features through the fusion of information of different dimensions, thereby increasing the expression ability of low-level features and improving performance. Targets of different scales can be assigned to different layers for prediction, thereby achieving full extraction of image feature information.

[0103] The backbone network is built with stacked StarBlocks. StarBlock has the ability to map inputs to extremely high-dimensional nonlinear feature spaces, thereby explaining the strong expressive power of star operations. It is more efficient in feature extraction, and the model is more compact, which is suitable for lightweight models.

[0104] Design the core feature extraction module. Design the MRB (Multi-channel Residual Block) module. By extracting feature information at multiple scales, the detailed information can be extracted as much as possible, and combined with channel segmentation, the feature extraction of different parts of the channel can be improved. Fine-tune the overall residual structure to improve the model's expressiveness. The module mainly uses convolution with a kernel of 3 to extract features and convolution with a kernel of 1 to adjust the number of channels. Finally, the channel adjustment structure and the kernel residual structure are combined to output the result.

[0105] Design the SRMhead detection head module, adjust the features by designing the MulBlock (Multi-scale Block) module, adjust the features through 3*3 convolution, further extract the feature information of different receptive fields for features at different levels, and make full use of multi-scale feature information. In order to lightweight the network and improve the extraction of multi-scale information, combine MulBlock to design the semi-decoupled head SRMhead and decoupled head DSRMhead.

[0106] The SRMpose network structure of the present invention is obtained by combining the above-mentioned module structure. Through the SRMpose network structure, the key point extraction method of the present invention can achieve better performance with low parameter quantity and calculation amount.

[0107] S3: Input the preprocessed long jump image into the SRMpose network structure, and output key point information through the non-maximum suppression algorithm.

[0108] In a specific embodiment, the non-maximum suppression algorithm adopts any one of the basic NMS algorithm, the Soft-NMS algorithm, the Softer-NMS algorithm, and the IoU-Net. It should be noted that each algorithm is a prior art, and the specific content is not repeated here. The key point information includes coordinate information and probability information.

[0109] On the other hand, the present invention also provides a long jump action key point detection system based on a multi-scale module, comprising:

[0110] A data acquisition and preprocessing module, used for acquiring long jump images and preprocessing the long jump images;

[0111] SRMpose network structure construction module, used to build SRMpose network structure based on multi-scale modules;

[0112] The key point detection module is used to input the preprocessed long jump image into the SRMpose network structure and output the key point information through the non-maximum suppression algorithm.

[0113] In a specific embodiment, the long jump action key point detection method based on the multi-scale module of the present invention is used to perform long jump action key point detection.

[0114] In this embodiment, the training parameters are: epoch: 500, warm-up times: 3, and an early stopping mechanism is provided. By judging the performance of the model on the validation set, it is determined whether convergence has been achieved and training is stopped, which effectively avoids the problem of overfitting. Set the optimizer: SGD, batchsize: 48, lr0: 0.01, lrf: 0.01, final learning rate: 0.0001, momentum parameter: 0.937. The loss function weights are λbox: 7.5, λcls: 0.5, λkpts: 12, λkptc: 1. The input image size is 640×640, and the IOU threshold for True is set to 0.7. The experimental environment is shown in Table 1:

[0115] Table 1 Experimental environment

[0116]

[0117] In this embodiment, mAP50, mAP50:95, parameter count, GFLOPs and inference time are used as evaluation indicators of the model. AP is the area enclosed by the PR curve (Presession, Recall) and the coordinate axis, and the OKS threshold is used to define the correct sample through the confusion matrix. The specific definition is shown in Table 2:

[0118] Table 2 Confusion matrix

[0119]

[0120] The precision rate can be used to measure the accuracy of the model's prediction, while the recall rate measures the probability of the model identifying the correct target. AP is calculated by the following formula:

[0121]

[0122] The average mAP of multiple people in the dataset is taken as the mAPn value, where n∈(0,1) is the OKS threshold for controlling positive and negative samples. The model performance is comprehensively judged by combining the model's inference time for each image, Inf (Inference), excluding pre-processing and post-processing time, and the model calculation amount (GFLOPs). Both inference and calculation are performed on the GPU.

[0123] In order to verify the effectiveness of the SRMpose network structure of the present invention, the performance is compared with the current mainstream key point model on the COCO and LJD datasets. The model is evaluated mainly by the number of parameters, GFLOPs and the performance indicator mAP50. The default input size is 640*640. The experimental results are shown in Tables 3 and 4:

[0124] Table 3 Comparison experiment table of different models (COCO)

[0125]

[0126] Table 4 Comparison of different data sets and model scales

[0127]

[0128] It can be seen from Table 3 that the SRMpose network structure of the present invention has improved 1.9 and 1.2 points respectively in the COCO dataset while having basically the same number of parameters as the current mainstream key point recognition algorithm yolov8-pose.

[0129] It can be seen from Table 4 that the SRMpose network structure of the present invention has a mAP50 value that is 1.4 points different from that of the benchmark model Yolov8s when the number of parameters is reduced by about 50%. Considering the comprehensive number of parameters, calculation amount and performance, the model effectively achieves lightweight and maintains a high detection accuracy.

[0130] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting key points of long jump action based on a multi-scale module, comprising the following steps: S1: Acquire a long jump image and preprocess the long jump image; S2: Construct SRMpose network structure based on multi-scale modules; S3: Input the preprocessed long jump image into the SRMpose network structure, and output key point information through the non-maximum suppression algorithm.

2. The long jump action key point detection method based on multi-scale module according to claim 1 is characterized in that: In step S1, the preprocessing includes data enhancement processing and data augmentation processing.

3. The long jump action key point detection method based on multi-scale module according to claim 2 is characterized in that: The data augmentation processing includes any one or more of random rotation, shearing, scaling, color change, random movement and mirroring, and the data augmentation processing is splicing multiple images.

4. The long jump action key point detection method based on multi-scale module according to claim 1 is characterized in that: In step S2, the SRMpose network structure includes a Conv module, a StarBlocks module, a SPPF module, a Seg_Attention module, a Concat module, an MRB module, a SRMhead module, an MRB_EMA module, a Dysample module, and an ADown module, wherein five StarBlocks modules are provided, four Concat modules are provided, two MRB modules, two MRB_EMA modules, two Dysample modules, and two ADown modules are provided, and three SRMhead modules are provided; The Conv module is sequentially connected to the first StarBlocks module, the second StarBlocks module, the third StarBlocks module, the fourth StarBlocks module, the fifth StarBlocks module, the SPPF module, the Seg_Attention module, the first Concat module, the first MRB module, and the first SRMhead module; The second StarBlocks module is connected to the second Concat module, the first MRB_EMA module, and the second SRMhead module in sequence; The third StarBlocks module is sequentially connected to the third Concat module, the second MRB_EMA module, the fourth Concat module, and the third SRMhead module; The output end of the Seg_Attention module is also connected to the third Concat module through the first Dysample module, and the output end of the second MRB_EMA module is also connected to the second Concat module through the second Dysample module; The output end of the first MRB_EMA module is also connected to the fourth Concat module through the first ADown module, and the output end of the fourth Concat module is also connected to the first Concat module through the second MRB module and the second ADown module connected in sequence.

5. The long jump action key point detection method based on multi-scale module according to claim 4 is characterized in that: The StarBlocks module includes an input module, a DW-Conv module, a FC module and an output module, two DW-Conv modules are provided, and three FC modules are provided; The output end of the input module is connected to the first DW-Conv module, the output end of the first DW-Conv module is connected to the parallel first FC module and the second FC module, the output feature matrices of the two parallel FC modules are added or matrix multiplied and then connected to the third FC module, the output end of the third FC module is connected to the second DW-Conv module, and the output end of the second DW-Conv module is added to the output feature matrix of the input module and then connected to the output module.

6. The long jump action key point detection method based on multi-scale module according to claim 4 is characterized in that: The MRB module includes an input module, a Conv module with a convolution kernel of 1, a Conv module with a convolution kernel of 3, a Concat module, and an output module, wherein four Conv modules with a convolution kernel of 3 are provided; The output end of the input module inputs the input feature map of half the number of channels into two parallel residual structures, wherein one residual structure is composed of a Conv module with a first convolution kernel of 3, and the other residual structure is composed of a connected Conv module with a convolution kernel of 1 and a Conv module with a second convolution kernel of 3, and the output feature matrices of the two parallel residual structures are added and connected to the Concat module; The output end of the input module inputs the input feature map with the other half of the number of channels to the third Conv module with a convolution kernel of 3, and the output end of the Conv module with a third convolution kernel of 3 is connected to the Concat module; The output end of the input module is connected to the Conv module with a fourth convolution kernel of 3, and the output end of the Conv module with a fourth convolution kernel of 3, the output end of the input module, and the output feature matrix of the Concat module are added and connected to the output module.

7. The long jump action key point detection method based on multi-scale module according to claim 4 is characterized in that: The SRMhead module includes a Detecthead module and a Posehead module; The Detecthead module and the Posehead module both adjust features and reduce dimensions through the MulBlock module, and then use a layer of kernel=3 convolution to adjust the number of feature channels and the size of the feature map. The Detecthead module detects targets from top to bottom, and the Posehead module detects key points.

8. The long jump action key point detection method based on multi-scale module according to claim 7 is characterized in that: The MulBlock module includes six Conv modules and one output module; The six Conv modules are Conv module 1 with kernel=3, Conv module 2 with kernel=3 and padding=1, Conv module 3 with kernel=3 and padding=3, Conv module 4 with kernel=3 and padding=5, Conv module 5, and Conv module 6; The output end of the Conv module one is connected to the Conv module two, Conv module three, and Conv module four respectively. The output features of the Conv module four are input into the Conv module five together with the output features of the Conv module three after BI bilinear interpolation. The output features of the Conv module five are input into the Conv module six together with the output features of the Conv module two after BI bilinear interpolation. The output feature matrices of the Conv module one, Conv module five, and Conv module six are added and input into the output module.

9. The long jump action key point detection method based on multi-scale module according to any one of claims 1 to 8, characterized in that: In step S3, the non-maximum suppression algorithm adopts any one of the basic NMS algorithm, the Soft-NMS algorithm, the Softer-NMS algorithm, and the IoU-Net.

10. A long jump action key point detection system based on multi-scale module, characterized in that: include: A data acquisition and preprocessing module, used for acquiring long jump images and preprocessing the long jump images; SRMpose network structure construction module, used to build SRMpose network structure based on multi-scale modules; The key point detection module is used to input the preprocessed long jump image into the SRMpose network structure and output the key point information through the non-maximum suppression algorithm.

Images