Detection model training method, detection method, device, equipment and medium

Abstract

The embodiment of the present invention discloses a training method, a detection method, an apparatus, a device and a medium for a detection model. The method comprises: training the detection model using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof; for each training sample, determining a first Euclidean distance between a head key point predicted by the detection model and a head key point marked in the training sample, and a second Euclidean distance between a predicted shoulder key point and a shoulder key point marked in the training sample; normalizing the predicted shoulder key point, and determining a probability value corresponding to the head key point corresponding to the normalized shoulder in a priori probability distribution of the head; calculating a weighted loss value corresponding to the head key point according to the probability value, the first Euclidean distance and the second Euclidean distance, and when the weighted loss value reaches convergence, the detection model training is completed; by adopting the above technical solution, the detection accuracy of the head and shoulder key points is effectively improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of target detection technology, and in particular, to a training method, detection method, device, equipment and medium for a head and shoulder key point detection model. Background Art

[0002] Object detection is a classic research topic in the field of computer vision. Its task is to point out the exact location of an object in an image. It has a wide range of applications in areas such as unmanned driving and crowd flow statistics analysis. In recent years, thanks to the introduction of deep convolutional networks, object detection algorithms have developed rapidly, bringing detection performance to a new level.

[0003] At present, there are generally two solutions for detecting key points of the head and shoulders in images: one is to first detect local straight line segments based on traditional image processing methods, and then search for key points of the head and shoulders through heuristic criteria. The other is the human key point detection method in human posture estimation. This type of method usually uses CNN to extract features and predict the heat map of key points, and then establishes the connection relationship between key points through the method of graph model.

[0004] However, the head and shoulder key point detection method based on traditional image processing relies on the accuracy and recall rate of straight line segment detection, which is easily affected by lighting and image quality and may cause serious degradation. In addition, the rule-based judgment method can retrieve multiple head and shoulder key points that meet the rules, so it is easy to cause false alarms. The human body key point detection method based on deep learning has a large number of key points to be estimated and the lower body key points are easily affected by perspective and posture, so the estimation is difficult and the accurate position of the head and shoulder key points in the image cannot be obtained. Summary of the invention

[0005] The embodiments of the present invention provide a training method, a detection method, an apparatus, a device and a medium for a head and shoulder key point detection model, which effectively improves the detection accuracy of the head and shoulder key points.

[0006] In a first aspect, the present invention provides a method for training a head and shoulder key point detection model, the method comprising:

[0007] The detection model is trained using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof;

[0008] During the training process, for each training sample, determining a first Euclidean distance between the head key points predicted by the detection model and the head key points annotated in the training sample, and a second Euclidean distance between the shoulder key points predicted by the detection model and the shoulder key points annotated in the training sample;

[0009] Normalizing the predicted shoulder key points, and determining the probability value of the head key points corresponding to the normalized shoulders in the head prior probability distribution; wherein the shoulder normalization makes the angle between the line connecting the two shoulder key points and the horizontal be 0 degrees when the face is facing backward, and the line connecting the two shoulder key points is the width of the image;

[0010] Calculate the weighted loss value corresponding to the head key point according to the probability value, the first Euclidean distance and the second Euclidean distance, and when the weighted loss value reaches convergence, the detection model training is completed;

[0011] According to the probability value, the first Euclidean distance and the second Euclidean distance, the weighted loss value corresponding to the head key point is calculated by the following loss function:

[0012] L=(1-p+0.5)*L h +0.5*L lr ;

[0013] Where L represents the weighted loss value; p represents the probability value of the corresponding head key point in the head prior probability distribution after the predicted shoulder key point is normalized; L h represents the first Euclidean distance, L lr represents the second Euclidean distance;

[0014] The prior probability distribution of the head is obtained in the following way:

[0015] The calibrated shoulder key points in the training sample are normalized, and a head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders. By normalizing the head key point statistical matrix, the prior probability distribution of the head key points is obtained.

[0016] Optionally, the symmetric feature map is obtained in the following manner:

[0017] Extract features from the original sample image based on the detection model to obtain a plurality of original feature maps including the head and shoulders;

[0018] The feature matrix corresponding to the last layer of original feature map obtained by the detection model is flipped upside down and flipped left side to side to obtain a symmetrical feature map symmetrical to the last layer of original feature map; wherein the upside down flipping operation is to swap the i-th row data with the hi-th row of the feature matrix; the left side to right flipping operation is to swap the i-th column data with the wi-th column data of the feature matrix;

[0019] The width and height of the image are w and h respectively.

[0020] Optionally, forming a head key point statistical matrix according to the coordinates of the head key points corresponding to the normalized shoulders includes:

[0021] Establishing an initial probability distribution matrix with a value of 0, wherein the number of rows in the initial probability distribution matrix is ​​the same as the number of image rows; and the number of columns in the initial probability distribution matrix is ​​the same as the number of image columns;

[0022] For any head key point, determine the coordinates of the head key point after shoulder normalization, and add one to the value of the corresponding position in the probability distribution matrix according to the coordinates until the head key point statistical matrix corresponding to all head key points is accumulated.

[0023] Optionally, after obtaining the head key point statistical matrix, the method further includes:

[0024] Gaussian filtering is performed on the head key point statistical matrix based on a preset convolution Gaussian blur kernel; wherein the preset convolution Gaussian blur kernel is:

[0025]

[0026] Correspondingly, the statistical matrix of the head key points after Gaussian filtering is normalized to obtain the prior probability distribution of the head key points.

[0027] In a second aspect, an embodiment of the present invention further provides a method for detecting head and shoulder key points, the method comprising:

[0028] Acquire an image to be tested including the head and shoulders;

[0029] Inputting the image to be tested into the trained detection model to obtain the position information of the key points of the head and the key points of the shoulders in the image to be tested;

[0030] The detection model is trained by the training method of the head and shoulder key point detection model provided by any embodiment of the present invention to establish an association relationship between the head key points and shoulder key points in the image to be tested and the corresponding position information.

[0031] In a third aspect, an embodiment of the present invention further provides a training device for a head and shoulder key point detection model, the device comprising:

[0032] The distance calculation module is configured to train the detection model using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof; during the training process, for each training sample, determine a first Euclidean distance between a head key point predicted by the detection model and a head key point annotated in the training sample, and a second Euclidean distance between a shoulder key point predicted by the detection model and a shoulder key point annotated in the training sample;

[0033] A probability value determination module is configured to normalize the predicted shoulder key points and determine the probability values ​​of the head key points corresponding to the normalized shoulders in the head prior probability distribution; wherein the shoulder normalization makes the angle between the line connecting the two shoulder key points and the horizontal be 0 degrees when the face is facing backward, and the line connecting the two shoulder key points is the width of the image;

[0034] A weighted loss value calculation module is configured to calculate a weighted loss value corresponding to the head key point according to the probability value, the first Euclidean distance and the second Euclidean distance, and when the weighted loss value reaches convergence, the detection model training is completed;

[0035] According to the probability value, the first Euclidean distance and the second Euclidean distance, the weighted loss value corresponding to the head key point is calculated by the following loss function:

[0036] L=(1-p+0.5)*L h +0.5*L lr ;

[0037] Where L represents the weighted loss value; p represents the probability value of the corresponding head key point in the head prior probability distribution after the predicted shoulder key point is normalized; L h represents the first Euclidean distance, L lr represents the second Euclidean distance;

[0038] The prior probability distribution of the head is obtained in the following way:

[0039] The calibrated shoulder key points in the training sample are normalized, and a head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders. By normalizing the head key point statistical matrix, the prior probability distribution of the head key points is obtained.

[0040] Optionally, the symmetric feature map is obtained in the following manner:

[0041] Extract features from the original sample image based on the detection model to obtain a plurality of original feature maps including the head and shoulders;

[0042] The feature matrix corresponding to the last layer of original feature map obtained by the detection model is flipped upside down and flipped left side to side to obtain a symmetrical feature map symmetrical to the last layer of original feature map; wherein the upside down flipping operation is to swap the i-th row data with the hi-th row of the feature matrix; the left side to right flipping operation is to swap the i-th column data with the wi-th column data of the feature matrix;

[0043] The width and height of the image are w and h respectively.

[0044] Optionally, the head prior probability distribution is obtained in the following manner:

[0045] Establishing an initial probability distribution matrix with a value of 0, wherein the number of rows in the initial probability distribution matrix is ​​the same as the number of image rows; and the number of columns in the initial probability distribution matrix is ​​the same as the number of image columns;

[0046] The calibrated shoulder key points in the training sample are normalized, and the coordinates of the head key points after shoulder normalization are determined, and the values ​​of the corresponding positions in the probability distribution matrix are increased by one according to the coordinates until the head key point statistical matrix corresponding to all head key points is accumulated.

[0047] Optionally, the device further comprises:

[0048] After obtaining the head key point statistical matrix, Gaussian filtering is performed on the head key point statistical matrix based on a preset convolution Gaussian blur kernel; wherein the preset convolution Gaussian blur kernel is:

[0049]

[0050] Correspondingly, the statistical matrix of the head key points after Gaussian filtering is normalized to obtain the prior probability distribution of the head key points.

[0051] In a fourth aspect, an embodiment of the present invention further provides a device for detecting head and shoulder key points, the device comprising:

[0052] The test image acquisition module is configured to acquire the test image including the head and shoulders;

[0053] A key point position determination module is configured to input the image to be tested into a trained detection model to obtain positions of the head key points and the shoulder key points in the image to be tested;

[0054] The detection model is trained by the training method of the head and shoulder key point detection model provided by any embodiment of the present invention to establish an association relationship between the head key points and shoulder key points in the image to be tested and the corresponding position information.

[0055] In a fifth aspect, an embodiment of the present invention further provides a computing device, including:

[0056] A memory storing executable program code;

[0057] a processor coupled to the memory;

[0058] The processor calls the executable program code stored in the memory to execute the training method of the head and shoulder key point detection model provided by any embodiment of the present invention.

[0059] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for the head and shoulder key point detection model provided by any embodiment of the present invention.

[0060] In a seventh aspect, an embodiment of the present invention further provides a computing device, including:

[0061] A memory storing executable program code;

[0062] a processor coupled to the memory;

[0063] The processor calls the executable program code stored in the memory to execute the head and shoulder key point detection method provided by any embodiment of the present invention.

[0064] In an eighth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the head and shoulder key point detection method provided by any embodiment of the present invention.

[0065] The technical solution provided by the embodiment of the present invention adopts a symmetrical feature map that is symmetrical to the original feature map during the training process of the detection model, increases the feature distribution pattern, increases the diversity of samples, and does not increase the network calculation complexity during forward reasoning. In addition, this embodiment proposes a new loss function for measuring the prediction results based on the structural relationship and relative position relationship of the distribution of the head and shoulder key points. According to the positional relationship of the head key points relative to the two shoulder key points in the sample, a normalized probability distribution map of the head key points is obtained by statistics. In the new loss function measurement, the loss of the head key points is weighted according to this probability distribution map, which can effectively avoid the phenomenon of outrageous prediction errors and effectively improve the detection accuracy of the head and shoulder key points.

[0066] The innovative features of the embodiments of the present invention include:

[0067] 1. During the detection model training process, a symmetrical feature map that is symmetrical to the original feature map is used, which increases the feature distribution pattern and the diversity of samples, which is one of the innovations of the embodiment of the present invention.

[0068] 2. According to the positional relationship between the head key points and the two shoulder key points in the sample, a normalized probability distribution map of the head key points is obtained. In the new loss function measurement, the head key point loss is weighted according to this probability distribution map, which can effectively avoid the phenomenon of outrageous prediction errors and effectively improve the detection accuracy of the head and shoulder key points, which is one of the innovations of the embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1a A flowchart of a method for training a head and shoulder key point detection model provided in Embodiment 1 of the present invention;

[0071] Figure 1b A schematic diagram of feature extraction of a detection model provided in the first embodiment of the present invention;

[0072] Figure 1c A schematic diagram of normalization of key shoulder points provided in Embodiment 1 of the present invention;

[0073] Figure 2 A flow chart of a method for detecting head and shoulder key points provided in Embodiment 2 of the present invention;

[0074] Figure 3 A structural block diagram of a training device for a head and shoulder key point detection model provided in Embodiment 3 of the present invention;

[0075] Figure 4 A structural block diagram of a detection device for head and shoulder key points provided in Embodiment 4 of the present invention;

[0076] Figure 5 A schematic diagram of the structure of a computing device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0079] The embodiments of the present invention disclose a training method, a detection method, a device, a equipment and a medium for detecting a head and shoulder key point detection model, which are described in detail below.

[0080] Embodiment 1

[0081] Figure 1a The flowchart of a method for training a head and shoulder key point detection model provided in the first embodiment of the present invention can be applied to the process of autonomous driving and crowd statistical analysis, especially to the process of estimating the head and shoulder key points in a scene viewed from a monocular camera. The method can be completed by a training device for the head and shoulder key point detection model, which can be implemented by software and / or hardware. Figure 1a As shown, the method includes:

[0082] 110. Train the detection model using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof.

[0083] Among them, the positions of the head and shoulder key points are marked in the training samples. The head key point is the center point of the head, and the shoulder key point is the two end points of the shoulder.

[0084] Among them, the feature map corresponding to the training sample can be extracted through the backbone network of the detection model, and the backbone network can be a convolutional neural network such as VGG (Visual Geometry Group Network) and Resnet (Residual Network).

[0085] After obtaining the original feature map, a symmetrical feature map can be obtained through symmetric transformation. Specifically, the feature matrix corresponding to the last layer of the original feature map obtained by the detection model is flipped up and down and flipped left and right to obtain a symmetrical feature map symmetrical to the last layer of the original feature map; wherein the up and down flipping operation is to swap the i-th row data of the feature matrix with the hi-th row; the left and right flipping operation is to swap the i-th column data of the feature matrix with the wi-th column data. wherein the width and height of the image are w and h, respectively. after the swap, the head and shoulder key point coordinates of the corresponding points can be obtained according to the swap relationship. it should be noted that in this flip processing, the feature value has not changed, but the corresponding feature distribution has changed, and the corresponding head key point annotations and left and right shoulder key point annotations also need to be swapped.

[0086] Specifically, taking the frontal face image as the reference, the coordinates of the head key point groundtruth (the real answer, that is, the result that the network hopes to predict) are (x h ,y h ), the left shoulder of the image is the right shoulder of the pedestrian, marked as (x r ,y r ), the right shoulder of the image is the left shoulder of the pedestrian, marked as (x l ,y l ). The width and height of the image are w and h respectively. The feature map of the last layer of the network can be represented as an m×n feature matrix M1. After two flip operations up and down and left and right, the m×n feature matrices M2 and M3 are obtained. Among them, the groundtruth corresponding to the M2 feature matrix is ​​the head key point (x h ,hy h ), the key points of the left and right shoulders of the pedestrian are (x l ,hy l ) and (x r ,hy r The groundtruth corresponding to the M3 feature matrix are the head key points (wx h ,y h ), the key points of the left and right shoulders of the pedestrian are (wx l ,y l ) and (wx r ,y r ). The above flip operation generates two feature maps that are symmetrical to the original features. The feature maps are different from the random maps generated by randomly cropping the image because a new image pattern is generated - the left and right shoulders are symmetrically swapped. The flip operation is also different from the original image. In this embodiment, the symmetric transformation of the feature map is preferably performed only on the last layer, so more than 90% of the calculation amount can be saved.

[0087] Specifically, Figure 1b A schematic diagram of feature extraction of a detection model provided in the first embodiment of the present invention is shown in FIG. Figure 1b As shown in the figure, after the sample image is input into the detection model, after feature extraction, multiple original feature maps including the head and shoulders can be obtained. After the feature matrix corresponding to the original feature map of the last layer is flipped up and down and left and right, a symmetrical feature map symmetrical to the original feature map of the last layer can be obtained, which can increase the diversity of samples during the training process.

[0088] 120. During the training process, for each training sample, determine the first Euclidean distance between the head key points predicted by the detection model and the head key points marked in the training sample, and the second Euclidean distance between the shoulder key points predicted by the model and the shoulder key points marked in the training sample.

[0089] Specifically, the first Euclidean distance and the second Euclidean distance can be calculated by the following formula:

[0090]

[0091]

[0092] Among them, L h represents the first Euclidean distance; (x h ,y h ) represents the coordinates of the key points of the head predicted by the detection model; (x gh ,y gh ) represents the calibrated coordinates of the key points of the head; L lr represents the second Euclidean distance; (x r ,y r ) represents the coordinates of the right shoulder key points predicted by the detection model; (x rg ,y rg ) represents the calibrated right shoulder key point; (x l ,y l ) represents the coordinates of the left shoulder key points predicted by the detection model; (x lg ,y lg ) represents the calibrated left shoulder key point coordinates.

[0093] 130. Normalize the predicted shoulder key points, and determine the probability values ​​of the head key points corresponding to the normalized shoulders in the head prior probability distribution.

[0094] The shoulder normalization makes the angle between the line connecting the shoulder key points and the horizontal line be 0 degrees when the face is facing backwards, and the line connecting the two shoulder key points is the image width. Figure 1c A schematic diagram of normalization of shoulder key points provided in the first embodiment of the present invention is shown in FIG. Figure 1cAs shown in the figure, the image is rotated according to the two calibrated shoulder key points and the face orientation. After the rotation, the angle between the line connecting the two shoulder key points and the horizontal is 0 degrees, and the face is facing backward. After the rotation, the image is scaled to 256 width according to the distance between the two shoulder key points, and the corresponding head key points are

[0095] Specifically, the head prior probability distribution can be obtained before model training in the following way:

[0096] The calibrated shoulder key points in the training sample are normalized, and a head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders. By normalizing the head key point statistical matrix, the prior probability distribution of the head key points is obtained.

[0097] Among them, the head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders, including:

[0098] An initial probability distribution matrix with a value of 0 is established, wherein the number of rows in the initial probability distribution matrix is ​​the same as the number of image rows; the number of columns in the initial probability distribution matrix is ​​the same as the number of image columns; Figure 1c As shown, a 256×128 matrix can be established.

[0099] For any head key point, determine the coordinates of the head key point after shoulder normalization, and according to the coordinates, add one to the value of the corresponding position in the probability distribution matrix until the head key point statistical matrix corresponding to all head key points is accumulated.

[0100] Furthermore, after obtaining the head key point statistical matrix, Gaussian filtering can be performed on the head key point statistical matrix based on a preset convolution Gaussian blur kernel; wherein the preset convolution Gaussian blur kernel is:

[0101]

[0102] The convolution of the matrix can be expressed as: Get the head key point statistical matrix P after Gaussian filtering.

[0103] Furthermore, by normalizing the statistical matrix of the head key points, the prior probability distribution of the head key points can be obtained.

[0104] The normalization of the head key point statistical matrix is ​​to divide each value in the matrix by the sum of all values ​​to obtain the probability value of the number of times each head key point appears in the image after shoulder normalization. It can be specifically expressed by the following formula:

[0105]

[0106] Among them, P(x,y) represents the head key point statistical matrix, Represents the prior probability distribution of head key points.

[0107] In this embodiment, the statistically obtained prior probability distribution is used to Figure 1b The loss function of the head key points is weighted according to this prior probability distribution map, which can effectively avoid the phenomenon of outrageous prediction errors of the head key points.

[0108] 140. According to the probability value of the head key point corresponding to the normalized shoulder in the head prior probability distribution, the first Euclidean distance and the second Euclidean distance, the weighted loss value corresponding to the head key point is calculated. When the weighted loss value converges, the detection model training is completed.

[0109] The training process of the detection model is as follows: after each batch of training samples is sent to the model, the predicted value is output through forward propagation, and then the difference between the predicted value and the true value, that is, the loss value, is calculated through the loss function. After obtaining the loss value, the model updates each parameter through back propagation to reduce the loss between the true value and the predicted value, so that the predicted value generated by the model moves closer to the true value until the value of the loss function converges and the model training is completed.

[0110] In this embodiment, the weighted loss value can be calculated by the following loss function:

[0111] L=(1-p+0.5)*L h +0.5*L lr ;

[0112] Where L represents the weighted loss value; p represents the probability value of the corresponding head key point in the head prior probability distribution after the predicted shoulder key point is normalized; L h represents the first Euclidean distance, L lr Represents the second Euclidean distance.

[0113] In the detection model training method provided in this embodiment, a symmetrical feature map that is symmetrical to the original feature map is used, which increases the feature distribution pattern, increases the diversity of samples, and does not increase the network calculation complexity during forward reasoning. In addition, this embodiment proposes a new loss function for measuring the prediction results based on the structural relationship and relative position relationship of the distribution of the head and shoulder key points. According to the positional relationship of the head key points relative to the two shoulder key points in the sample, a normalized probability distribution map of the head key points is obtained by statistics. In the new loss function measurement, the loss of the head key points is weighted according to this probability distribution map, which can effectively avoid the phenomenon of outrageous prediction errors and effectively improve the detection accuracy of the head and shoulder key points.

[0114] Embodiment 2

[0115] Figure 2 This is a flow chart of a method for detecting head and shoulder key points provided in the second embodiment of the present invention. The detection method can be completed by a detection device for head and shoulder key points. The device can be implemented by software and / or hardware. Figure 2 As shown, the method includes:

[0116] 210. Acquire an image to be tested including a head and shoulders.

[0117] 220. Input the image to be tested into the trained detection model to obtain the position information of the head key points and the shoulder key points in the image to be tested.

[0118] After the detection model is trained, the association between the key points of the head and the key points of the shoulders in the image and their position information is established. The specific training process of the detection model can refer to the description of the above embodiment, which will not be repeated here.

[0119] In this embodiment, the detection model that has been trained in the above embodiment is used to perform head and shoulder key point detection on the image to be tested, thereby effectively improving the detection accuracy of the head and shoulder key points.

[0120] Embodiment 3

[0121] Figure 3 The structural block diagram of the training device of the head and shoulder key point detection model provided in the third embodiment of the present invention is as follows: Figure 3 As shown, the device includes: a distance calculation module 310, a probability value determination module 320 and a weighted loss value calculation module 330. Among them,

[0122] The distance calculation module 310 is configured to train the detection model using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof; during the training process, for each training sample, determine a first Euclidean distance between a head key point predicted by the detection model and a head key point annotated in the training sample, and a second Euclidean distance between a shoulder key point predicted by the detection model and a shoulder key point annotated in the training sample;

[0123] The probability value determination module 320 is configured to normalize the predicted shoulder key points and determine the probability values ​​of the head key points corresponding to the normalized shoulders in the head prior probability distribution; wherein the shoulder normalization makes the angle between the line connecting the two shoulder key points and the horizontal be 0 degrees when the face is facing backward, and the line connecting the two shoulder key points is the image width;

[0124] The weighted loss value calculation module 330 is configured to calculate the weighted loss value corresponding to the head key point according to the probability value, the first Euclidean distance and the second Euclidean distance. When the weighted loss value reaches convergence, the detection model training is completed;

[0125] According to the probability value, the first Euclidean distance and the second Euclidean distance, the weighted loss value corresponding to the head key point is calculated by the following loss function:

[0126] L=(1-p+0.5)*L h +0.5*L lr ;

[0127] Where L represents the weighted loss value; p represents the probability value of the corresponding head key point in the head prior probability distribution after the predicted shoulder key point is normalized; L h represents the first Euclidean distance, L lr represents the second Euclidean distance;

[0128] The prior probability distribution of the head is obtained in the following way:

[0129] The calibrated shoulder key points in the training sample are normalized, and a head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders. By normalizing the head key point statistical matrix, the prior probability distribution of the head key points is obtained.

[0130] Optionally, the symmetric feature map is obtained in the following manner:

[0131] Extract features from the original sample image based on the detection model to obtain a plurality of original feature maps including the head and shoulders;

[0132] The feature matrix corresponding to the last layer of original feature map obtained by the detection model is flipped upside down and left-right to obtain a symmetric feature map symmetrical to the last layer of original feature map; wherein the upside-down flipping operation is to swap the i-th row data with the hi-th row of the feature matrix; the left-right flipping operation is to swap the i-th column data with the wi-th column data of the feature matrix.

[0133] Optionally, the head prior probability distribution is obtained in the following manner:

[0134] Establishing an initial probability distribution matrix with a value of 0, wherein the number of rows in the initial probability distribution matrix is ​​the same as the number of image rows; and the number of columns in the initial probability distribution matrix is ​​the same as the number of image columns;

[0135] Normalize the calibrated shoulder key points in the training sample, and determine the coordinates of the head key points after shoulder normalization; add one to the value of the corresponding position in the probability distribution matrix according to the coordinates, until the head key point statistical matrix corresponding to all head key points is accumulated.

[0136] Optionally, the device further comprises:

[0137] After obtaining the head key point statistical matrix, Gaussian filtering is performed on the head key point statistical matrix based on a preset convolution Gaussian blur kernel; wherein the preset convolution Gaussian blur kernel is:

[0138]

[0139] Correspondingly, the statistical matrix of the head key points after Gaussian filtering is normalized to obtain the prior probability distribution of the head key points.

[0140] The training device for the head and shoulder key point detection model provided in the embodiment of the present invention can execute the training method for the head and shoulder key point detection model provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in the above embodiments, please refer to the training method for the head and shoulder key point detection model provided in any embodiment of the present invention.

[0141] Embodiment 4

[0142] Figure 4 This is a structural block diagram of a head and shoulder key point detection device provided in Embodiment 4 of the present invention, the device comprises: a test image acquisition module 410 and a key point position determination module 420; wherein,

[0143] The test image acquisition module 410 is configured to acquire the test image including the head and shoulders;

[0144] The key point position determination module 420 is configured to input the image to be tested into the trained detection model to obtain the positions of the head key points and the shoulder key points in the image to be tested;

[0145] The detection model is trained by the training method of the head and shoulder key point detection model provided by any embodiment of the present invention to establish an association relationship between the head key points and shoulder key points in the image to be tested and the corresponding position information.

[0146] The detection device for head and shoulder key points provided in the embodiment of the present invention can execute the detection method for head and shoulder key points provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in the above embodiment, please refer to the detection method for head and shoulder key points provided in any embodiment of the present invention.

[0147] Embodiment 5

[0148] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of a computing device provided in Embodiment 5 of the present invention. Figure 5 As shown, the computing device may include:

[0149] A memory 701 storing executable program codes;

[0150] a processor 702 coupled to the memory 701;

[0151] The processor 702 calls the executable program code stored in the memory 701 to execute the training method of the head and shoulder key point detection model provided by any embodiment of the present invention.

[0152] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the training method for a head and shoulder key point detection model provided by any embodiment of the present invention.

[0153] An embodiment of the present invention further provides a computing device, including:

[0154] A memory storing executable program code;

[0155] a processor coupled to the memory;

[0156] The processor calls the executable program code stored in the memory to execute the head and shoulder key point detection method provided by any embodiment of the present invention.

[0157] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the head and shoulder key point detection method provided by any embodiment of the present invention.

[0158] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0159] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0160] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above methods of various embodiments of the present invention.

[0162] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0163] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0164] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a head and shoulder key point detection model, characterized in that: include: The detection model is trained using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof; During the training process, for each training sample, determining a first Euclidean distance between the head key points predicted by the detection model and the head key points annotated in the training sample, and a second Euclidean distance between the shoulder key points predicted by the detection model and the shoulder key points annotated in the training sample; Normalizing the predicted shoulder key points, and determining the probability value of the head key points corresponding to the normalized shoulders in the head prior probability distribution; wherein the shoulder normalization makes the angle between the line connecting the two shoulder key points and the horizontal be 0 degrees when the face is facing backward, and the line connecting the two shoulder key points is the width of the image; Calculate the weighted loss value corresponding to the head key point according to the probability value, the first Euclidean distance and the second Euclidean distance, and when the weighted loss value reaches convergence, the detection model training is completed; According to the probability value, the first Euclidean distance and the second Euclidean distance, the weighted loss value corresponding to the head key point is calculated by the following loss function: L=(1-p+0.5)*L h +0.5*L lr ; Where L represents the weighted loss value; p represents the probability value of the corresponding head key point in the head prior probability distribution after the predicted shoulder key point is normalized; L h represents the first Euclidean distance, L lr represents the second Euclidean distance; The prior probability distribution of the head is obtained in the following way: The calibrated shoulder key points in the training sample are normalized, and a head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders. By normalizing the head key point statistical matrix, the prior probability distribution of the head key points is obtained.

2. The method according to claim 1, characterized in that The symmetric feature map is obtained in the following way: Extract features from the original sample image based on the detection model to obtain a plurality of original feature maps including the head and shoulders; The feature matrix corresponding to the last layer of original feature map obtained by the detection model is flipped upside down and flipped left side to side to obtain a symmetrical feature map symmetrical to the last layer of original feature map; wherein the upside down flipping operation is to swap the i-th row data with the hi-th row of the feature matrix; the left side to right flipping operation is to swap the i-th column data with the wi-th column data of the feature matrix; The width and height of the image are w and h respectively.

3. The method according to any one of claims 1-2, characterized in that: The forming of the head key point statistical matrix according to the coordinates of the head key points corresponding to the normalized shoulders includes: Establishing an initial probability distribution matrix with a value of 0, wherein the number of rows in the initial probability distribution matrix is ​​the same as the number of image rows; and the number of columns in the initial probability distribution matrix is ​​the same as the number of image columns; For any head key point, determine the coordinates of the head key point after shoulder normalization, and add one to the value of the corresponding position in the probability distribution matrix according to the coordinates until the head key point statistical matrix corresponding to all head key points is accumulated.

4. The method according to any one of claims 1 to 3, characterized in that: After obtaining the head key point statistical matrix, the method further includes: Gaussian filtering is performed on the head key point statistical matrix based on a preset convolution Gaussian blur kernel; wherein the preset convolution Gaussian blur kernel is: Correspondingly, the statistical matrix of the head key points after Gaussian filtering is normalized to obtain the prior probability distribution of the head key points.

5. A method for detecting head and shoulder key points, characterized in that: include: Acquire an image to be tested that includes the head and shoulders; Inputting the image to be tested into the trained detection model to obtain the position information of the head key points and the shoulder key points in the image to be tested; The detection model is trained by the method according to any one of claims 1 to 4 to establish an association relationship between the head key points and shoulder key points in the image to be tested and the corresponding position information.

6. A training device for a head and shoulder key point detection model, characterized in that: include: The distance calculation module is configured to train the detection model using a training sample set, wherein the training sample set includes an original feature map and a symmetric feature map thereof; during the training process, for each training sample, determine a first Euclidean distance between a head key point predicted by the detection model and a head key point annotated in the training sample, and a second Euclidean distance between a shoulder key point predicted by the detection model and a shoulder key point annotated in the training sample; A probability value determination module is configured to normalize the predicted shoulder key points and determine the probability values ​​of the head key points corresponding to the normalized shoulders in the head prior probability distribution; wherein the shoulder normalization makes the angle between the line connecting the two shoulder key points and the horizontal be 0 degrees when the face is facing backward, and the line connecting the two shoulder key points is the width of the image; A weighted loss value calculation module is configured to calculate a weighted loss value corresponding to the head key point according to the probability value, the first Euclidean distance and the second Euclidean distance, and when the weighted loss value reaches convergence, the detection model training is completed; According to the probability value, the first Euclidean distance and the second Euclidean distance, the weighted loss value corresponding to the head key point is calculated by the following loss function: L=(1-p+0.5)*L h +0.5*L lr ; Where L represents the weighted loss value; p represents the probability value of the corresponding head key point in the head prior probability distribution after the predicted shoulder key point is normalized; L h represents the first Euclidean distance, L lr represents the second Euclidean distance; The prior probability distribution of the head is obtained in the following way: The calibrated shoulder key points in the training sample are normalized, and a head key point statistical matrix is ​​formed according to the coordinates of the head key points corresponding to the normalized shoulders. By normalizing the head key point statistical matrix, the prior probability distribution of the head key points is obtained.

7. A detection device for head and shoulder key points, characterized in that: include: The test image acquisition module is configured to acquire the test image including the head and shoulders; A key point position determination module is configured to input the image to be tested into a trained detection model to obtain positions of the head key points and the shoulder key points in the image to be tested; The detection model is trained by the method according to any one of claims 1 to 4 to establish an association relationship between the head key points and shoulder key points in the image to be tested and the corresponding position information.

8. A computing device, characterized in that The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the training method for the head and shoulder key point detection model as described in any one of claims 1-4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the training method of the head and shoulder key point detection model as described in any one of claims 1 to 4 is implemented.

10. A computing device, characterized in that The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the head and shoulder key point detection method as described in claim 5.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for detecting head and shoulder key points as claimed in claim 5 is implemented.

Images