Key Point Determination Method, Apparatus, Device, and Storage Medium

Abstract

The present application discloses a key point determination method, apparatus, device, and storage medium. The method includes the steps of: obtaining a first reference point of a first heat map of a first video frame in a preset video stream; the first reference point is a point in the first heat map where the response value ranking is greater than or equal to a preset ranking; determining a local receptive field of the first reference point, and based on the local receptive field, selecting a preset number of first local points; calculating a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local points; the second reference point corresponds to the first reference point of the first video frame; the second video frame and the first video frame are adjacent frames to each other; determining a first key point based on the first offset weight. The present application achieves the avoidance of the phenomenon of random jitter of key points in adjacent frames of a video stream.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a method, apparatus, device, and storage medium for determining key points. Background Art

[0002] With the development of deep learning technology, as a key point detection technology of deep learning technology, its application is becoming more and more extensive. For example, at present, human body beautification and action recognition based on human key point detection technology are relatively popular.

[0003] However, currently, when detecting key points in a video stream through key point detection technology, there will be a large difference in the positions of two key points detected in adjacent frames of the video stream, resulting in random jitter of key points in adjacent frames of the video stream. Summary of the Invention

[0004] The main purpose of this application is to provide a method, apparatus, device, and storage medium for determining key points, aiming to solve the technical problem of how to avoid the random jitter of key points in adjacent frames of a video stream.

[0005] To achieve the above object, this application provides a method for determining key points, the method includes:

[0006] Obtain a first reference point of a first heat map of a first video frame in a preset video stream; the first reference point is a point whose response value ranking in the first heat map is greater than or equal to a preset ranking;

[0007] Determine the local receptive field of the first reference point, and based on the local receptive field, select a preset number of first local points;

[0008] Calculate a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local points; the second reference point corresponds to the first reference point of the first video frame; the second video frame and the first video frame are adjacent frames to each other;

[0009] Determine a first key point based on the first offset weight.

[0010] Exemplarily, the calculating a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local points includes:

[0011] Calculate a second offset weight based on the coordinate difference between the first reference point and the first local points;

[0012] Determine a first initial key point based on the second offset weight;

[0013] Calculate a first offset weight based on the coordinate differences between the second reference point of the second video frame in the preset video stream, the first initial key point, and the first local point.

[0014] Exemplarily, the first video frame further includes other first heatmaps; after determining the first key point based on the first offset weight, it includes:

[0015] Obtain other first key points of the other first heatmaps;

[0016] Create a first initial video frame key point group based on the first key point and the other first key points;

[0017] Determine a target video frame key point group based on the first initial video frame key point group.

[0018] Exemplarily, the local receptive field includes multiple scales, and determining the target video frame key point group based on the first initial video frame key point group includes:

[0019] Obtain other first initial video frame key point groups corresponding to local receptive fields of other scales;

[0020] Determine the target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group.

[0021] Exemplarily, determining the target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group includes:

[0022] Determine the first distance average value of each first initial video frame key point group;

[0023] Calculate the average difference value between the first distance average value and the second distance average value of the second video frame;

[0024] Determine the target video frame key point group based on the average difference value.

[0025] Exemplarily, determining the first distance average value of each first initial video frame key point group includes:

[0026] Determine the geometric center of each first initial video frame key point group;

[0027] Calculate the first point distance from the geometric center to each first key point in the first initial video frame key point group corresponding to the geometric center, obtaining multiple first point distances;

[0028] Calculate the average value of the multiple first point distances, obtaining the first distance average value.

[0029] Exemplarily, determining the target video frame key point group based on the average value difference includes:

[0030] Determine the minimum average value difference from each average value difference;

[0031] Determine the first initial video frame key point group corresponding to the minimum average value difference as the target video frame key point group.

[0032] In addition, to achieve the above object, the present application further provides a key point determination device, the device includes:

[0033] A first acquisition module, configured to acquire a first reference point of a first heat map of a first video frame in a preset video stream; the first reference point is a point in the first heat map whose response value ranking is greater than or equal to a preset ranking;

[0034] A selection module, configured to determine a local receptive field of the first reference point, and based on the local receptive field, select a preset number of first local points;

[0035] A calculation module, configured to calculate a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local point; the second reference point corresponds to a first key point of the first video frame; the second video frame and the first video frame are adjacent frames to each other;

[0036] A first determination module, configured to determine a first key point based on the first offset weight.

[0037] In addition, to achieve the above object, the present application further provides a key point determination device, the key point determination device includes a memory, a processor, and a key point determination program stored on the memory and executable on the processor, and when the key point determination program is executed by the processor, the steps of the above-mentioned key point determination method are implemented.

[0038] In addition, to achieve the above object, the present application further provides a computer-readable storage medium, on which a key point determination program is stored, and when the key point determination program is executed by a processor, the steps of the above-mentioned key point determination method are implemented.

[0039] Compared with the prior art where key points in adjacent frames of a video stream jitter, resulting in low accuracy in determining key points from the video stream, in this application, by obtaining a first reference point and selecting a preset number of first local points from the local receptive field of the first reference point, local optimal information around the first key point can be obtained. It can be understood that the requirement for the first reference point to be a point with a response value ranking greater than or equal to a preset ranking in the first heat map is to ensure that the first reference point and the first local points contain local optimal information around the first key point, that is, to improve the accuracy of determining the first key point through local optimal information; on this basis, by calculating the coordinate difference between the second reference point and the first local points, a first offset weight is calculated, where the second reference point corresponds to the first reference point, and the second video frame and the first video frame are adjacent frames to each other. This not only avoids a large positional difference between the first base point and the second base point to improve the accuracy of determining the first key point, but also further reduces the positional difference between the key points detected in adjacent frames of the video stream by approaching the second reference point, thereby avoiding the phenomenon of random jitter of key points in adjacent frames of the video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a schematic flowchart of the first embodiment of the key point determination method of this application;

[0041] Figure 2 is a schematic flowchart of the second embodiment of the key point determination method of this application;

[0042] Figure 3 is a schematic structural diagram of the hardware operating environment involved in the embodiment solution of this application.

[0043] The implementation, functional features, and advantages of the purpose of this application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0045] This application provides a key point determination method. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the key point determination method of this application.

[0046] The embodiments of this application provide an embodiment of the key point determination method. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than here. The key point determination method can be applied to a terminal (such as a mobile terminal, a personal computer, etc.). For the convenience of description, the execution subject is omitted below to describe each step of the key point determination method. The key point determination method includes:

[0047] Step S10, obtain a first reference point of a first heat map of a first video frame in a preset video stream; the first reference point is a point in the first heat map where the response value rank is greater than or equal to a preset rank.

[0048] Exemplarily, after detecting a target from the first video frame through a preset target detection model, determine a first heat map of each key point in the target. It can be understood that when only one target is included in the first video frame, the first heat map is the heat map of the target; when multiple targets are included in the first video frame, the first heat map is the heat map of one target, and the acquisition method of the heat map of each target is basically the same.

[0049] Exemplarily, the preset target detection model is obtained by training an initial target detection model, and the initial target detection model is constructed by a neural network algorithm (such as CNN (Convolutional Neural Network)).

[0050] Exemplarily, the first reference point is a point in the first heat map where the response value rank is greater than or equal to a preset rank. Preferably, the first reference point is the point with the largest response value in the first heat map. The preset rank can be set as needed, and this embodiment does not make specific limitations. The response values are ranked based on size. The purpose of the response value rank being greater than or equal to the preset rank is to make the response value of the first reference point the maximum response value or a relatively large response value. It should be noted that the larger the response value, the higher the accuracy of the finally determined first key point.

[0051] Exemplarily, the preset video stream includes video streams of movies, TV dramas, live broadcasts, etc.

[0052] Step S20, determine the local receptive field of the first reference point, and based on the local receptive field, select a preset number of first local points.

[0053] The local receptive field is an area determined on the first video frame with the first reference point as the center. Generally, the first key point is located in this local receptive field. Therefore, the local receptive field includes the first key point and its surrounding local optimal information. The local receptive field can be used to assist in determining the first key point. Specifically, select a preset number of first local points from the local receptive field and make the first local points participate in the determination process of the first key point, where the preset number is determined by the scale of the local receptive field.

[0054] For example, when the scale is 3*3 and the coordinates of the first reference point are (2,2), the preset number of the first local points is 3*3 - 1 = 8, and the coordinates of the first local points are (1,1), (1,2), (1,3), (2,1), (2,3), (3,1), (3,2), (3,3); Another example is when the scale is 5*5 and the coordinates of the first reference point are (4,4), the preset number of the first local points is 5*5 - 1 = 24, and the coordinates of the first local points are (2,2), (2,3), (2,4), (2,5), (2,6), (3,2), (3,3), (3,4), (3,5), (3,6), (4,4), (4,3), (4,5), (4,6), (5,3), (5,3), (5,4), (5,5), (5,6), (6,2), (6,3), (6,4), (6,5), (6,6).

[0055] Step S30, calculate a first offset weight based on the coordinate difference between the second reference point of the second video frame in the preset video stream and the first local point; the second reference point corresponds to the first key point of the first video frame; the second video frame and the first video frame are adjacent frames to each other.

[0056] Exemplarily, this embodiment does not limit the order of the first video frame and the second video frame in the preset video stream, that is, the playing time of the first video frame can be before the second video frame or after the second video frame.

[0057] Exemplarily, the second reference point and the first reference point are related to the same key point in the same target, the difference being the video frames they are in. For example, the target is a person, and the second key point and the first key point are key points representing the chin of the person.

[0058] Exemplarily, the calculating the first offset weight based on the coordinate difference between the second reference point of the second video frame in the preset video stream and the first local point includes:

[0059] Step a, calculate a second offset weight based on the coordinate difference between the first reference point and the first local point.

[0060] The coordinate difference includes the difference in the abscissa and the difference in the ordinate, and the second offset weight is calculated by the following formula:

[0061]

[0062] where x i is the difference between the abscissa of the i-th first local point and the abscissa of the first reference point, and y i is the difference between the ordinate of the i-th first local point and the ordinate of the first reference point.

[0063] It can be understood that since the second offset weight is determined by the coordinate difference between the first local point and the first reference point, the second offset weight is used to make the first local point approach the first reference point.

[0064] Step b: Based on the second offset weight, determine the first initial key point.

[0065] The coordinates (x_prev, y_prev) of the first initial key point are calculated by the following formula:

[0066] x_prev = (x_max + (1 + W x_1 ) × x_1 +... + (1 + W x_i ) × x_i) / (i + 1)

[0067] y_prev = (y_max + (1 + W y_1 ) × y_1 +... + (1 + W y_i ) × y_i) / (i + 1)

[0068] Where x_max is the abscissa of the first reference point, y_max is the ordinate of the first reference point; x_i is the abscissa of the i-th first local point, and y_i is the ordinate of the i-th first local point.

[0069] It can be understood that the coordinates of the first initial key point calculated by the above formula are close to the coordinates of the first reference point.

[0070] Step c: Based on the second reference point of the second video frame in the preset video stream, and the coordinate differences between the first initial key point and the first local points, calculate the first offset weight.

[0071] The coordinate differences include the difference in abscissa and the difference in ordinate. Similar to the calculation method of the second offset weight, the first offset weight is calculated by the following formula:

[0072]

[0073] Where x k is the difference between the abscissa of the k-th point in the local receptive field point group composed of the first local point and the first initial key point and the abscissa of the second reference point, and yk is the difference between the ordinate of the k-th point in the local receptive field point group and the ordinate of the second reference point.

[0074] It can be understood that since the first offset weight is determined by the coordinate differences between the points in the local receptive field point group and the second reference point, the first offset weight is used to make the points in the local receptive field point group approach the second reference point.

[0075] Step S40: Determine a first key point based on the first offset weight.

[0076] Similar to the calculation process of the first initial key point, the coordinates (x_cur, y_cur) of the first key point are calculated by the following formula:

[0077] x_cur = (x_prev + (1 + W x_1 ) × x_1 +... + (1 + W x_k ) × x_k) / (k + 1)

[0078] y_cur = (y_prev + (1 + W y_1 ) × y_1 +... + (1 + W y_k ) × y_k) / (k + 1)

[0079] where x_prev is the abscissa of the first initial key point, y_prev is the ordinate of the first initial key point; x_k is the abscissa of the k-th point in the local receptive field point group, and y_k is the ordinate of the k-th point in the local receptive field point group.

[0080] It should be noted that the method for obtaining the second key point is different from that for obtaining the first key point. The process of obtaining the second key point does not involve the second reference points of other video frames, that is, the method for obtaining the second key point is basically the same as that for obtaining the first initial key point, that is, the embodiment for obtaining the second key point is basically the same as the embodiment for obtaining the first initial key point, and will not be elaborated here.

[0081] Compared with the prior art where the key points of adjacent frames in a video stream jitter, resulting in low accuracy in determining key points from the video stream, in this application, by obtaining a first reference point and selecting a preset number of first local points from the local receptive field of the first reference point to obtain local optimal information around the first key point, it can be understood that the requirement for the first reference point to be a point with a response value ranking greater than or equal to a preset ranking in the first heat map is to ensure that the first reference point and the first local points contain local optimal information around the first key point, that is, to improve the accuracy of determining the first key point through local optimal information; on this basis, by calculating the coordinate difference between the second reference point and the first local points, the first offset weight is calculated, where the second reference point corresponds to the first reference point, and the second video frame and the first video frame are adjacent frames to each other. This not only avoids a large positional difference between the first reference point and the second reference point to improve the accuracy of determining the first key point, but also further reduces the positional difference between the key points detected in adjacent frames of the video stream by approaching the second reference point, thereby avoiding the phenomenon of random jitter of key points in adjacent frames of the video stream.

[0082] Exemplarily, referring toFigure 2 , based on the first embodiment of the above method for determining the key points of the present application, a second embodiment is proposed. The first video frame further includes other first heat maps. After determining the first key points based on the first offset weights, it includes:

[0083] Step A10: Obtain other first key points of the other first heat maps.

[0084] The target in the above first embodiment does not only include one first key point, but also includes other first key points. For example, when the target is a human face, generally, a human face includes 106 key points. It can be understood that each first key point has a corresponding first heat map.

[0085] It should be noted that the embodiment of obtaining other first key points is basically the same as the embodiment of obtaining the first key points, and will not be elaborated here.

[0086] Step A20: Create a first initial video frame key point group based on the first key points and the other first key points.

[0087] The first initial video frame key point group can be used to characterize the features of the target. For example, when the target is a human face, the first initial video frame key point group can form the contour of the human face. For example, the first initial video frame key point group is [(x_cur_1, y_cur_1),..., (x_cur_N, y_cur_N)], where N is the number of first key points.

[0088] Step A30: Determine the target video frame key point group based on the first initial video frame key point group.

[0089] Exemplarily, the local receptive field includes multiple scales. The determining of the target video frame key point group based on the first initial video frame key point group includes:

[0090] Step d: Obtain other first initial video frame key point groups corresponding to local receptive fields of other scales.

[0091] Exemplarily, the scale represents the size of the local receptive field, and the scale can be k*k, where the value range of k is [3, 5, 7, 9, 11, 13, 15,...].

[0092] It should be noted that each scale of local receptive field corresponds to a first initial video frame key point group, and the obtaining method of each first initial video frame key point group is basically the same, and will not be elaborated here.

[0093] Correspondingly, other first initial video frame key point groups are [(x_cur_3_1, y_cur_3_1),...,(x_cur_3_N, y_cur_3_N)], [(x_cur_5_1, y_cur_5_1),...,(x_cur_5_N, y_cur_5_N)], [(x_cur_7_1, y_cur_7_1),...,(x_cur_7_N, y_cur_7_N)] and / or [(x_cur_k_1, y_cur_k_1),...,(x_cur_k_N, y_cur_k_N)], etc.

[0094] Step e: Determine a target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group.

[0095] The first initial video frame key point groups at different scales are different. The target video frame key point group is the first initial video frame key point group that is closest to the actual video frame key point group among all the first initial video frame key point groups.

[0096] Exemplarily, the determining of the target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group includes:

[0097] Step e1: Determine the first distance average value of each first initial video frame key point group.

[0098] Exemplarily, the determining of the first distance average value of each first initial video frame key point group includes:

[0099] Step e11: Determine the geometric center of each first initial video frame key point group.

[0100] The coordinates (x_center, y_center) of the geometric center are determined by the average coordinates of the first initial video frame key point group, and are specifically obtained through the following calculation formula:

[0101] x_center = (x_cur_k_1 + x_cur_k_2 +... + x_cur_k_N) / N

[0102] y_center = (y_cur_k_1 + y_cur_k_2 +... + y_cur_k_N) / N

[0103] Where, x_cur_k_N is the abscissa of the Nth first key point at the scale of k*k, y_cur_k_N is the ordinate of the Nth first key point at the scale of k*k, and N is the number of first key points in the first initial video frame key point group.

[0104] Step e12, calculate the first point distance from the geometric center to each first key point in the first initial video frame key point group corresponding to the geometric center, obtaining a plurality of first point distances.

[0105] The first point distance dist_k_N is calculated by the following formula:

[0106] x_dist_k_N = x_cur_k_N - x_center

[0107] y_dist_k_N = y_cur_k_N - y_center

[0108]

[0109] Wherein, x_cur_k_N is the abscissa of the Nth first key point with a scale of k*k, y_cur_k_N is the ordinate of the Nth first key point with a scale of k*k, and N is the number of first key points in the first initial video frame key point group.

[0110] That is, for each first key point, a x_dist_k_N, a y_dist_k_N, and a first point distance dist_k_N can be calculated.

[0111] Step e13, calculate the average value of the plurality of first point distances, obtaining a first distance average value.

[0112]

[0113] Wherein, is the first point distance of the Nth first key point, and N is the number of first key points in the first initial video frame key point group.

[0114] Step e2, calculate the average value difference between the first distance average value and the second distance average value of the second video frame.

[0115] The embodiment of the second distance average value is basically the same as that of the first distance average value, and will not be elaborated here.

[0116] It can be understood that there is a one-to-one correspondence between the first distance average value and the second distance average value, that is, the calculated average value difference is the average value difference at the same scale.

[0117] Step e3, based on the average value difference, determine the target video frame key point group.

[0118] Exemplarily, the determining the target video frame key point group based on the average value difference includes:

[0119] Step e31: Determine the minimum average value difference from each average value difference.

[0120] Step e32: Determine the first initial video frame key point group corresponding to the minimum average value difference as the target video frame key point group.

[0121] It can be understood that for the first key point in the first initial video frame key point group corresponding to the minimum average value difference, its position is closest to the position of the second key point in the second video frame. Therefore, it can also best reduce the difference in the positions of the two key points in adjacent frames, thereby avoiding the phenomenon of random jitter of key points in adjacent frames of the video stream.

[0122] Taking the scale including 7 scales as an example, which are 3*3, 5*5, 7*7, 9*9, 11*11, 13*13, 15*15 respectively. The average value difference calculated through the scale of 3*3 is 3, the average value difference calculated through the scale of 5*5 is 4, the average value difference calculated through the scale of 7*7 is 3.1, the average value difference calculated through the scale of 9*9 is 2.6, the average value difference calculated through the scale of 11*11 is 2.8, the average value difference calculated through the scale of 13*13 is 3.7, and the average value difference calculated through the scale of 15*15 is 3.3. It can be determined that the average value of the scale of 9*9 is the smallest at 2.6. Therefore, the first initial video frame key point group calculated at the scale of 9*9 is determined as the target video frame key point group.

[0123] In this embodiment, by calculating the first initial video frame key point groups under different scales, and selecting the first initial video frame key point group with the smallest average value difference from multiple first initial video frame key point groups according to the average value difference, and determining it as the target video frame key point group, rather than calculating the first initial video frame key point group under one scale and directly using this first initial video frame key point group as the target video frame key point group, it further improves the accuracy of determining the key point position, and further avoids the phenomenon of random jitter of key points in adjacent frames of the video stream.

[0124] In addition, the present application also provides a key point determination device, and the device includes:

[0125] The first acquisition module is used to acquire the first reference point of the first heat map of the first video frame in the preset video stream; the first reference point is the point in the first heat map where the response value ranking is greater than or equal to the preset ranking.

[0126] The selection module is used to determine the local receptive field of the first reference point, and select a preset number of first local points based on the local receptive field.

[0127] A calculation module, configured to calculate a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local point; the second reference point corresponds to a first key point of the first video frame; the second video frame and the first video frame are adjacent frames to each other.

[0128] A first determination module, configured to determine a first key point based on the first offset weight.

[0129] Exemplarily, the calculation module is specifically configured to:

[0130] Calculate a second offset weight based on the coordinate difference between the first reference point and the first local point;

[0131] Determine a first initial key point based on the second offset weight;

[0132] Calculate a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first initial key point and the first local point.

[0133] Exemplarily, the first video frame further includes other first heat maps, and the key point determination device further includes:

[0134] A second acquisition module, configured to acquire other first key points of the other first heat maps;

[0135] A creation module, configured to create a first initial video frame key point group based on the first key point and the other first key points;

[0136] A second determination module, configured to determine a target video frame key point group based on the first initial video frame key point group.

[0137] Exemplarily, the local receptive field includes multiple scales, and the second determination module is specifically configured to:

[0138] Acquire other first initial video frame key point groups corresponding to local receptive fields of other scales;

[0139] Determine a target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group.

[0140] Exemplarily, the second determination module is further configured to:

[0141] Determine a first distance average value of each first initial video frame key point group;

[0142] Calculate an average value difference between the first distance average value and a second distance average value of the second video frame;

[0143] Based on the average value difference, determine the key point group of the target video frame.

[0144] Exemplarily, the second determination module is further configured to:

[0145] Determine the geometric center of each first initial video frame key point group;

[0146] Calculate the first point distance from the geometric center to each first key point in the first initial video frame key point group corresponding to the geometric center, obtaining a plurality of first point distances;

[0147] Calculate the average value of the plurality of first point distances, obtaining the first distance average value.

[0148] Exemplarily, the second determination module is further configured to:

[0149] Determine the minimum average value difference from each average value difference;

[0150] Determine the first initial video frame key point group corresponding to the minimum average value difference as the key point group of the target video frame.

[0151] The specific implementation manner of the key point determination device in this application is basically the same as that of each embodiment of the above key point determination method, and will not be elaborated here.

[0152] In addition, this application also provides a key point determination device. As Figure 3 shown, Figure 3 is a schematic structural diagram of the hardware operating environment involved in the solution of the embodiment of this application.

[0153] It should be noted that Figure 3 can be the schematic structural diagram of the hardware operating environment of the key point determination device.

[0154] As Figure 3 shown, the key point determination device may include: a processor 3001, such as a CPU, a memory 3005, a user interface 3003, a network interface 3004, and a communication bus 3002. Among them, the communication bus 3002 is used to realize the connection and communication between these components. The user interface 3003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 3003 may further include a standard wired interface and a wireless interface. The network interface 3004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 3005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. Optionally, the memory 3005 may also be a storage device independent of the foregoing processor 3001.

[0155] Optionally, the key point determination device may further include an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and so on.

[0156] Those skilled in the art can understand that Figure 3 the structure of the key point determination device shown in

[0157] does not constitute a limitation on the key point determination device, and may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Figure 3 As shown in

[0158] In Figure 3 the key point determination device shown, the user interface 3003 is mainly used to connect to the terminal and communicate with the terminal for data, such as receiving user signaling data sent by the terminal; the network interface 3004 is mainly used for the background server and communicates with the background server for data; the processor 3001 can be used to call the key point determination program stored in the memory 3005 and execute the steps of the key point determination method as described above.

[0159] The specific implementation manner of the key point determination device in this application is basically the same as that of each embodiment of the above key point determination method, and will not be repeated here.

[0160] In addition, an embodiment of this application also proposes a computer-readable storage medium, on which a key point determination program is stored, and when the key point determination program is executed by a processor, the steps of the key point determination method as described above are implemented.

[0161] The specific implementation manner of the computer-readable storage medium in this application is basically the same as that of each embodiment of the above key point determination method, and will not be repeated here.

[0162] In addition, to achieve the above object, this application also provides a computer program product, including: a computer program, and when the computer program is executed by a processor, the steps of the key point determination method as described above are implemented.

[0163] The specific implementation manner of the computer program product in this application is basically the same as that of each embodiment of the above key point determination method, and will not be repeated here.

[0164] It should be noted that, in this document, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device that includes a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device that includes such element.

[0165] The serial numbers of the embodiments of the present application above are for description only and do not represent the superiority or inferiority of the embodiments.

[0166] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes: several instructions to enable a terminal device (which can be a mobile phone, computer, server, device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0167] The above are only the preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall equally be included in the patent protection scope of the present application.

Claims

1. A key point determination method, characterized in that, The method includes: Obtaining a first reference point of a first heat map of a first video frame in a preset video stream; the first reference point is a point in the first heat map where the response value ranking is greater than or equal to a preset ranking; Determining a local receptive field of the first reference point, and based on the local receptive field, selecting a preset number of first local points; Calculating a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local points; the second reference point corresponds to the first reference point of the first video frame; the second video frame and the first video frame are adjacent frames to each other; Determining a first key point based on the first offset weight; The first video frame further includes other first heat maps; after determining the first key point based on the first offset weight, it includes: Obtaining other first key points of the other first heat maps; Creating a first initial video frame key point group based on the first key point and the other first key points; Determining a target video frame key point group based on the first initial video frame key point group.

2. The method according to claim 1, wherein The calculating a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local points includes: Calculating a second offset weight based on the coordinate difference between the first reference point and the first local points; Determining a first initial key point based on the second offset weight; Calculating a first offset weight based on the coordinate difference between the second reference point of the second video frame in the preset video stream, the first initial key point, and the first local points.

3. The method according to claim 1, characterized in that, The local receptive field includes multiple scales, and the determining a target video frame key point group based on the first initial video frame key point group includes: Obtaining other first initial video frame key point groups corresponding to local receptive fields of other scales; Determining a target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group.

4. The method according to claim 3, wherein The determining a target video frame key point group from the other first initial video frame key point groups and the first initial video frame key point group includes: Determining a first distance average value of each first initial video frame key point group; Calculating an average value difference between the first distance average value and a second distance average value of the second video frame; Determining a target video frame key point group based on the average value difference.

5. The method according to claim 4, characterized in that, The determining a first distance average value of each first initial video frame key point group includes: Determining the geometric center of each first initial video frame key point group; Calculating a first point distance from the geometric center to each first key point in the first initial video frame key point group corresponding to the geometric center, obtaining a plurality of first point distances; Calculating an average value of the plurality of first point distances to obtain a first distance average value.

6. The method according to claim 4, wherein The determining a target video frame key point group based on the average value difference includes: Determining a minimum average value difference from each average value difference; Determining the first initial video frame key point group corresponding to the minimum average value difference as the target video frame key point group.

7. A key point determination device, characterized in that, The device includes: A first acquisition module, configured to acquire a first reference point of a first heat map of a first video frame in a preset video stream; the first reference point is a point in the first heat map where the response value rank is greater than or equal to a preset rank; A selection module, configured to determine a local receptive field of the first reference point, and based on the local receptive field, select a preset number of first local points; A calculation module, configured to calculate a first offset weight based on the coordinate difference between a second reference point of a second video frame in the preset video stream and the first local points; the second reference point corresponds to the first reference point of the first video frame; the second video frame and the first video frame are adjacent frames to each other; A first determination module, configured to determine a first key point based on the first offset weight; The key point determination device is used to implement: Acquire other first key points of other first heat maps; Create a first initial video frame key point group based on the first key point and the other first key points; Determine a target video frame key point group based on the first initial video frame key point group.

8. A key point determination device, characterized in that, The key point determination device includes a memory, a processor, and a key point determination program stored on the memory and executable on the processor. When the key point determination program is executed by the processor, the steps of the key point determination method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that, A key point determination program is stored on the computer-readable storage medium. When the key point determination program is executed by a processor, the steps of the key point determination method according to any one of claims 1 to 6 are implemented.

Images