Model performance determination method, device, server and storage medium

Abstract

The present disclosure relates to the field of image processing technology, and in particular to a model performance determination method, device, server and storage medium. Among them, a model performance determination method includes: obtaining at least one first key point output by a first key point acquisition model for each video frame in video data, and obtaining at least one second key point output by a second key point acquisition model for each video frame in the video data; normalizing the at least one first key point to obtain at least one third key point, and normalizing the at least one second key point to obtain at least one fourth key point; based on the stability data set corresponding to the at least one third key point and the at least one fourth key point, determining the performance of the first key point acquisition model and the second key point acquisition model. By adopting the present disclosure, the cost of model performance determination can be reduced while improving the reliability of model performance determination.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, server, and storage medium for determining model performance. Background Art

[0002] With the advancement of science and technology and the increasing maturity of image processing techniques, key point acquisition can be used to add display information to target objects in images. The reliability of key point acquisition directly affects the display quality of the target objects. For example, in a beauty application scenario, the terminal can use a deep learning algorithm to obtain key points in a facial image. Based on these key points, the terminal can render the facial image to obtain a makeup-enhanced facial image. Summary of the Invention

[0003] The present disclosure provides a method, device, server, and storage medium for determining model performance to at least address the problem of complex model performance determination steps in related technologies. The technical solutions of the present disclosure are as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a method for determining model performance is provided, comprising:

[0005] Acquire at least one first key point output by a first key point acquisition model for each video frame in the video data, and acquire at least one second key point output by a second key point acquisition model for each video frame in the video data;

[0006] Normalizing the at least one first key point to obtain at least one third key point, and normalizing the at least one second key point to obtain at least one fourth key point;

[0007] Performance of the first keypoint acquisition model and the second keypoint acquisition model is determined based on a set of stability data corresponding to the at least one third keypoint and the at least one fourth keypoint.

[0008] Optionally, the stability data set includes a first stability data set and a second stability data set, and determining the performance of the first key point acquisition model and the second key point acquisition model based on the stability data sets corresponding to the at least one third key point and the at least one fourth key point includes:

[0009] Acquire the first stability data set corresponding to the at least one third key point;

[0010] acquiring the second stability data set corresponding to the at least one fourth key point;

[0011] Based on the first stability data set and the second stability data set, performance of the first keypoint acquisition model and the second keypoint acquisition model is determined.

[0012] Optionally, the video data is synthesized from a single picture, and obtaining the first stability data set corresponding to the at least one third key point includes:

[0013] Obtaining a first distance value of any third key point among the at least one third key point in adjacent video frames;

[0014] Traversing the video frames to obtain a first distance value set corresponding to the third key point;

[0015] acquiring first stability data corresponding to the third key point based on the first distance value set, and adding the first stability data to the first stability data set;

[0016] Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point;

[0017] The acquiring of the second stability data set corresponding to the at least one fourth key point includes:

[0018] Obtaining a second distance value of any fourth key point among the at least one fourth key point in adjacent video frames;

[0019] Traversing the video frames to obtain a second distance value set corresponding to the fourth key point;

[0020] acquiring first stability data corresponding to the fourth key point based on the second distance value set, and adding the second stability data to the second stability data set;

[0021] The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

[0022] Optionally, the first stability parameter is a first average distance value, and the second stability parameter is a second average distance value. Determining the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set includes:

[0023] calculating an average of at least one first average distance value in the first stability data set to obtain a third average distance value;

[0024] calculating an average of at least one second average distance value in the second stability data set to obtain a fourth average distance value;

[0025] If the third average distance value is greater than the fourth average distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0026] If the third average distance value is equal to the fourth average distance value, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model;

[0027] If the third average distance value is smaller than the fourth average distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0028] Optionally, the first stability parameter is a first maximum distance value, and the second stability parameter is a second maximum distance value. Determining the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set includes:

[0029] Obtaining a third maximum distance value among at least one first maximum distance value in the first stability data set, wherein the third maximum distance value is a maximum first maximum distance value among the at least one first maximum distance value;

[0030] obtaining a fourth maximum distance value of at least one second maximum distance value in the second stability data set, the fourth maximum distance value being a maximum second maximum distance value among the at least one second maximum distance value;

[0031] If the third maximum distance value is greater than the fourth maximum distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0032] If the third maximum distance value is equal to the fourth maximum distance value, determining that the performance of the first key point acquisition model is the same as the performance of the second key point acquisition model;

[0033] If the third maximum distance value is smaller than the fourth maximum distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0034] Optionally, obtaining the first stability data set corresponding to the at least one third key point includes:

[0035] Obtaining a first coordinate data set of any third key point among the at least one third key point in each video frame;

[0036] performing smoothing processing on the first coordinate data set to obtain a second coordinate data set;

[0037] acquiring first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set, and adding the first stability data to the first stability data set;

[0038] Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point;

[0039] The acquiring of the second stability data set corresponding to the at least one fourth key point includes:

[0040] Obtaining a third coordinate data set of any fourth key point among the at least one fourth key point in each video frame;

[0041] performing smoothing processing on the third coordinate data set to obtain a fourth coordinate data set;

[0042] acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set, and adding the second stability data to the second stability data set;

[0043] The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

[0044] Optionally, the first coordinate data set includes first abscissa data and first ordinate data, and the second coordinate data set includes second abscissa data and second ordinate data, and acquiring first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set includes:

[0045] Calculating a first offset corresponding to the third key point based on the first horizontal coordinate data and the second horizontal coordinate data corresponding to the third key point;

[0046] Calculating a second offset corresponding to the third key point based on the first vertical coordinate data and the second vertical coordinate data in each video frame;

[0047] Summing the absolute values ​​of the first offset and the second offset to obtain a fifth offset corresponding to the third key point;

[0048] The third coordinate data set includes third abscissa data and third ordinate data, the fourth coordinate data set includes fourth abscissa data and fourth ordinate data, and acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set includes:

[0049] Calculating a third offset corresponding to the fourth key point based on the third horizontal coordinate data and the fourth horizontal coordinate data corresponding to the fourth key point;

[0050] Calculating a fourth offset corresponding to the fourth key point based on the sum of the absolute values ​​of the third ordinate data and the fourth ordinate data corresponding to the fourth key point;

[0051] An absolute value sum is performed on the third offset and the fourth offset to obtain a sixth offset corresponding to the fourth key point.

[0052] Optionally, the first stability data set includes at least one fifth offset, and the second stability data set includes at least one sixth offset, and determining the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set includes:

[0053] Obtaining an average value of the at least one fifth offset to obtain a first average offset;

[0054] Obtaining an average value of the at least one sixth offset to obtain a second average offset;

[0055] If the first average offset is greater than the second average offset, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0056] If the first average offset is equal to the second average offset, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model;

[0057] If the first average offset is smaller than the second average offset, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0058] Optionally, the video data includes a facial image, and the normalizing the at least one first key point to obtain the at least one third key point includes:

[0059] Obtaining the output image plane size of the model by acquiring the first key point;

[0060] Obtaining a minimum rectangular image plane covering the at least one first key point in the facial image;

[0061] Based on the output image plane size and the size corresponding to the minimum rectangular image plane, the at least one first key point is normalized to obtain at least one third key point.

[0062] Optionally, the video data includes a facial image, and the normalizing the at least one first key point to obtain the at least one third key point includes:

[0063] Obtaining a minimum rectangular image plane covering the at least one first key point in the facial image;

[0064] Obtaining the diagonal distance of the minimum rectangular image plane;

[0065] Based on the diagonal distance, the at least one first key point is normalized to obtain at least one third key point.

[0066] Optionally, the video data includes a facial image, and the normalizing the at least one first key point to obtain the at least one third key point includes:

[0067] Obtaining an interpupillary distance value in the face image;

[0068] Based on the pupillary distance value, the at least one first key point is normalized to obtain at least one third key point;

[0069] or,

[0070] Obtaining a distance between the outer corners of both eyes in the face image;

[0071] Based on the distance between the outer canthi of both eyes, the at least one first key point is normalized to obtain at least one third key point.

[0072] According to a second aspect of an embodiment of the present disclosure, there is provided a model performance determination device, comprising:

[0073] a key point acquisition unit configured to execute acquisition of at least one first key point output by a first key point acquisition model for each video frame in the video data, and acquisition of at least one second key point output by a second key point acquisition model for each video frame in the video data;

[0074] a key point processing unit configured to perform normalization processing on the at least one first key point to obtain at least one third key point, and perform normalization processing on the at least one second key point to obtain at least one fourth key point;

[0075] The performance determination unit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the stability data set corresponding to the at least one third key point and the at least one fourth key point.

[0076] Optionally, the performance determination unit includes a data acquisition subunit and a performance determination subunit, the stability data set includes a first stability data set and a second stability data set, and the performance determination unit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the stability data sets corresponding to the at least one third key point and the at least one fourth key point:

[0077] The data acquisition subunit is configured to acquire the first stability data set corresponding to the at least one third key point;

[0078] The data acquisition subunit is further configured to execute acquisition of the second stability data set corresponding to the at least one fourth key point;

[0079] The performance determination subunit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set.

[0080] Optionally, the video data is synthesized from a single picture, and the data acquisition subunit is configured to execute, when acquiring the first stability data set corresponding to the at least one third key point, specifically configured to execute:

[0081] Obtaining a first distance value of any third key point among the at least one third key point in adjacent video frames;

[0082] Traversing the video frames to obtain a first distance value set corresponding to the third key point;

[0083] acquiring first stability data corresponding to the third key point based on the first distance value set, and adding the first stability data to the first stability data set;

[0084] Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point;

[0085] The data acquisition subunit is configured to acquire the second stability data set corresponding to the at least one fourth key point, and is specifically configured to execute:

[0086] Obtaining a second distance value of any fourth key point among the at least one fourth key point in adjacent video frames;

[0087] Traversing the video frames to obtain a second distance value set corresponding to the fourth key point;

[0088] acquiring first stability data corresponding to the fourth key point based on the second distance value set, and adding the second stability data to the second stability data set;

[0089] The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

[0090] Optionally, the first stability parameter is a first average distance value, and the second stability parameter is a second average distance value. When the performance determination subunit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, the subunit is specifically configured to perform:

[0091] calculating an average of at least one first average distance value in the first stability data set to obtain a third average distance value;

[0092] calculating an average of at least one second average distance value in the second stability data set to obtain a fourth average distance value;

[0093] If the third average distance value is greater than the fourth average distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0094] If the third average distance value is equal to the fourth average distance value, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model;

[0095] If the third average distance value is smaller than the fourth average distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0096] Optionally, the first stability parameter is a first maximum distance value, and the second stability parameter is a second maximum distance value. When the performance determination subunit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, the subunit is specifically configured to perform:

[0097] Obtaining a third maximum distance value among at least one first maximum distance value in the first stability data set, wherein the third maximum distance value is a maximum first maximum distance value among the at least one first maximum distance value;

[0098] obtaining a fourth maximum distance value of at least one second maximum distance value in the second stability data set, the fourth maximum distance value being a maximum second maximum distance value among the at least one second maximum distance value;

[0099] If the third maximum distance value is greater than the fourth maximum distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0100] If the third maximum distance value is equal to the fourth maximum distance value, determining that the performance of the first key point acquisition model is the same as the performance of the second key point acquisition model;

[0101] If the third maximum distance value is smaller than the fourth maximum distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0102] Optionally, when the data acquisition subunit is configured to acquire the first stability data set corresponding to the at least one third key point, it is specifically configured to execute:

[0103] Obtaining a first coordinate data set of any third key point among the at least one third key point in each video frame;

[0104] performing smoothing processing on the first coordinate data set to obtain a second coordinate data set;

[0105] acquiring first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set, and adding the first stability data to the first stability data set;

[0106] Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point;

[0107] The data acquisition subunit is configured to acquire the second stability data set corresponding to the at least one fourth key point, and is specifically configured to execute:

[0108] Obtaining a third coordinate data set of any fourth key point among the at least one fourth key point in each video frame;

[0109] performing smoothing processing on the third coordinate data set to obtain a fourth coordinate data set;

[0110] acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set, and adding the second stability data to the second stability data set;

[0111] The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

[0112] Optionally, the first coordinate data set includes first abscissa data and first ordinate data, and the second coordinate data set includes second abscissa data and second ordinate data. When the data acquisition subunit is configured to acquire the first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set, the data acquisition subunit is specifically configured to execute:

[0113] Calculating a first offset corresponding to the third key point based on the first horizontal coordinate data and the second horizontal coordinate data corresponding to the third key point;

[0114] Calculating a second offset corresponding to the third key point based on the first vertical coordinate data and the second vertical coordinate data in each video frame;

[0115] Summing the absolute values ​​of the first offset and the second offset to obtain a fifth offset corresponding to the third key point;

[0116] The third coordinate data set includes third abscissa data and third ordinate data, the fourth coordinate data set includes fourth abscissa data and fourth ordinate data, and acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set includes:

[0117] Calculating a third offset corresponding to the fourth key point based on the third horizontal coordinate data and the fourth horizontal coordinate data corresponding to the fourth key point;

[0118] Calculating a fourth offset corresponding to the fourth key point based on the sum of the absolute values ​​of the third ordinate data and the fourth ordinate data corresponding to the fourth key point;

[0119] An absolute value sum is performed on the third offset and the fourth offset to obtain a sixth offset corresponding to the fourth key point.

[0120] Optionally, the first stability data set includes at least one fifth offset, and the second stability data set includes at least one sixth offset. When the performance determination subunit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, the subunit is specifically configured to perform:

[0121] Obtaining an average value of the at least one fifth offset to obtain a first average offset;

[0122] Obtaining an average value of the at least one sixth offset to obtain a second average offset;

[0123] If the first average offset is greater than the second average offset, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0124] If the first average offset is equal to the second average offset, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model;

[0125] If the first average offset is smaller than the second average offset, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0126] Optionally, the video data includes a facial image, the key point processing unit includes a size acquisition subunit, an image plane acquisition subunit, and a key point processing subunit, and the key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point:

[0127] The size acquisition subunit is configured to acquire the output image plane size of the first key point acquisition model;

[0128] The image plane acquisition subunit is configured to execute acquisition of a minimum rectangular image plane covering the at least one first key point in the face image;

[0129] The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one third key point.

[0130] Optionally, the video data includes a facial image, the key point processing unit includes an image plane acquisition subunit, a distance acquisition subunit, and a key point processing subunit, and the key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point:

[0131] The image plane acquisition subunit is configured to execute acquisition of a minimum rectangular image plane covering the at least one first key point in the face image;

[0132] The distance acquisition subunit is configured to acquire the diagonal distance of the minimum rectangular image plane;

[0133] The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the diagonal distance to obtain at least one third key point.

[0134] Optionally, the video data includes a facial image, the key point processing unit includes a spacing value acquisition subunit and a key point processing subunit, and the key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point:

[0135] The distance value acquisition subunit is configured to execute acquisition of the pupil distance value in the face image;

[0136] The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the pupil distance value to obtain at least one third key point;

[0137] or,

[0138] The distance value acquisition subunit is configured to execute acquisition of the distance value between the outer canthi of both eyes in the face image;

[0139] The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the binocular outer canthus distance value to obtain at least one third key point.

[0140] According to a third aspect of the present application, a terminal is provided, including:

[0141] at least one processor; and

[0142] a memory communicatively connected to the at least one processor; wherein,

[0143] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the methods described in the above aspect.

[0144] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the aforementioned aspect.

[0145] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program implements any one of the methods described in the aforementioned aspect when executed by a processor.

[0146] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0147] In some or related embodiments, by obtaining at least one first key point output by a first key point acquisition model for each video frame in the video data, and obtaining at least one second key point output by a second key point acquisition model for each video frame in the video data, the at least one first key point can be normalized to obtain at least one third key point, and the at least one second key point can be normalized to obtain at least one fourth key point. The at least one first key point and the at least one second key point can be converted into the same dimension for comparison, which can reduce the situation where the at least one first key point and the at least one second key point are in different dimensions, making the model performance determination unreliable. Based on the stability data set corresponding to the at least one third key point and the at least one fourth key point, the performance of the first key point acquisition model and the second key point acquisition model is determined. Based on the stability data sets corresponding to different key points, the performance of different key point acquisition models can be determined without manual labeling, which can reduce the impact of manual subjective labeling on model performance and the cost of model performance determination. At the same time, the model performance is determined based on the stability data set, which can improve the reliability of the model performance determination.

[0148] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0149] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0150] Figure 1 is a background diagram illustrating a method for determining model performance according to an exemplary embodiment;

[0151] Figure 2 is a flow chart showing a method for determining model performance according to an exemplary embodiment;

[0152] Figure 3 is a flow chart showing a method for determining model performance according to an exemplary embodiment;

[0153] Figure 4 is an adjustment change diagram of a minimum rectangular image plane according to an exemplary embodiment;

[0154] Figure 5 is a schematic diagram showing an example of a diagonal line of a minimum rectangular image plane according to an exemplary embodiment;

[0155] Figure 6 is an adjustment change diagram of a minimum rectangular image plane according to an exemplary embodiment;

[0156] Figure 7 is a schematic diagram showing an example of a diagonal line of a minimum rectangular image plane according to an exemplary embodiment;

[0157] Figure 8 is a schematic diagram illustrating an example of a terminal interface according to an exemplary embodiment;

[0158] Figure 9 is a block diagram of a device for determining model performance according to an exemplary embodiment;

[0159] Figure 10 is a block diagram of a device for determining model performance according to an exemplary embodiment;

[0160] Figure 11 is a block diagram of a device for determining model performance according to an exemplary embodiment;

[0161] Figure 12 is a block diagram of a device for determining model performance according to an exemplary embodiment;

[0162] Figure 13 is a block diagram of a device for determining model performance according to an exemplary embodiment;

[0163] Figure 14 It is a block diagram of a terminal according to an exemplary embodiment. DETAILED DESCRIPTION

[0164] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0165] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0166] With the advancement of science and technology and the increasing maturity of image processing techniques, key point acquisition can be used to add display information to target objects in images. The reliability of key point acquisition directly affects the display quality of the target objects. For example, deep learning algorithms can be used to acquire key points in images. To improve the reliability of key point acquisition, the parameters of the deep learning algorithms can be continuously adjusted to improve the reliability of key point acquisition.

[0167] According to some embodiments, Figure 1 FIG. 1 is a background diagram of a method for determining model performance according to an exemplary embodiment. Figure 1 For example, when predicting key points in a facial image, a user can manually annotate the facial image. Alternatively, the terminal can use a deep learning algorithm to obtain key points in the image. The terminal can compare the manually annotated key points with those obtained by the deep learning algorithm. However, the excessive amount of manually annotated data leads to high costs and high subjectivity, making the performance of the deep learning algorithm inaccurate.

[0168] Figure 2 FIG. 1 is a flow chart showing a method for determining model performance according to an exemplary embodiment. Figure 2 As shown, the model performance determination method is used in the key point determination scenario. The method can be implemented by a computer program and can be run on a server or a terminal. The method includes the following steps:

[0169] In step S11, at least one first key point output by the first key point acquisition model for each video frame in the video data is obtained, and at least one second key point output by the second key point acquisition model for each video frame in the video data is obtained;

[0170] In some embodiments, the execution subject of the embodiments of the present disclosure may be, for example, a terminal, which includes but is not limited to: a wearable device, a handheld device, a personal computer, a tablet computer, a vehicle-mounted device, a smart phone, a computing device, or other processing device connected to a wireless modem. In different networks, terminal devices may be called different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), terminal device in the fifth generation mobile communication technology (5G) network or future evolution network, etc. The live broadcast terminal may be installed with an operating system, which refers to an operating system that can run in the terminal, a program that manages and controls terminal hardware and terminal applications, and is an indispensable system application in the terminal. The operating system includes but is not limited to Android system, IOS system, Windows phone (WP) system and Ubuntu mobile version operating system, etc.

[0171] According to some embodiments, a key point refers to a point in each video frame that is important for analyzing a problem. A key point acquisition model refers to a model used to determine key points in a video frame. The first key point acquisition model and the second key point acquisition model are different key point acquisition models. The first key point acquisition model does not specifically refer to a fixed key point acquisition model. For example, when the parameters corresponding to the first key point acquisition model change, the first key point acquisition model may also change accordingly. The second key point acquisition model does not specifically refer to a fixed key point acquisition model. For example, when a modification instruction for the key point acquisition model is received, the second key point acquisition model may also change accordingly.

[0172] In some embodiments, video data refers to a continuous sequence of images, which is essentially composed of a series of continuous images. The video data may include only one video data, or may include multiple video data, wherein the multiple video data may correspond to different video contents. The video data does not specifically refer to a fixed video data. For example, when the number of videos included in the video data changes, the video data may also change accordingly. For example, when the length of the video corresponding to the video data changes, the video data may also change accordingly. For example, when the video content corresponding to the video data changes, the video data may also change accordingly.

[0173] It is easy to understand that a video frame can be a still image in the video data, and a series of video frames can form the video data. The number of video frames included in the video data is not specifically a fixed number of frames. For example, when the video length corresponding to the video data changes, the number of video frames will also change accordingly.

[0174] According to some embodiments, a first key point refers to a key point output by a first key point acquisition model for identifying each video frame in the video data. There is at least one first key point, and the first key point does not specifically refer to a fixed key point. For example, when the number of first key points changes, the first key point may also change accordingly. When the video content corresponding to the video data changes, the first key point may also change accordingly. When the first key point acquisition model changes, the first key point may also change accordingly.

[0175] In some embodiments, the second key point refers to the key point output by the second key point acquisition model for identifying each video frame in the video data. The number of the second key point is at least one, and the second key point does not specifically refer to a fixed key point. For example, when the number of second key points changes, the second key point may also change accordingly. When the video content corresponding to the video data changes, the second key point may also change accordingly. When the second key point acquisition model changes, the second key point may also change accordingly.

[0176] In some embodiments, when executing the model performance determination method, the terminal may, for example, obtain at least one first key point output by the first key point acquisition model for each video frame in the video data. The terminal may also, for example, obtain at least one second key point output by the second key point acquisition model for each video frame in the video data.

[0177] In step S12, normalizing the at least one first key point to obtain at least one third key point, and normalizing the at least one second key point to obtain at least one fourth key point;

[0178] According to some embodiments, the third key point refers to a key point obtained by normalizing the first key point. The number of the third key points can be the same as or less than the number of the first key points. The third key point does not specifically refer to a fixed key point. For example, when the first key point changes, the third key point may also change accordingly. For example, when the method for normalizing the first key point changes, the third key point may also change accordingly.

[0179] In some embodiments, the fourth key point refers to a key point obtained by normalizing the second key point. The number of the fourth key points can be the same as or less than the number of the second key points. The fourth key point does not specifically refer to a fixed key point. For example, when the second key point changes, the fourth key point may also change accordingly. For example, when the method for normalizing the second key point changes, the fourth key point may also change accordingly.

[0180] It's easy to understand that normalization is a computational method that transforms a dimensioned expression into a dimensionless expression, a scalar. This normalization method isn't fixed. For example, if a modification instruction is received regarding the normalization method, the normalization method can be changed accordingly.

[0181] In some embodiments, when executing the model performance determination method, the terminal may, for example, obtain at least one first key point output by the first key point acquisition model for each video frame in the video data. The terminal may also, for example, obtain at least one second key point output by the second key point acquisition model for each video frame in the video data. The terminal may normalize the at least one first key point to obtain at least one third key point, and normalize the at least one second key point to obtain at least one fourth key point.

[0182] In step S13 , performance of the first keypoint acquisition model and the second keypoint acquisition model is determined based on the stability data set corresponding to the at least one third keypoint and the at least one fourth keypoint.

[0183] According to some embodiments, a stability data set refers to a collection of at least one stability data set. The stability data set does not specifically refer to a fixed set. For example, when the third key point or the fourth key point changes, the stability data set may also change accordingly. For example, when the type of data included in the stability data set changes, the stability data set may also change accordingly.

[0184] In some embodiments, when executing the model performance determination method, the terminal may, for example, obtain at least one first key point output by the first key point acquisition model for each video frame in the video data. The terminal may also, for example, obtain at least one second key point output by the second key point acquisition model for each video frame in the video data. The terminal may normalize the at least one first key point to obtain at least one third key point, and normalize the at least one second key point to obtain at least one fourth key point.

[0185] In some or related embodiments, by obtaining at least one first key point output by a first key point acquisition model for each video frame in the video data, and obtaining at least one second key point output by a second key point acquisition model for each video frame in the video data, the at least one first key point can be normalized to obtain at least one third key point, and the at least one second key point can be normalized to obtain at least one fourth key point. The at least one first key point and the at least one second key point can be converted into the same dimension for comparison, which can reduce the situation where the at least one first key point and the at least one second key point are in different dimensions, making the model performance determination unreliable. Based on the stability data set corresponding to the at least one third key point and the at least one fourth key point, the performance of the first key point acquisition model and the second key point acquisition model is determined. Based on the stability data sets corresponding to different key points, the performance of different key point acquisition models can be determined without manual labeling, which can reduce the impact of manual subjective labeling on model performance and the cost of model performance determination. At the same time, the model performance is determined based on the stability data set, which can improve the reliability of the model performance determination.

[0186] Figure 3 FIG. 1 is a flow chart showing a method for determining model performance according to an exemplary embodiment. Figure 3 As shown, the model performance determination method includes the following steps:

[0187] In step S21, at least one first key point output by the first key point acquisition model for each video frame in the video data is obtained, and at least one second key point output by the second key point acquisition model for each video frame in the video data is obtained;

[0188] The specific process is as described above and will not be repeated here.

[0189] In some embodiments, the model performance determination method can be applied to the acquisition of key points of human body parts, and can also be applied to the acquisition of key points of various objects in an image.

[0190] According to some embodiments, the first key point acquisition model may be, for example, an A key point acquisition model, and the second key point acquisition model may be, for example, a B key point acquisition model. The video data may be, for example, Q video data. The A key point acquisition model outputs at least one first key point for each video frame in the Q video data, and the B key point acquisition model outputs at least one second key point for each video frame in the Q video data.

[0191] In step S22, normalizing the at least one first key point to obtain at least one third key point, and normalizing the at least one second key point to obtain at least one fourth key point;

[0192] The specific process is as described above and will not be repeated here.

[0193] According to some embodiments, video data may include, for example, a facial image. When the terminal normalizes at least one first key point to obtain at least one third key point, the terminal may, for example, obtain the output image plane size of the first key point acquisition model and obtain the minimum rectangular image plane covering the at least one first key point in the facial image. The terminal may normalize at least one first key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one third key point, may convert at least one first key point and at least one second key point into the same dimension for comparison, may reduce the situation where at least one first key point and at least one second key point are in different dimensions, making the model performance determination unreliable, may improve the reliability of obtaining the stability parameter set, and may improve the reliability of the model performance determination. Wherein, the output image plane size may, for example, be the first output image plane size, and the minimum rectangular image plane may, for example, be the first minimum rectangular image plane.

[0194] In some embodiments, when the terminal normalizes the key points, the terminal may detect whether the minimum rectangular image plane and the output image plane are consistent. If the minimum rectangular image plane and the output image plane are inconsistent, the terminal may adjust the minimum rectangular image plane based on the output image plane to obtain an adjusted rectangular image plane. For example, the terminal may normalize at least one first key point based on the size of the output image plane and the size corresponding to the adjusted rectangular image plane to obtain at least one third key point.

[0195] It is easy to understand that the minimum rectangular image plane does not specifically refer to a fixed image plane. The minimum rectangular image plane is determined based on at least one first key point. For example, when the number corresponding to at least one first key point changes, the minimum rectangular image plane may also change accordingly.

[0196] In some embodiments, when the terminal can normalize at least one first key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one third key point, the terminal can, for example, obtain the ratio value of the output image plane size and the size corresponding to the minimum rectangular image plane. The terminal can obtain the coordinate data of at least one first key point, and based on the coordinate data and the ratio value, obtain the coordinate data corresponding to at least one third key point, that is, at least one third key point can be obtained.

[0197] Optionally, when the terminal is normalizing the key points, the terminal can detect whether the minimum rectangular image plane and the output image plane are consistent. The minimum rectangular image plane can be, for example, a rectangle, and the output image plane can be, for example, a square. If the minimum rectangular image plane and the output image plane are inconsistent, the terminal can adjust the minimum rectangular image plane based on the output image plane to obtain an adjusted rectangular image plane. The terminal can adjust the minimum rectangular image plane based on the output image plane, for example, by keeping the long side of the minimum rectangular image plane unchanged and extending the short side to both sides by the same length to obtain a square with the long side of the minimum rectangular image plane as the side length.

[0198] It is easy to understand that the output image plane size of the key point model A obtained by the terminal can be, for example, 64 pixels (pixel) * 64 pixels, and the minimum rectangular image plane covering at least one first key point in the face image obtained by the terminal can be, for example, 640 pixels * 540 pixels. The terminal can keep the long side of the minimum rectangular image plane 640 pixels unchanged, and extend the short side 540 pixels to both sides by the same length of 50 pixels. The terminal can adjust the minimum rectangular image plane to 640 pixels * 640 pixels. At this time, the adjustment change diagram of the minimum rectangular image plane can be, for example, as shown below. Figure 4 shown.

[0199] According to some embodiments, the video data may include, for example, a facial image. When the terminal normalizes at least one second key point to obtain at least one fourth key point, the terminal may, for example, obtain the output image plane size of the second key point acquisition model and obtain the minimum rectangular image plane covering the at least one second key point in the facial image. The terminal may normalize the at least one second key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one fourth key point. The output image plane size may, for example, be the second output image plane size, and the minimum rectangular image plane may, for example, be the second minimum rectangular image plane.

[0200] In some embodiments, when the terminal can normalize at least one second key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one fourth key point, the terminal can, for example, obtain the ratio value of the output image plane size and the size corresponding to the minimum rectangular image plane. The terminal can obtain the coordinate data of at least one second key point, and based on the coordinate data and the ratio value, obtain the coordinate data corresponding to at least one fourth key point, that is, at least one fourth key point can be obtained.

[0201] According to some embodiments, the video data may include, for example, a facial image. When the terminal performs normalization processing on at least one first key point to obtain at least one third key point, the terminal may obtain a minimum rectangular image plane in the facial image that covers the at least one first key point and obtain a diagonal distance of the minimum rectangular image plane. The terminal may perform normalization processing on the at least one first key point based on the diagonal distance to obtain the at least one third key point.

[0202] In some embodiments, the terminal obtains the minimum rectangular image plane covering at least one first key point in the face image, for example, Figure 5 The terminal obtains the diagonal diagram of the minimum rectangular image plane, for example, as shown in FIG. Figure 5 As shown. The terminal can obtain coordinate data corresponding to at least one first key point. For example, the terminal can divide the coordinate data corresponding to at least one first key point by the diagonal distance to obtain coordinate data corresponding to at least one third key point, that is, to obtain at least one third key point.

[0203] Optionally, the diagonal distance does not specifically refer to a fixed distance. For example, when the minimum rectangular image plane changes, the diagonal distance may also change accordingly. For example, when the size of the long side or the short side of the minimum rectangular image plane changes, the diagonal distance may also change accordingly.

[0204] According to some embodiments, the video data may include, for example, a facial image. When the terminal performs normalization processing on the at least one second key point to obtain the at least one fourth key point, the terminal may obtain a minimum rectangular image plane in the facial image that covers the at least one second key point and obtain a diagonal distance of the minimum rectangular image plane. The terminal may perform normalization processing on the at least one second key point based on the diagonal distance to obtain the at least one fourth key point.

[0205] In some embodiments, the terminal obtains the adjustment change map of the minimum rectangular image plane covering at least one second key point in the face image, for example, Figure 6 The terminal obtains the diagonal diagram of the minimum rectangular image plane, for example, as shown in FIG. Figure 7 As shown. The terminal can obtain the coordinate data corresponding to the at least one second key point. For example, the terminal can divide the coordinate data corresponding to the at least one second key point by the diagonal distance to obtain the coordinate data corresponding to the at least one fourth key point, that is, to obtain the at least one fourth key point.

[0206] According to some embodiments, the video data may include a facial image, for example. When the terminal normalizes at least one first key point to obtain at least one third key point, the terminal may obtain an interpupillary distance value from the facial image. The terminal normalizes the at least one first key point based on the interpupillary distance value to obtain the at least one third key point. Alternatively, the terminal may obtain an interocular distance value from the facial image and, based on the interocular distance value, normalize the at least one first key point to obtain the at least one third key point.

[0207] It is easy to understand that the video data may include, for example, a facial image. When the terminal normalizes the at least one second key point to obtain the at least one fourth key point, the terminal may obtain the pupillary distance value in the facial image. The terminal normalizes the at least one second key point based on the pupillary distance value to obtain the at least one fourth key point. Alternatively, the terminal may obtain the distance between the outer canthi of both eyes in the facial image and, based on the distance between the outer canthi of both eyes, normalize the at least one second key point to obtain the at least one fourth key point.

[0208] In some embodiments, the pupillary distance value refers to the distance between the pupils in a facial image. The pupillary distance value does not specifically refer to a fixed distance value. For example, when the facial image changes, the pupillary distance value may also change accordingly. The pupillary distance value may, for example, be the distance between the centers of the two eyeballs of a person's eyes.

[0209] It is easy to understand that the distance between the outer corners of the eyes refers to the distance between the outer corners of the eyes in the facial image. The distance between the outer corners of the eyes does not specifically refer to a fixed distance value. For example, when the facial image changes, the distance between the outer corners of the eyes may also change accordingly.

[0210] In some embodiments, since the first keypoint acquisition model is a different model from the second keypoint acquisition model, the number of keypoints corresponding to the at least one first keypoint may be different from the number of keypoints corresponding to the at least one second keypoint.

[0211] In step S23, a first stability data set corresponding to at least one third key point is obtained;

[0212] In some embodiments, when the terminal normalizes at least one first key point to obtain at least one third key point, the terminal can obtain a first stability data set corresponding to the at least one third key point. The stability data set includes a first stability data set and a second stability data set. The first stability data set refers to a set corresponding to the at least one third key point. The first stability data set does not specifically refer to a fixed data set. For example, when the number corresponding to the at least one third key point changes, the first stability data set may also change accordingly.

[0213] According to some embodiments, the second stability data set refers to a set corresponding to at least one fourth key point. The second stability data set is not specifically a fixed data set. For example, when the number corresponding to at least one fourth key point changes, the second stability data set may also change accordingly.

[0214] According to some embodiments, when the video data is synthesized from a single image and the terminal obtains a first stability data set corresponding to at least one third key point, the terminal may, for example, obtain a first distance value of any third key point in an adjacent video frame. The terminal traverses each video frame and obtains a first distance value set corresponding to the third key point. Based on the first distance value set, the terminal may obtain first stability data corresponding to the third key point and add the first stability data to the first stability data set. By traversing at least one third key point, the terminal may obtain the first stability data set corresponding to the at least one third key point.

[0215] In some embodiments, the video data is composed of a single image, that is, the video data includes only one image. The single image does not specifically refer to a fixed image. For example, when the image elements included in the single image change, the single image may also change accordingly, and the video data may also change accordingly.

[0216] Optionally, the first distance value set refers to a collection of distance values ​​corresponding to the third key point. The first distance value set does not specifically refer to a fixed distance value set. For example, when the number of video frames included in the video data changes, the first distance value set may also change accordingly. For example, when the value of the first distance value changes, the first distance value set may also change accordingly.

[0217] It is easy to understand that at least one third key point may include, for example, 100 key points, such as key points A1 to A100. The video data may include, for example, 100 video frames. The terminal may, for example, obtain the first distance value of the A1 key point between adjacent video frames in 100 frames. The terminal traverses 100 video frames and, for example, obtains 99 first distance values ​​corresponding to the A1 key point, that is, the first distance value set includes 99 first distance values. The terminal may obtain the first stability data corresponding to the A1 key point based on the first distance value set. The terminal traverses at least one third key point and obtains the first stability data corresponding to the A1 to A100 key points respectively, and the terminal may obtain the first stability data set corresponding to the A1 to A100 key points.

[0218] According to some embodiments, when the video data is not synthesized from a single image, the video data is composed of a continuous video stream, and the image elements between two adjacent frames in the video data are gradually changing. The terminal obtains a first stability data set corresponding to at least one third key point, and can obtain a first coordinate data set of any third key point in each video frame of at least one third key point. The terminal can smooth the first coordinate data set to obtain a second coordinate data set. Based on the first coordinate data set and the second coordinate data set, the terminal obtains the first stability data corresponding to the third key point, and adds the first stability data to the first stability data set. The terminal traverses at least one third key point and can obtain the first stability data set corresponding to at least one third key point, which can improve the reliability of obtaining the second coordinate data set and improve the reliability of determining the model performance.

[0219] In some embodiments, the first coordinate data set refers to coordinate data corresponding to the third key point. The first coordinate data set does not specifically refer to a particular coordinate data set. The first coordinate data set may, for example, include first horizontal coordinate data and first vertical coordinate data. For example, when the first horizontal coordinate data or the first vertical coordinate data changes, the first coordinate data set may also change accordingly. For example, the terminal may obtain the first coordinate data set corresponding to the third key point in each video frame that includes the third key point.

[0220] Optionally, the second coordinate data set refers to a data set obtained by the terminal smoothing each coordinate data in the first coordinate data set. This smoothing process does not specifically refer to a fixed processing method. This smoothing process includes, but is not limited to, Gaussian smoothing, mean filtering, median filtering, and the like. The number of smoothing processes is at least one. For example, the terminal may smooth each coordinate data in the first coordinate data set three times.

[0221] In some embodiments, the terminal may perform smoothing on the first coordinate data set to obtain the second coordinate data set. For example, the terminal may perform smoothing on the horizontal coordinate data of the first coordinate data set and on the vertical coordinate data of the first coordinate data set.

[0222] It is easy to understand that at least one third key point may include, for example, 100 key points, such as key points A1 to A100. The video data may include, for example, 100 video frames. For example, the video frames 1-10 include the A1 key point. The terminal can obtain the coordinate data of the A1 key point in 10 video frames, that is, the first coordinate data set corresponding to the A1 key point obtained by the terminal may include 10 first coordinate data. For example, the terminal may perform three smoothing processes on each first coordinate data in the first coordinate data set to obtain a second coordinate data set. Based on the first coordinate data set and the second coordinate data set, the terminal can obtain the first stability data corresponding to the A1 key point. The terminal traverses the A1 to A100 key points and can obtain the first stability data set corresponding to the A1 to A100 key points.

[0223] In step S24, a second stability data set corresponding to at least one fourth key point is obtained;

[0224] According to some embodiments, when the video data is synthesized from a single image and the terminal obtains a second stability data set corresponding to at least one fourth key point, the terminal may, for example, obtain a second distance value of any of the at least one fourth key point in an adjacent video frame. The terminal traverses each video frame and obtains a second distance value set corresponding to the fourth key point. Based on the second distance value set, the terminal may obtain second stability data corresponding to the fourth key point and add the second stability data to the second stability data set. By traversing the at least one fourth key point, the terminal may obtain a second stability data set corresponding to the at least one fourth key point.

[0225] In some embodiments, the video data is composed of a single image, that is, the video data includes only one image. The single image does not specifically refer to a fixed image. For example, when the image elements included in the single image change, the single image may also change accordingly, and the video data may also change accordingly.

[0226] Optionally, the second distance value set refers to a collection of distance values ​​corresponding to the fourth key point. The second distance value set does not specifically refer to a fixed distance value set. For example, when the number of video frames included in the video data changes, the second distance value set may also change accordingly. For example, when the value of the second distance value changes, the second distance value set may also change accordingly.

[0227] It is easy to understand that at least one fourth key point may include, for example, 100 key points, such as key points B1 to B100. The video data may include, for example, 100 video frames. The terminal may, for example, obtain the second distance value of the B1 key point in the adjacent video frames in the 100 frames. The terminal traverses the 100 video frames and, for example, obtains 99 second distance values ​​corresponding to the B1 key point, that is, the second distance value set includes 99 second distance values. The terminal may obtain the second stability data corresponding to the B1 key point based on the second distance value set. The terminal traverses at least one fourth key point and obtains the second stability data corresponding to the B1 to B100 key points respectively, and the terminal may obtain the second stability data set corresponding to the B1 to B100 key points.

[0228] According to some embodiments, when the video data is not synthesized by a single picture, the image elements between two adjacent frames in the video data are gradually changing. When the terminal obtains the second stability data set corresponding to at least one fourth key point, the terminal can obtain the third coordinate data set of any fourth key point in each video frame of the at least one fourth key point, and the terminal can smooth the third coordinate data set to obtain the fourth coordinate data set. Based on the third coordinate data set and the fourth coordinate data set, the terminal can obtain the second stability data corresponding to the fourth key point, and add the second stability data to the second stability data set. By traversing at least one fourth key point, the terminal can obtain the second stability data set corresponding to at least one fourth key point, which can improve the reliability of the acquisition of the third coordinate data set and the fourth coordinate data set, and improve the reliability of the determination of the model performance.

[0229] In some embodiments, the third coordinate data set refers to coordinate data corresponding to the fourth key point. The third coordinate data set does not specifically refer to a particular coordinate data set. The third coordinate data set may, for example, include second horizontal coordinate data and second vertical coordinate data. For example, when the second horizontal coordinate data or the second vertical coordinate data changes, the third coordinate data set may also change accordingly. For example, the terminal may obtain the second coordinate data set corresponding to the fourth key point in each video frame that includes the fourth key point.

[0230] Optionally, the fourth coordinate data set refers to a data set obtained by the terminal smoothing each coordinate data in the third coordinate data set. This smoothing process does not specifically refer to a fixed processing method. This smoothing process includes but is not limited to Gaussian smoothing, mean filtering, median filtering, etc. The number of smoothing processes is at least one. For example, the number of times the terminal smoothes each coordinate data in the third coordinate data set may be three times.

[0231] It is easy to understand that at least one fourth key point may include, for example, 100 key points, such as key points B1 to B100. The video data may include, for example, 100 video frames. For example, the video frames 1 to 12 include the B1 key point. The terminal can obtain the coordinate data of the B1 key point in 12 video frames, that is, the terminal obtains the third coordinate data set corresponding to the B1 key point, which may include 12 coordinate data. The terminal performs three smoothing processes on each second coordinate data in the second coordinate data set to obtain a fourth coordinate data set. Based on the third coordinate data set and the fourth coordinate data set, the terminal can obtain the second stability data corresponding to the B1 key point. The terminal traverses the B1 to B100 key points and can obtain the second stability data set corresponding to the B1 to B100 key points.

[0232] In step S25 , the performance of the first keypoint acquisition model and the second keypoint acquisition model is determined based on the first stability data set and the second stability data set.

[0233] In some embodiments, the stability data set may, for example, include a first stability data set and a second stability data set. When the terminal determines the performance of the first key point acquisition model and the second key point acquisition model based on the stability data set, the terminal may, for example, determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set.

[0234] According to some embodiments, the first stability parameter may be, for example, a first average distance value, and the second stability parameter may be, for example, a second average distance value. When the terminal determines the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, the terminal may calculate the average of at least one first average distance value in the first stability data set to obtain a third average distance value, and calculate the average of at least one second average distance value in the second stability data set to obtain a fourth average distance value. If the third average distance value is greater than the fourth average distance value, the terminal may determine that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model. If the third average distance value is equal to the fourth average distance value, the terminal may determine that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model, that is, the performance of the first key point acquisition model is the same as the performance of the second key point acquisition model. If the third average distance value is less than the fourth average distance value, the terminal may determine that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0235] It is easy to understand that the at least one third key point may include, for example, 100 key points, such as key points A1 to A100. The video data may include, for example, 100 video frames. The terminal may, for example, obtain the first distance value of the A1 key point between adjacent video frames in the 100 frames. The terminal traverses the 100 video frames and, for example, obtains 99 first distance values ​​corresponding to the A1 key point, i.e., the first distance value set includes 99 first distance values. The terminal may, for example, obtain the average of the 99 first distance values ​​to obtain a first average distance value. The terminal traverses the at least one third key point and obtains 100 first average distance values. The terminal may obtain the average of the 100 first average distance values ​​to obtain a third average distance value.

[0236] It is easy to understand that the at least one fourth key point may, for example, include 100 key points, such as key points B1 to B100. The video data may, for example, include 100 video frames. The terminal may, for example, obtain the second distance value of the B1 key point between adjacent video frames in the 100 frames. The terminal traverses the 100 video frames and, for example, obtains 99 second distance values ​​corresponding to the B1 key point, i.e., the second distance value set includes 99 second distance values. The terminal may, for example, obtain the average of the 99 second distance values ​​to obtain a second average distance value. The terminal traverses the at least one fourth key point and obtains 100 second average distance values. The terminal may obtain the average of the 100 second average distance values ​​to obtain a fourth average distance value.

[0237] Optionally, the third average distance value may be, for example, 0.0125. The fourth average distance value may be, for example, 0.015. When the terminal determines that the third average distance value is less than the fourth average distance value, the terminal may determine that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0238] Optionally, the third average distance value may be, for example, 0.0125. The fourth average distance value may be, for example, 0.011. When the terminal determines that the third average distance value is greater than the fourth average distance value, the terminal may determine that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model.

[0239] According to some embodiments, the first stability parameter may be, for example, a first maximum distance value, and the second stability parameter may be, for example, a second maximum distance value. When the terminal determines the performance of the first keypoint acquisition model and the second keypoint acquisition model based on the first stability data set and the second stability data set, the terminal may obtain a third maximum distance value from at least one first maximum distance value in the first stability data set, the third maximum distance value being the maximum value among the at least one first maximum distance values, and obtain a fourth maximum distance value from at least one second maximum distance value in the second stability data set, the fourth maximum distance value being the maximum value among the at least one second maximum distance values. If the third maximum distance value is greater than the fourth maximum distance value, the terminal may determine that the performance of the first keypoint acquisition model is lower than that of the second keypoint acquisition model. If the third maximum distance value is equal to the fourth maximum distance value, the terminal may determine that the performance of the first keypoint acquisition model is equal to that of the second keypoint acquisition model, i.e., the performance of the first keypoint acquisition model is the same as that of the second keypoint acquisition model. If the third maximum distance value is less than the fourth maximum distance value, the terminal may determine that the performance of the first keypoint acquisition model is higher than that of the second keypoint acquisition model.

[0240] Optionally, the terminal may, for example, obtain the maximum distance value of 10 first distance values ​​corresponding to a third key point to obtain the first maximum distance value. The terminal traverses at least one third key point and obtains 100 first maximum distance values. The terminal may obtain the third maximum distance value from the 100 first maximum distance values. The terminal may, for example, obtain the second maximum distance value from the maximum value of 10 second distance values ​​corresponding to a fourth key point. The terminal traverses at least one fourth key point and obtains 100 second maximum distance values. The terminal may obtain the fourth maximum distance value from the 100 second maximum distance values.

[0241] In some embodiments, the third maximum distance value may be, for example, 2.0125. The fourth maximum distance value may be, for example, 2.015. When the terminal determines that the third maximum distance value is less than the fourth maximum distance value, the terminal may determine that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0242] Optionally, the third maximum distance value may be, for example, 2.0125. The fourth maximum distance value may be, for example, 2.011. When the terminal determines that the third maximum distance value is greater than the fourth maximum distance value, the terminal may determine that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model.

[0243] According to some embodiments, the first coordinate data set includes first horizontal coordinate data and first vertical coordinate data, and the second coordinate data set includes second horizontal coordinate data and second vertical coordinate data. When the terminal obtains first stability data corresponding to a third key point based on the first coordinate data set and the second coordinate data set, the terminal may calculate a first offset corresponding to the third key point based on the first horizontal coordinate data and the second horizontal coordinate data corresponding to the third key point. Based on the first vertical coordinate data and the second vertical coordinate data in each video frame, the terminal may calculate a second offset corresponding to the third key point and sum the absolute values ​​of the first offset and the second offset to obtain a fifth offset corresponding to the third key point.

[0244] In some embodiments, the third coordinate data set includes third horizontal coordinate data and third vertical coordinate data, and the fourth coordinate data set includes fourth horizontal coordinate data and fourth vertical coordinate data. When the terminal obtains the second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set, the terminal may calculate a third offset corresponding to the fourth key point based on the third horizontal coordinate data and the fourth horizontal coordinate data corresponding to the fourth key point. Based on the absolute sum of the third vertical coordinate data and the fourth vertical coordinate data corresponding to the fourth key point, the terminal may calculate a fourth offset corresponding to the fourth key point. By summing the absolute values ​​of the third offset and the fourth offset, the terminal may obtain a sixth offset corresponding to the fourth key point.

[0245] According to some embodiments, the first stability data set includes at least one fifth offset, and the second stability data set includes at least one sixth offset. The number of the at least one fifth offset corresponds to the number of the at least one third key point. The number of the at least one sixth offset corresponds to the number of the at least one third key point. When the terminal determines the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, it may obtain an average value of the at least one fifth offset to obtain a first average offset, and obtain an average value of the at least one sixth offset to obtain a second average offset. If the first average offset is greater than the second average offset, the terminal may determine that the performance of the first key point acquisition model is lower than that of the second key point acquisition model; if the first average offset is equal to the second average offset, the terminal may determine that the performance of the first key point acquisition model is equal to that of the second key point acquisition model; and if the first average offset is less than the second average offset, the terminal may determine that the performance of the first key point acquisition model is higher than that of the second key point acquisition model.

[0246] In some embodiments, the at least one third key point may include, for example, 100 key points, such as key points A1 to A100. The video data may include, for example, 100 video frames. For example, video frames 1-10 include key point A1. The terminal may obtain the coordinate data of key point A1 in 10 video frames, i.e., the terminal obtains the first coordinate data set corresponding to key point A1. The terminal performs three smoothing processes on each first horizontal coordinate data set of the first coordinate data set to obtain a second horizontal coordinate data set, and performs three smoothing processes on each first vertical coordinate data set of the first coordinate data set to obtain a second vertical coordinate data set, i.e., the terminal can obtain the second coordinate data set. When the terminal obtains the first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set, the terminal can, for example, obtain the first offset between the first horizontal coordinate data and the second horizontal coordinate data of the A1 key point in the same video frame, and the second offset between the first vertical coordinate data and the second vertical coordinate data of the A1 key point in the same video frame. The terminal can obtain 10 first offsets and 10 second offsets. The terminal can sum the absolute values ​​of the 10 first offsets and the 10 second offsets to obtain the fifth offset corresponding to the A1 key point.

[0247] It is easy to understand that the at least one fourth key point may, for example, include 100 key points, such as key points B1 to B100. The video data may, for example, include 100 video frames. For example, video frames 1-12 include key point B1. The terminal may obtain coordinate data for key point B1 in 12 video frames, i.e., the terminal obtains a third coordinate data set corresponding to key point B1. The terminal performs a three-way smoothing process on the second horizontal coordinate data set of the third coordinate data set to obtain a fourth horizontal coordinate data set. When the terminal obtains second stability data corresponding to the fourth key point based on the third and fourth coordinate data sets, the terminal may, for example, obtain a third offset between the third horizontal coordinate data and the fourth horizontal coordinate data of key point B1 in the same video frame, and a fourth offset between the third vertical coordinate data and the fourth vertical coordinate data of key point B1 in the same video frame. The terminal may obtain 12 first offsets and 12 second offsets, and may sum the absolute values ​​of the 12 third offsets and the 12 fourth offsets to obtain a sixth offset corresponding to key point B1.

[0248] Optionally, the terminal may obtain an average value of 100 fifth offsets to obtain a first average offset, obtain an average value of 100 sixth offsets to obtain a second average offset, and obtain a third average offset.

[0249] Optionally, the first average offset may be, for example, 0.325. The second average offset may be, for example, 0.3025. When the terminal determines that the first average offset is greater than the second average offset, the terminal may determine that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model.

[0250] Optionally, the first average offset may be, for example, 0.325. The second average offset may be, for example, 0.3725. When the terminal determines that the first average offset is smaller than the second average offset, the terminal may determine that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0251] According to some embodiments, when the terminal determines that the performance of the first key point acquisition model is higher than that of the second key point acquisition model, the terminal may further issue performance determination information, for example, which may be "the performance of the first key point acquisition model is higher than that of the second key point acquisition model." In this case, the example schematic diagram of the terminal interface may be as follows: Figure 8 shown.

[0252] In some or related embodiments, by obtaining at least one first key point output by a first key point acquisition model for each video frame in the video data and obtaining at least one second key point output by a second key point acquisition model for each video frame in the video data, the at least one first key point can be normalized to obtain at least one third key point, and the at least one second key point can be normalized to obtain at least one fourth key point. The at least one first key point and the at least one second key point can be converted to the same dimension for comparison, which can reduce the situation where the at least one first key point and the at least one second key point are in different dimensions, making the model performance determination unreliable, thereby improving the reliability of the model performance determination. Secondly, by obtaining a first stability data set corresponding to the at least one third key point and a second stability data set corresponding to the at least one fourth key point, the performance of the first key point acquisition model and the second key point acquisition model can be determined based on the first stability data set and the second stability data set. The performance of different key point acquisition models can be determined based on the stability data sets corresponding to different key points, without the need for manual labeling, which can reduce the impact of manual subjective labeling on model performance and reduce the cost of model performance determination. At the same time, determining the model performance based on the stability data set can improve the reliability of the model performance determination.

[0253] Figure 9 FIG. 1 is a block diagram of a device for determining model performance according to an exemplary embodiment. Figure 9 The model performance determination device includes a key point acquisition unit 901, a key point processing unit 902 and a performance determination unit 903, wherein:

[0254] The key point acquisition unit 901 is configured to execute the steps of acquiring at least one first key point output by the first key point acquisition model for each video frame in the video data, and acquiring at least one second key point output by the second key point acquisition model for each video frame in the video data;

[0255] A key point processing unit 902 is configured to perform normalization processing on at least one first key point to obtain at least one third key point, and perform normalization processing on at least one second key point to obtain at least one fourth key point;

[0256] The performance determination unit 903 is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the stability data set corresponding to the at least one third key point and the at least one fourth key point.

[0257] According to some embodiments, Figure 10 is a block diagram of a device for determining model performance according to an exemplary embodiment, referring to Figure 10 The performance determination unit 903 includes a data acquisition subunit 913 and a performance determination subunit 923. The stability data set includes a first stability data set and a second stability data set. The performance determination unit 903 is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the stability data sets corresponding to at least one third key point and at least one fourth key point:

[0258] The data acquisition subunit 913 is configured to acquire a first stability data set corresponding to at least one third key point;

[0259] The data acquisition subunit 913 is further configured to acquire a second stability data set corresponding to at least one fourth key point;

[0260] The performance determination subunit 923 is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set.

[0261] According to some embodiments, the video data is synthesized from a single image, and the data acquisition subunit 913 is configured to execute, when acquiring a first stability data set corresponding to at least one third key point, specifically configured to execute:

[0262] Obtaining a first distance value of any third key point among the at least one third key point in adjacent video frames;

[0263] Traversing each video frame to obtain a first distance value set corresponding to the third key point;

[0264] Based on the first distance value set, obtaining first stability data corresponding to the third key point, and adding the first stability data to the first stability data set;

[0265] Traversing at least one third key point, and acquiring a first stability data set corresponding to the at least one third key point;

[0266] The data acquisition subunit 913 is configured to acquire a second stability data set corresponding to at least one fourth key point, and is specifically configured to execute:

[0267] Obtaining a second distance value of any fourth key point of the at least one fourth key point in adjacent video frames;

[0268] Traversing each video frame to obtain a second distance value set corresponding to the fourth key point;

[0269] Based on the second distance value set, obtaining first stability data corresponding to the fourth key point, and adding the second stability data to the second stability data set;

[0270] At least one fourth key point is traversed to obtain a second stability data set corresponding to the at least one fourth key point.

[0271] According to some embodiments, the first stability parameter is a first average distance value, and the second stability parameter is a second average distance value. The performance determination subunit 923 is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, and is specifically configured to perform:

[0272] calculating an average of at least one first average distance value in the first stability data set to obtain a third average distance value;

[0273] calculating an average of at least one second average distance value in the second stability data set to obtain a fourth average distance value;

[0274] If the third average distance value is greater than the fourth average distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0275] If the third average distance value is equal to the fourth average distance value, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model;

[0276] If the third average distance value is less than the fourth average distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0277] According to some embodiments, the first stability parameter is a first maximum distance value, and the second stability parameter is a second maximum distance value. The performance determination subunit 923 is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set. Specifically, it is configured to perform:

[0278] Obtaining a third maximum distance value among the at least one first maximum distance value in the first stability data set, wherein the third maximum distance value is a maximum first maximum distance value among the at least one first maximum distance value;

[0279] obtaining a fourth maximum distance value of the at least one second maximum distance value in the second stability data set, the fourth maximum distance value being a maximum second maximum distance value among the at least one second maximum distance value;

[0280] If the third maximum distance value is greater than the fourth maximum distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0281] If the third maximum distance value is equal to the fourth maximum distance value, determining that the performance of the first key point acquisition model is the same as the performance of the second key point acquisition model;

[0282] If the third maximum distance value is less than the fourth maximum distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0283] According to some embodiments, the data acquisition subunit 913 is configured to acquire a first stability data set corresponding to at least one third key point, and is specifically configured to perform:

[0284] Obtaining a first coordinate data set of any third key point among the at least one third key point in each video frame;

[0285] Smoothing the first coordinate data set to obtain a second coordinate data set;

[0286] acquiring first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set, and adding the first stability data to the first stability data set;

[0287] Traversing at least one third key point, and acquiring a first stability data set corresponding to the at least one third key point;

[0288] The data acquisition subunit 913 is configured to acquire a second stability data set corresponding to at least one fourth key point, and is specifically configured to execute:

[0289] Obtaining a third coordinate data set of any fourth key point among at least one fourth key point in each video frame;

[0290] Smoothing the third coordinate data set to obtain a fourth coordinate data set;

[0291] acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set, and adding the second stability data to the second stability data set;

[0292] At least one fourth key point is traversed to obtain a second stability data set corresponding to the at least one fourth key point.

[0293] According to some embodiments, the first coordinate data set includes first abscissa data and first ordinate data, and the second coordinate data set includes second abscissa data and second ordinate data. The data acquisition subunit 913 is configured to acquire first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set. Specifically, the data acquisition subunit 913 is configured to perform the following:

[0294] Calculating a first offset corresponding to the third key point based on the first abscissa data and the second abscissa data corresponding to the third key point;

[0295] Calculating a second offset corresponding to a third key point based on the first ordinate data and the second ordinate data in each video frame;

[0296] Summing the absolute values ​​of the first offset and the second offset to obtain a fifth offset corresponding to the third key point;

[0297] The third coordinate data set includes third abscissa data and third ordinate data, and the fourth coordinate data set includes fourth abscissa data and fourth ordinate data. The data acquisition subunit 913 is configured to acquire the second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set. Specifically, the data acquisition subunit 913 is configured to execute:

[0298] Calculating a third offset corresponding to the fourth key point based on the third horizontal coordinate data and the fourth horizontal coordinate data corresponding to the fourth key point;

[0299] Calculate a fourth offset corresponding to the fourth key point based on the absolute sum of the third ordinate data and the fourth ordinate data corresponding to the fourth key point;

[0300] The absolute values ​​of the third offset and the fourth offset are summed to obtain a sixth offset corresponding to the fourth key point.

[0301] According to some embodiments, the first stability data set includes at least one fifth offset, and the second stability data set includes at least one sixth offset. The performance determination subunit 923 is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set. Specifically, the subunit is configured to perform the following:

[0302] Obtaining an average value of at least one fifth offset to obtain a first average offset;

[0303] Obtaining an average value of at least one sixth offset to obtain a second average offset;

[0304] If the first average offset is greater than the second average offset, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model;

[0305] If the first average offset is equal to the second average offset, determining that the performance of the first keypoint acquisition model is equal to the performance of the second keypoint acquisition model;

[0306] If the first average offset is smaller than the second average offset, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

[0307] According to some embodiments, Figure 11 is a block diagram of a device for determining model performance according to an exemplary embodiment, referring to Figure 11 The video data includes a face image, the key point processing unit 902 includes a size acquisition subunit 912, an image plane acquisition subunit 922, and a key point processing subunit 932, and the key point processing unit 902 is configured to perform normalization processing on at least one first key point to obtain at least one third key point:

[0308] The size acquisition subunit 912 is configured to acquire the output image plane size of the first key point acquisition model;

[0309] The image plane acquisition subunit 922 is configured to acquire a minimum rectangular image plane covering at least one first key point in the face image;

[0310] The key point processing subunit 932 is configured to perform normalization processing on the at least one first key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one third key point.

[0311] According to some embodiments, Figure 12 is a block diagram of a device for determining model performance according to an exemplary embodiment, referring to Figure 12The video data includes a face image, the key point processing unit 902 includes an image plane acquisition subunit 922, a distance acquisition subunit 942, and a key point processing subunit 932, and the key point processing unit 902 is configured to perform normalization processing on at least one first key point to obtain at least one third key point:

[0312] The image plane acquisition subunit 922 is configured to acquire a minimum rectangular image plane covering at least one first key point in the face image;

[0313] The distance acquisition subunit 942 is configured to acquire the diagonal distance of the minimum rectangular image plane;

[0314] The key point processing subunit 932 is configured to perform normalization processing on the at least one first key point based on the diagonal distance to obtain at least one third key point.

[0315] According to some embodiments, Figure 13 is a block diagram of a device for determining model performance according to an exemplary embodiment, referring to Figure 13 The video data includes a face image, the key point processing unit 902 includes a spacing value acquisition subunit 952 and a key point processing subunit 932, and the key point processing unit 902 is configured to perform normalization processing on at least one first key point to obtain at least one third key point:

[0316] The distance value obtaining subunit 952 is configured to obtain the pupil distance value in the face image;

[0317] The key point processing subunit 932 is configured to perform normalization processing on the at least one first key point based on the pupil distance value to obtain at least one third key point;

[0318] or,

[0319] The distance value obtaining subunit 952 is configured to obtain the distance value between the outer corners of the eyes in the face image;

[0320] The key point processing subunit 932 is configured to perform normalization processing on the at least one first key point based on the distance between the outer canthi of the binocular eyes to obtain at least one third key point.

[0321] In some or related embodiments, the key point acquisition unit can obtain at least one first key point output by the first key point acquisition model for each video frame in the video data, and obtain at least one second key point output by the second key point acquisition model for each video frame in the video data. The key point processing unit can normalize the at least one first key point to obtain at least one third key point, and normalize the at least one second key point to obtain at least one fourth key point. The at least one first key point and the at least one second key point can be converted into the same dimension for comparison, which can reduce the situation where the at least one first key point and the at least one second key point are in different dimensions, making the model performance determination unreliable. Based on the stability data set corresponding to the at least one third key point and the at least one fourth key point, the performance determination unit can determine the performance of the first key point acquisition model and the second key point acquisition model. Based on the stability data sets corresponding to different key points, the performance of different key point acquisition models can be determined without manual labeling, which can reduce the impact of manual subjective labeling on model performance and the cost of model performance determination. At the same time, the model performance is determined based on the stability data set, which can improve the reliability of the model performance determination.

[0322] See Figure 14 , is a block diagram of a terminal according to an exemplary embodiment. Figure 14 As shown, the terminal 1400 may include: at least one processor 1401 , at least one network interface 1404 , a user interface 1403 , a memory 1405 , and at least one communication bus 1402 .

[0323] The communication bus 1402 is used to implement the connection and communication between these components.

[0324] The user interface 1403 may include a speaker and a display screen. The optional user interface 1403 may also include a standard wired interface or a wireless interface.

[0325] The network interface 1404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0326] Processor 1401 may include one or more processing cores. Processor 1401 utilizes various interfaces and circuits to connect various components within terminal 1400. It executes instructions, programs, code sets, or instruction sets stored in memory 1405, and accesses data stored in memory 1405 to perform various functions and process data for terminal 1400. Optionally, processor 1401 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). Processor 1401 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display; and the modem handles wireless communications. It is understood that the modem may not be integrated into processor 1401 but implemented as a separate chip.

[0327] Among them, the memory 1405 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 1405 includes a non-transitory computer-readable storage medium. The memory 1405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1405 may also be optionally at least one storage device located away from the aforementioned processor 1401. As Figure 14 As shown, the memory 1405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application for identification binding.

[0328] exist Figure 14In the terminal 1400 shown, the user interface 1403 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 1401 can be used to call the application program bound to the identifier stored in the memory 1405 and specifically execute Figure 2-8 Each step in the method embodiment.

[0329] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer-readable storage medium stores the computer program and the computer program is executed by one or more processors, the one or more processors execute Figure 2-8 Each step in the method embodiment.

[0330] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0331] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0332] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0333] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0334] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0335] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0336] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0337] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0338] The above are merely specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to these embodiments herein, but is intended to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

[0339] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0340] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for determining model performance, characterized in that: include: Acquire at least one first key point output by a first key point acquisition model for each video frame in the video data, and acquire at least one second key point output by a second key point acquisition model for each video frame in the video data; Normalizing the at least one first key point to obtain at least one third key point, and normalizing the at least one second key point to obtain at least one fourth key point, wherein the video data includes a face image, and the normalizing the at least one first key point to obtain at least one third key point comprises: obtaining an output image plane size of a model for obtaining the first key point, obtaining a minimum rectangular image plane in the face image covering the at least one first key point, and based on the output image plane size and a size corresponding to the minimum rectangular image plane, normalizing the at least one first key point to obtain at least one third key point; Based on the stability data set corresponding to the at least one third key point and the at least one fourth key point, the performance of the first key point acquisition model and the second key point acquisition model is determined, the stability data set includes a first stability data set and a second stability data set, the first stability parameter in the first stability data set is a first average distance value, and the second stability parameter in the second stability data set is a second average distance value. Based on the first stability data set and the second stability data set, the performance of the first key point acquisition model and the second key point acquisition model is determined, including: calculating the average value of at least one first average distance value in the first stability data set to obtain a third average distance value, calculating the average value of at least one second average distance value in the second stability data set to obtain a fourth average distance value, if the third average distance value is greater than the fourth average distance value, then determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model, if the third average distance value is equal to the fourth average distance value, then determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model, if the third average distance value is less than the fourth average distance value, then determining that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

2. The method according to claim 1, characterized in that The determining the performance of the first key point acquisition model and the second key point acquisition model based on the stability data set corresponding to the at least one third key point and the at least one fourth key point includes: Acquire the first stability data set corresponding to the at least one third key point; Acquire the second stability data set corresponding to the at least one fourth key point; Based on the first stability data set and the second stability data set, performance of the first keypoint acquisition model and the second keypoint acquisition model is determined.

3. The method according to claim 2, characterized in that The video data is synthesized from a single picture, and the acquiring of the first stability data set corresponding to the at least one third key point includes: Obtaining a first distance value of any third key point among the at least one third key point in an adjacent video frame; Traversing the video frames to obtain a first distance value set corresponding to the third key point; Based on the first distance value set, acquiring first stability data corresponding to the third key point, and adding the first stability data to the first stability data set; Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point; The acquiring the second stability data set corresponding to the at least one fourth key point comprises: Obtaining a second distance value of any fourth key point among the at least one fourth key point in an adjacent video frame; Traversing the video frames to obtain a second distance value set corresponding to the fourth key point; Based on the second distance value set, acquiring first stability data corresponding to the fourth key point, and adding the second stability data to the second stability data set; The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

4. The method according to claim 3, characterized in that The first stability parameter is a first maximum distance value, the second stability parameter is a second maximum distance value, and determining the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set includes: Obtaining a third maximum distance value among at least one first maximum distance value in the first stability data set, wherein the third maximum distance value is a maximum first maximum distance value among the at least one first maximum distance value; Acquire a fourth maximum distance value of at least one second maximum distance value in the second stability data set, wherein the fourth maximum distance value is a maximum second maximum distance value among the at least one second maximum distance value; If the third maximum distance value is greater than the fourth maximum distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model; If the third maximum distance value is equal to the fourth maximum distance value, determining that the performance of the first key point acquisition model is the same as the performance of the second key point acquisition model; If the third maximum distance value is smaller than the fourth maximum distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

5. The method according to claim 2, characterized in that: The acquiring the first stability data set corresponding to the at least one third key point comprises: Obtaining a first coordinate data set of any third key point among the at least one third key point in each video frame; Performing smoothing on the first coordinate data set to obtain a second coordinate data set; Based on the first coordinate data set and the second coordinate data set, acquiring first stability data corresponding to the third key point, and adding the first stability data to the first stability data set; Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point; The acquiring the second stability data set corresponding to the at least one fourth key point comprises: Obtaining a third coordinate data set of any fourth key point among the at least one fourth key point in each video frame; Performing smoothing on the third coordinate data set to obtain a fourth coordinate data set; Based on the third coordinate data set and the fourth coordinate data set, acquiring second stability data corresponding to the fourth key point, and adding the second stability data to the second stability data set; The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

6. The method according to claim 5, characterized in that The first coordinate data set includes first abscissa data and first ordinate data, the second coordinate data set includes second abscissa data and second ordinate data, and acquiring first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set includes: Calculating a first offset corresponding to the third key point based on the first horizontal coordinate data and the second horizontal coordinate data corresponding to the third key point; Calculating a second offset corresponding to the third key point based on the first ordinate data and the second ordinate data in each video frame; Summing the absolute values ​​of the first offset and the second offset to obtain a fifth offset corresponding to the third key point; The third coordinate data set includes third abscissa data and third ordinate data, the fourth coordinate data set includes fourth abscissa data and fourth ordinate data, and acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set includes: Calculating a third offset corresponding to the fourth key point based on the third horizontal coordinate data and the fourth horizontal coordinate data corresponding to the fourth key point; Calculate a fourth offset corresponding to the fourth key point based on the sum of the absolute values ​​of the third ordinate data and the fourth ordinate data corresponding to the fourth key point; An absolute value of the third offset and the fourth offset is summed to obtain a sixth offset corresponding to the fourth key point.

7. The method according to claim 6, characterized in that The first stability data set includes at least one fifth offset, the second stability data set includes at least one sixth offset, and determining the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set includes: Obtaining an average value of the at least one fifth offset to obtain a first average offset; Obtaining an average value of the at least one sixth offset to obtain a second average offset; If the first average offset is greater than the second average offset, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model; If the first average offset is equal to the second average offset, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model; If the first average offset is smaller than the second average offset, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

8. The method according to claim 1, characterized in that The video data includes a face image, and the normalizing the at least one first key point to obtain at least one third key point includes: Acquire a minimum rectangular image plane covering the at least one first key point in the face image; Obtaining the diagonal distance of the minimum rectangular image plane; Based on the diagonal distance, the at least one first key point is normalized to obtain at least one third key point.

9. The method according to claim 1, characterized in that: The video data includes a face image, and the normalizing the at least one first key point to obtain at least one third key point includes: Obtaining a pupil distance value in the face image; Based on the pupil distance value, the at least one first key point is normalized to obtain at least one third key point; or, Obtaining a distance between the outer corners of both eyes in the face image; Based on the distance value between the outer canthi of both eyes, the at least one first key point is normalized to obtain at least one third key point.

10. A model performance determination device, characterized in that: include: A key point acquisition unit is configured to execute the acquisition of at least one first key point output by the first key point acquisition model for each video frame in the video data, and the acquisition of at least one second key point output by the second key point acquisition model for each video frame in the video data; A key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point, and to perform normalization processing on the at least one second key point to obtain at least one fourth key point, wherein the video data includes a face image, and the key point processing unit includes a size acquisition subunit, an image plane acquisition subunit and a key point processing subunit, and when the key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point: the size acquisition subunit is configured to perform acquisition of the output image plane size of the first key point acquisition model, and the image plane acquisition subunit is configured to perform acquisition of the minimum rectangular image plane covering the at least one first key point in the face image; the key point processing subunit is configured to perform normalization processing on the at least one first key point based on the output image plane size and the size corresponding to the minimum rectangular image plane to obtain at least one third key point; A performance determination unit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on a stability data set corresponding to the at least one third key point and the at least one fourth key point, the stability data set includes a first stability data set and a second stability data set, the first stability parameter in the first stability data set is a first average distance value, and the second stability parameter in the second stability data set is a second average distance value, the performance determination unit includes a performance determination subunit, which is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set. The body is configured to perform: calculating the average value of at least one first average distance value in the first stability data set to obtain a third average distance value, calculating the average value of at least one second average distance value in the second stability data set to obtain a fourth average distance value, if the third average distance value is greater than the fourth average distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model, if the third average distance value is equal to the fourth average distance value, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model, if the third average distance value is less than the fourth average distance value, determining that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

11. The device according to claim 10, characterized in that The performance determination unit includes a data acquisition subunit, and the performance determination unit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the stability data set corresponding to the at least one third key point and the at least one fourth key point: The data acquisition subunit is configured to execute acquisition of the first stability data set corresponding to the at least one third key point; The data acquisition subunit is further configured to acquire the second stability data set corresponding to the at least one fourth key point.

12. The device according to claim 11, characterized in that The video data is synthesized from a single picture, and the data acquisition subunit is configured to execute, when acquiring the first stability data set corresponding to the at least one third key point, specifically configured to execute: Obtaining a first distance value of any third key point among the at least one third key point in an adjacent video frame; Traversing the video frames to obtain a first distance value set corresponding to the third key point; Based on the first distance value set, acquiring first stability data corresponding to the third key point, and adding the first stability data to the first stability data set; Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point; The data acquisition subunit, when configured to acquire the second stability data set corresponding to the at least one fourth key point, is specifically configured to execute: Obtaining a second distance value of any fourth key point among the at least one fourth key point in an adjacent video frame; Traversing the video frames to obtain a second distance value set corresponding to the fourth key point; Based on the second distance value set, acquiring first stability data corresponding to the fourth key point, and adding the second stability data to the second stability data set; The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

13. The device according to claim 12, characterized in that The first stability parameter is a first maximum distance value, the second stability parameter is a second maximum distance value, and the performance determination subunit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, and is specifically configured to execute: Obtaining a third maximum distance value among at least one first maximum distance value in the first stability data set, wherein the third maximum distance value is a maximum first maximum distance value among the at least one first maximum distance value; Acquire a fourth maximum distance value of at least one second maximum distance value in the second stability data set, wherein the fourth maximum distance value is a maximum second maximum distance value among the at least one second maximum distance value; If the third maximum distance value is greater than the fourth maximum distance value, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model; If the third maximum distance value is equal to the fourth maximum distance value, determining that the performance of the first key point acquisition model is the same as the performance of the second key point acquisition model; If the third maximum distance value is smaller than the fourth maximum distance value, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

14. The device according to claim 11, characterized in that The data acquisition subunit, when configured to acquire the first stability data set corresponding to the at least one third key point, is specifically configured to execute: Obtaining a first coordinate data set of any third key point among the at least one third key point in each video frame; Performing smoothing on the first coordinate data set to obtain a second coordinate data set; Based on the first coordinate data set and the second coordinate data set, acquiring first stability data corresponding to the third key point, and adding the first stability data to the first stability data set; Traversing the at least one third key point, and acquiring the first stability data set corresponding to the at least one third key point; The data acquisition subunit, when configured to acquire the second stability data set corresponding to the at least one fourth key point, is specifically configured to execute: Obtaining a third coordinate data set of any fourth key point among the at least one fourth key point in each video frame; Performing smoothing on the third coordinate data set to obtain a fourth coordinate data set; Based on the third coordinate data set and the fourth coordinate data set, acquiring second stability data corresponding to the fourth key point, and adding the second stability data to the second stability data set; The at least one fourth key point is traversed to obtain the second stability data set corresponding to the at least one fourth key point.

15. The device according to claim 14, characterized in that The first coordinate data set includes first abscissa data and first ordinate data, the second coordinate data set includes second abscissa data and second ordinate data, and the data acquisition subunit is configured to acquire first stability data corresponding to the third key point based on the first coordinate data set and the second coordinate data set, and is specifically configured to execute: Calculating a first offset corresponding to the third key point based on the first horizontal coordinate data and the second horizontal coordinate data corresponding to the third key point; Calculating a second offset corresponding to the third key point based on the first ordinate data and the second ordinate data in each video frame; Summing the absolute values ​​of the first offset and the second offset to obtain a fifth offset corresponding to the third key point; The third coordinate data set includes third abscissa data and third ordinate data, the fourth coordinate data set includes fourth abscissa data and fourth ordinate data, and acquiring second stability data corresponding to the fourth key point based on the third coordinate data set and the fourth coordinate data set includes: Calculating a third offset corresponding to the fourth key point based on the third horizontal coordinate data and the fourth horizontal coordinate data corresponding to the fourth key point; Calculate a fourth offset corresponding to the fourth key point based on the sum of the absolute values ​​of the third ordinate data and the fourth ordinate data corresponding to the fourth key point; An absolute value of the third offset and the fourth offset is summed to obtain a sixth offset corresponding to the fourth key point.

16. The device according to claim 15, characterized in that The first stability data set includes at least one fifth offset, the second stability data set includes at least one sixth offset, and the performance determination subunit is configured to determine the performance of the first key point acquisition model and the second key point acquisition model based on the first stability data set and the second stability data set, and is specifically configured to execute: Obtaining an average value of the at least one fifth offset to obtain a first average offset; Obtaining an average value of the at least one sixth offset to obtain a second average offset; If the first average offset is greater than the second average offset, determining that the performance of the first key point acquisition model is lower than the performance of the second key point acquisition model; If the first average offset is equal to the second average offset, determining that the performance of the first key point acquisition model is equal to the performance of the second key point acquisition model; If the first average offset is smaller than the second average offset, it is determined that the performance of the first key point acquisition model is higher than the performance of the second key point acquisition model.

17. The device according to claim 10, characterized in that The video data includes a face image, the key point processing unit includes an image plane acquisition subunit, a distance acquisition subunit and a key point processing subunit, and the key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point: The image plane acquisition subunit is configured to execute acquisition of a minimum rectangular image plane covering the at least one first key point in the face image; The distance acquisition subunit is configured to acquire the diagonal distance of the minimum rectangular image plane; The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the diagonal distance to obtain at least one third key point.

18. The device according to claim 10, characterized in that The video data includes a face image, the key point processing unit includes a spacing value acquisition subunit and a key point processing subunit, and the key point processing unit is configured to perform normalization processing on the at least one first key point to obtain at least one third key point: The distance value acquisition subunit is configured to execute acquisition of the pupil distance value in the face image; The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the pupil distance value to obtain at least one third key point; or, The distance value acquisition subunit is configured to execute acquisition of the distance value between the outer corners of both eyes in the face image; The key point processing subunit is configured to perform normalization processing on the at least one first key point based on the binocular outer canthus distance value to obtain at least one third key point.

19. A terminal, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images