A method, apparatus, computer device, and storage medium

Abstract

Embodiments of the present application disclose a method, device, computer equipment and storage medium, wherein the method comprises the following steps: obtaining a standard picture and an initial face pose model; generating an initial logical value matrix of the standard picture, an initial prediction box matrix and an initial local degree of freedom matrix through the standard picture and the initial face pose model; generating a category loss function according to the standard logical value matrix and the initial logical value matrix, and generating a standard box loss function according to the standard prediction box matrix and the initial prediction box matrix; generating a candidate prediction box matrix based on the standard prediction box matrix and the initial prediction box matrix; generating a pose loss function according to a standard global degree of freedom matrix, the candidate prediction box matrix and the initial local degree of freedom matrix; and generating a face pose model based on the category loss function, the standard box loss function and the pose loss function. By using the present application, the training speed of a single-stage face detection method in a training process can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and in particular to a method, device, computer equipment and storage medium. BACKGROUND

[0002] It is of great significance to estimate the six degrees of freedom (6DOF) of head pose through a single picture for the interaction between real people and virtual world. At present, the common method is to use a two-stage face detection method and a single-stage face detection method. However, the two-stage face detection algorithm depends on the judgment of a large number of candidate regions, and the calculation amount is large. In addition, there are too many intermediate steps from the detection frame to the 6DOF in the two-stage face detection algorithm, which is easy to cause cumulative error. Although the single-stage face detection can solve the problem of candidate frame dependence, reduce the calculation amount, and reduce the cumulative error by directly predicting the 6DOF, the initial prediction frame of the single-stage face detection method is randomly initialized, so the single-stage face detection method will lead to the problem of non-convergence in the training process. SUMMARY

[0003] The embodiments of the present application provide a method, device, computer equipment and storage medium, which can improve the training speed of the single-stage face detection method in the training process.

[0004] In one aspect, the embodiments of the present application provide a method, which can include:

[0005] obtaining a standard picture and an initial face pose model; the standard picture carries a standard logical value matrix, a standard prediction frame matrix and a standard global degree of freedom matrix;

[0006] generating the initial logical value matrix, the initial prediction frame matrix and the initial local degree of freedom matrix of the standard picture through the standard picture and the initial face pose model;

[0007] generating a category loss function according to the standard logical value matrix and the initial logical value matrix, and generating a standard frame loss function according to the standard prediction frame matrix and the initial prediction frame matrix;

[0008] generating a candidate prediction frame matrix based on the standard prediction frame matrix and the initial prediction frame matrix;

[0009] generating a pose loss function according to the standard global degree of freedom matrix, the candidate prediction frame matrix and the initial local degree of freedom matrix;

[0010] generating a face pose model based on the category loss function, the standard frame loss function and the pose loss function; the face pose model is used to predict the pose information of the face in the picture.

[0011] In one possible implementation, the method further includes:

[0012] The marking of the face in the standard picture generates a standard logic value matrix, a standard prediction frame matrix and a standard degree of freedom matrix of the standard picture.

[0013] In an implementable embodiment, the generating of the initial logic value matrix, the initial prediction frame matrix and the initial local degree of freedom matrix of the standard picture by the standard picture and the initial face posture model comprises:

[0014] The feature extraction module in the initial face posture model extracts features of the standard picture to generate the initial logic value matrix, the initial prediction frame matrix and the initial degree of freedom matrix of the standard picture.

[0015] The label selection module in the initial face posture model filters the initial degree of freedom matrix to generate an initial local degree of freedom matrix.

[0016] In an implementable embodiment, the generating of the candidate prediction frame matrix based on the standard prediction frame matrix and the initial prediction frame matrix comprises:

[0017] The label selection module in the initial face posture model filters the initial prediction frame matrix to generate an initial filtered prediction frame matrix and filters the standard prediction frame matrix to generate a standard filtered prediction frame matrix.

[0018] Based on the standard filtered prediction frame matrix and the initial filtered prediction frame matrix, a first prediction frame loss value at a first time is obtained and a second prediction frame loss value at a target time is obtained; the target time is a current training time of the initial face posture model, and the first time is a training time of a previous round of training of the current training time.

[0019] The confidence state of the target time is determined according to the first prediction frame loss value and the second prediction frame loss value, and the candidate prediction frame matrix is determined according to the confidence state.

[0020] In an implementable embodiment, the determining of the confidence state of the target time according to the first prediction frame loss value and the second prediction frame loss value and the determining of the candidate prediction frame matrix according to the confidence state comprises:

[0021] The ratio between the second prediction frame loss value and the first prediction frame loss value is obtained.

[0022] If the ratio is greater than a ratio threshold value, the confidence state of the target time is not passed, and the standard filtered prediction frame matrix is determined as the candidate prediction frame matrix.

[0023] If the ratio is less than or equal to a ratio threshold, the confidence state of the target moment is passed, and the initial screening prediction frame matrix is determined as the candidate prediction frame matrix.

[0024] In an implementable embodiment, the generating a pose loss function according to the standard global degree of freedom matrix, the candidate prediction frame matrix and the initial local degree of freedom matrix comprises:

[0025] The initial local degree of freedom matrix is converted into an initial global degree of freedom matrix based on the candidate prediction frame matrix.

[0026] The pose loss function is generated according to the initial global degree of freedom matrix and the standard global degree of freedom matrix.

[0027] In an implementable embodiment, the generating a face pose model based on the category loss function, the standard frame loss function and the pose loss function comprises:

[0028] The parameters of the initial face pose model are adjusted according to the category loss function, the standard frame loss function and the pose loss function.

[0029] When the parameters of the initial face pose model satisfy a convergence condition, the initial face pose model after the adjustment of the parameters is determined as the face pose model.

[0030] Embodiments of the present application provide, on one hand, an apparatus, which can comprise:

[0031] A data acquisition unit is configured to acquire a standard picture and an initial face pose model, wherein the standard picture carries a standard logic value matrix, a standard prediction frame matrix and a standard global degree of freedom matrix.

[0032] An initial matrix generation unit is configured to generate the standard picture initial logic value matrix, the initial prediction frame matrix and the initial local degree of freedom matrix by using the standard picture and the initial face pose model.

[0033] A first loss function generation unit is configured to generate a category loss function according to the standard logic value matrix and the initial logic value matrix, and generate a standard frame loss function according to the standard prediction frame matrix and the initial prediction frame matrix.

[0034] A candidate frame generation unit is configured to generate a candidate prediction frame matrix based on the standard prediction frame matrix and the initial prediction frame matrix.

[0035] A second loss function generation unit is configured to generate a pose loss function according to the standard global degree of freedom matrix, the candidate prediction frame matrix and the initial local degree of freedom matrix.

[0036] The model generation unit is configured to generate a face pose model based on the category loss function, the standard frame loss function, and the pose loss function. The face pose model is configured to predict pose information of a face in a picture.

[0037] In an implementation, the method further includes:

[0038] The standard matrix generation unit is configured to generate a standard logical value matrix, a standard prediction frame matrix, and a standard degree of freedom matrix for the face in the standard picture.

[0039] In an implementation, the initial matrix generation unit is specifically configured to:

[0040] The feature extraction module in the initial face pose model is configured to perform feature extraction on the standard picture to generate an initial logical value matrix, an initial prediction frame matrix, and an initial degree of freedom matrix for the standard picture.

[0041] The label selection module in the initial face pose model is configured to perform screening processing on the initial degree of freedom matrix to generate an initial local degree of freedom matrix.

[0042] In an implementation, the candidate frame generation unit includes:

[0043] The confidence state determination subunit is configured to perform screening processing on the initial prediction frame matrix by the label selection module in the initial face pose model to generate an initial screened prediction frame matrix, and perform screening processing on the standard prediction frame matrix to generate a standard screened prediction frame matrix.

[0044] Based on the standard screened prediction frame matrix and the initial screened prediction frame matrix, a first prediction frame loss value at a first time is obtained, and a second prediction frame loss value at a target time is obtained. The target time is a current training time of the initial face pose model, and the first time is a training time of a previous round of training of the current training time.

[0045] The candidate frame generation subunit is configured to determine a confidence state of the target time according to the first prediction frame loss value and the second prediction frame loss value, and determine a candidate prediction frame matrix according to the confidence state.

[0046] In an implementation, the candidate frame generation subunit is specifically configured to:

[0047] A ratio between the second prediction frame loss value and the first prediction frame loss value is obtained.

[0048] If the ratio is greater than a ratio threshold value, the confidence state of the target time is not passed, and the standard screened prediction frame matrix is determined as the candidate prediction frame matrix.

[0049] If the ratio is less than or equal to a ratio threshold value, the confidence state of the target moment is passed, and the initial screening prediction frame matrix is determined as the candidate prediction frame matrix.

[0050] In an implementation, the second loss function generation unit is specifically configured to:

[0051] Based on the candidate prediction frame matrix, the initial local degree of freedom matrix is converted into an initial global degree of freedom matrix;

[0052] According to the initial global degree of freedom matrix and the standard global degree of freedom matrix, a pose loss function is generated.

[0053] In an implementation, the model generation unit is specifically configured to:

[0054] According to the category loss function, the standard frame loss function and the pose loss function, the parameters of the initial face pose model are adjusted.

[0055] When the parameters of the initial face pose model satisfy a convergence condition, the initial face pose model after the parameters are adjusted is determined as a face pose model.

[0056] Embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program is adapted to be loaded by a processor and execute the above method steps.

[0057] Embodiments of the present application provide a computer device, which includes a processor, a memory and a network interface. The processor is connected with the memory and the network interface. The network interface is configured to provide network communication function. The memory is configured to store program code. The processor is configured to call the program code to execute the above method steps.

[0058] Embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the computer device executes the above method steps.

[0059] In the embodiment of the present application, by acquiring a standard picture and an initial face pose model, the standard picture carries a standard logical value matrix, a standard prediction frame matrix and a standard global degree of freedom matrix, further generating the standard picture initial logical value matrix, the initial prediction frame matrix and the initial local degree of freedom matrix from the standard picture and the initial face pose model, generating a category loss function according to the standard logical value matrix and the initial logical value matrix, generating a standard frame loss function according to the standard prediction frame matrix and the initial prediction frame matrix, generating a candidate prediction frame matrix based on the standard prediction frame matrix and the initial prediction frame matrix, further generating a pose loss function according to the standard global degree of freedom matrix, the candidate prediction frame matrix and the initial local degree of freedom matrix, and finally generating a face pose model based on the category loss function, the standard frame loss function and the pose loss function. By using the above method, the problem of non-convergence in the training process of the single-stage face detection method due to the random initialization of the initial prediction frame can be avoided, and the training speed of the single-stage face detection method in the training process is improved. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0061] Figure 1 is a network architecture diagram of a data processing method provided by an embodiment of the present application;

[0062] Figure 2 is a flowchart of a data processing method provided by an embodiment of the present application;

[0063] Figure 3 is a flowchart of a data processing method provided by an embodiment of the present application;

[0064] Figure 4 is an example schematic diagram of a data processing method provided by an embodiment of the present application;

[0065] Figure 5 is a structural schematic diagram of a data processing device provided by an embodiment of the present application;

[0066] Figure 6 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0067] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0068] Please refer to Figure 1 , Figure 1 is a network architecture diagram of data processing provided by the embodiments of the present application. The network architecture diagram can include a business server 100 and a user terminal cluster, which can include a user terminal 10a, a user terminal 10b,..., and a user terminal 10c. There can be a communication connection between the user terminal cluster, for example, there is a communication connection between the user terminal 10a and the user terminal 10b, and there is a communication connection between the user terminal 10b and the user terminal 10c. Any user terminal in the user terminal cluster can have a communication connection with the business server 100, for example, there is a communication connection between the user terminal 10a and the business server 100, and there is a communication connection between the user terminal 10b and the business server 100.

[0069] The user terminal cluster (also including the user terminal 10a, the user terminal 10b, and the user terminal 10c) described above can be integrated with a target application. Optionally, the target application can include an application with the function of displaying data information such as text, images, and videos. The database 10d stores an initial face pose model and a training number of a training model. In a feasible implementation manner, the business server 100 obtains a standard picture and an initial face pose model, the standard picture carries a standard logical value matrix, a standard prediction box matrix, and a standard global degree of freedom matrix, further generates an initial logical value matrix, an initial prediction box matrix, and an initial local degree of freedom matrix from the standard picture and the initial face pose model, generates a category loss function according to the standard logical value matrix and the initial logical value matrix, generates a standard box loss function according to the standard prediction box matrix and the initial prediction box matrix, generates a candidate prediction box matrix based on the standard prediction box matrix and the initial prediction box matrix, and further, the business server 100 generates a pose loss function according to the standard global degree of freedom matrix, the candidate prediction box matrix, and the initial local degree of freedom matrix. Finally, the business server 100 generates a face pose model based on the category loss function, the standard box loss function, and the pose loss function. Optionally, the user terminal described above can be any one selected from the user terminal cluster in the corresponding embodiment, for example, the user terminal can be the user terminal 10b. Figure 1 The user terminal selected from the user terminal cluster in the corresponding embodiment, for example, the user terminal can be the user terminal 10b.

[0070] It can be understood that the method provided in the embodiments of the present application can be executed by a computer device, which includes but is not limited to a terminal or a server. The service server 100 in the embodiments of the present application can be a computer device, and the user terminals in the user terminal cluster can also be computer devices, which are not limited herein. The service server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can include a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart television, a smart speaker, a desktop computer, a smart watch, and other smart terminals with image recognition functions, but is not limited thereto. The user terminal and the service server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0071] Further, for the convenience of understanding, please refer to Figure 2 , Figure 2 is a flow diagram of a data processing method provided by the embodiments of the present application. The method can be executed by a user terminal (for example, the user terminal shown in the above Figure 1 , or can be executed by a user terminal and a service server (such as the service server 100 in the corresponding embodiments of the above Figure 1 . For the convenience of understanding, the embodiments will be described by taking the method executed by the above user terminal as an example. The data processing method can at least include the following steps S101-S106:

[0072] S101, obtaining a standard picture and an initial face pose model;

[0073] Specifically, the user terminal can obtain a standard picture and an initial face pose model. It can be understood that the initial face pose model is a face pose model before training. The face pose model can predict the pose information of the face in the picture. The standard picture is used to train the initial face pose model. The standard picture carries a training label. The training label includes a standard logic value matrix, a standard prediction box matrix, and a standard global degree of freedom matrix. The logic value matrix is a two-dimensional vector, which is used to identify whether it contains a face. The prediction box matrix is a four-dimensional vector, which is used to identify the position information of the detection box in the face detection process. The global degree of freedom matrix is the degree of freedom information or the pose information of the face, which is used to identify the direction and angle information of the face.

[0074] S102, generating an initial logic value matrix, an initial prediction box matrix, and an initial local degree of freedom matrix of the standard picture from the standard picture and the initial face pose model;

[0075] Specifically, the user terminal extracts features of the standard picture through a feature extraction module in the initial face posture model to generate an initial logical value matrix, an initial prediction box matrix and an initial degree of freedom matrix of the standard picture. The feature extraction module is an important component of the initial face posture model. For example, the feature extraction module can be a feature pyramid network (FPN). Further, the initial degree of freedom matrix is filtered through a label selection module in the initial face posture model to generate an initial local degree of freedom matrix. The filtering process can obtain a more representative matrix. For example, 200*200 initial degree of freedom matrices are obtained through feature extraction, and 38 initial local degree of freedom matrices are generated after filtering by the label selection module.

[0076] S103, generating a category loss function according to the standard logical value matrix and the initial logical value matrix, and generating a standard box loss function according to the standard prediction box matrix and the initial prediction box matrix;

[0077] Specifically, the user terminal generates a category loss function according to the standard logical value matrix and the initial logical value matrix, and generates a standard box loss function according to the standard prediction box matrix and the initial prediction box matrix. It can be understood that the category loss function is determined according to the difference between the standard logical value matrix and the initial logical value matrix, and the standard box loss function is determined according to the difference between the standard prediction box matrix and the initial prediction box matrix.

[0078] S104, generating a candidate prediction box matrix based on the standard prediction box matrix and the initial prediction box matrix;

[0079] Specifically, the user terminal filters the initial prediction box matrix through a label selection module in the initial face posture model to generate an initial filtered prediction box matrix, and filters the standard prediction box matrix to generate a standard filtered prediction box matrix. Further, based on the standard filtered prediction box matrix and the initial filtered prediction box matrix, a first prediction box loss value at a first time is obtained, and a second prediction box loss value at a target time is obtained. The prediction box loss value is the difference between the standard filtered prediction box matrix and the initial filtered prediction box matrix. The target time is the current training time of the initial face posture model, and the first time is the training time of the previous round of training of the current training time.

[0080] Further, the confidence state of the target time is determined according to the first prediction box loss value and the second prediction box loss value, and a candidate prediction box matrix is determined according to the confidence state. The specific process of determining the candidate prediction box matrix is as follows: a ratio between the second prediction box loss value and the first prediction box loss value is obtained, if the ratio is greater than a ratio threshold, the confidence state of the target time is not passed, the standard screening prediction box matrix is determined as the candidate prediction box matrix, if the ratio is less than or equal to the ratio threshold, the confidence state of the target time is passed, and the initial screening prediction box matrix is determined as the candidate prediction box matrix.

[0081] In S105, a pose loss function is generated according to the standard global degree of freedom matrix, the candidate prediction box matrix, and the initial local degree of freedom matrix.

[0082] Specifically, the user terminal converts the initial local degree of freedom matrix into an initial global degree of freedom matrix based on the candidate prediction box matrix, and the global degree of freedom matrix has a unified scale relative to the local degree of freedom matrix. Further, the pose loss function is generated according to the initial global degree of freedom matrix and the standard global degree of freedom matrix, that is, the pose loss function is determined according to the difference between the initial global degree of freedom matrix and the standard global degree of freedom matrix.

[0083] In S106, a face pose model is generated based on the category loss function, the standard box loss function, and the pose loss function. The face pose model is used to predict the pose information of a face in a picture.

[0084] Specifically, if the category loss function, the standard box loss function, and the pose loss function are not less than a function threshold, the user terminal adjusts the parameters of the initial face pose model, and further generates the category loss function, the standard box loss function, and the pose loss function according to the model after adjusting the parameters, if the category loss function, the standard box loss function, and the pose loss function are less than the function threshold, that is, when the parameters of the initial face pose model satisfy the convergence condition, the initial face pose model after adjusting the parameters is determined as the face pose model.

[0085] In the embodiment of the present application, by acquiring a standard picture and an initial face pose model, the standard picture carries a standard logical value matrix, a standard prediction box matrix and a standard global degree of freedom matrix, further generating the standard picture initial logical value matrix, the initial prediction box matrix and the initial local degree of freedom matrix from the standard picture and the initial face pose model, generating a category loss function according to the standard logical value matrix and the initial logical value matrix, generating a standard box loss function according to the standard prediction box matrix and the initial prediction box matrix, generating a candidate prediction box matrix based on the standard prediction box matrix and the initial prediction box matrix, further generating a pose loss function according to the standard global degree of freedom matrix, the candidate prediction box matrix and the initial local degree of freedom matrix, and finally generating a face pose model based on the category loss function, the standard box loss function and the pose loss function. By using the above method, the problem of non-convergence in the training process of the single-stage face detection method due to the random initialization of the initial prediction box can be avoided, and the training speed of the single-stage face detection method in the training process is improved.

[0086] Please refer to Figure 3 , Figure 3 is a flowchart of a data processing method provided by the embodiment of the present application. The method can be executed by a user terminal (for example, the user terminal shown in the above Figure 1 , or can be executed by the user terminal and a service server (for example, the service server 100 in the corresponding embodiment of the above Figure 1 . For ease of understanding, the embodiment is described by taking the method executed by the above user terminal as an example. The data processing method can at least include the following steps S201-S207:

[0087] S201, marking the face in the standard picture to generate a standard logical value matrix, a standard prediction box matrix and a standard degree of freedom matrix of the standard picture.

[0088] Specifically, the user terminal marks the face in the standard picture to generate a standard logical value matrix, a standard prediction box matrix and a standard degree of freedom matrix of the standard picture. It can be understood that the user can mark the standard picture by using a manual marking method or a standard model to obtain the standard logical value matrix, the standard prediction box matrix and the standard degree of freedom matrix of the standard picture.

[0089] S202, acquiring a standard picture and an initial face pose model; the standard picture carries a standard logical value matrix, a standard prediction box matrix and a standard global degree of freedom matrix;

[0090] S203, generating the standard picture initial logical value matrix, the initial prediction box matrix and the initial local degree of freedom matrix from the standard picture and the initial face pose model;

[0091] S204, generating a category loss function according to the standard logic value matrix and the initial logic value matrix, and generating a standard frame loss function according to the standard prediction frame matrix and the initial prediction frame matrix;

[0092] S205, generating a candidate prediction frame matrix based on the standard prediction frame matrix and the initial prediction frame matrix;

[0093] S206, generating a pose loss function according to the standard global degree of freedom matrix, the candidate prediction frame matrix and the initial local degree of freedom matrix;

[0094] S207, generating a face pose model based on the category loss function, the standard frame loss function and the pose loss function; the face pose model is used to predict the pose information of the face in the picture.

[0095] Wherein, the steps S202-S207 of the embodiment of the present application are described in detail in the specific description of the steps S101-S106 of the embodiment shown in Figure 2 The specific description of the steps S101-S106 of the embodiment shown in

[0096] Please refer to Figure 4 , Figure 4 is the flowchart of the data processing method provided by the embodiment of the present application. As shown in Figure 4As shown, the standard picture is an RGB three-channel picture with a length and width of 800 [800, 800, 3], and the features are extracted by the feature extraction module (backbone) to obtain the initial logical value matrix pred_class_logits representing the predicted category, such as [200, 200, 2], for example, the possible value of the prediction vector corresponding to the 5th row and the 10th column is [0.1, 0.9], which indicates that the class label of the pixel point corresponding to the 20th row and the 40th column in the 800x 800 image is the subscript 1 corresponding to the larger logical value 0.9, which means that the subscript 1 represents a person, and it means that there is a 0.9 probability of a person at the corresponding pixel point; After feature extraction, an initial prediction box matrix Proposals representing the predicted box is obtained, such as [200, 200, 4], for example, the possible value of the prediction vector corresponding to the 5th row and the 10th column is [0.1, 0.2, 0.1, 0.2], which corresponds to [left, top, right, bottom], indicating the coordinates of the left upper corner and the right lower corner of a matrix box, and the predicted box of the face region corresponding to the pixel point at the 20th row and the 40th column in the 800x 800 image is [800*0.1, 800*0.2, 800*0.1, 800*0.2] = [80, 160, 80, 160], that is, the coordinate value of the left upper corner of the predicted box is [x, y] = [80, 160], and the coordinate value of the right lower corner is [x, y] = [80, 160]; After feature extraction, an initial freedom matrix pred_local_poses_feature representing the pose is obtained, such as [200, 200, 6], for example, the possible value of the prediction vector corresponding to the 5th row and the 10th column is [pitch, roll, yaw, x, y, z] = [0, 0, 0, 0.1, 0.2, 0.3], which indicates the pose angle information of the person head image at the 20th row and the 40th column in the original image.

[0097] Further, a category loss function is generated according to the standard logical value matrix and the initial logical value matrix, and a standard box loss function is generated according to the standard prediction box matrix and the initial prediction box matrix.

[0098] Since the above initial prediction box matrix is randomly generated, directly generating a pose loss function using the initial prediction box matrix can easily lead to non-convergent training results.

[0099] Therefore, the specific method is to record the first prediction box loss value box_loss@t=0 obtained by prediction at the first time t=0 in the training, and a represents a ratio threshold, which is usually required to be small to an acceptable range (specifically according to the task itself, which can be taken as 0.01 here).

[0100] Case one: if t=t0 target moment, the second prediction box loss value box_loss@t=t0<=a*box_loss@t=0 predicted is obtained, which indicates that the initial prediction box obtained by prediction has deviated from the initial random state and is close to the label standard prediction box, that is, the network can predict a series of coarse-grained prediction boxes at t=t0 target moment, and the initial screening prediction box matrix corresponding to the actual target obtained by the label selection module also has a certain degree of credibility, so the initial screening prediction box matrix is used as the candidate prediction box matrix, that is, the actual prediction box is used as the input of the pose loss module.

[0101] Case two: if t=t0 target moment, the second prediction box loss value box_loss@t=t0>a*box_loss@t=0 predicted is obtained, which indicates that the initial prediction box has high credibility and has not deviated from the initial random initialization state, so if the actual prediction box is used as the input of the pose loss module, it is easy to cause training not to converge, so the standard screening prediction box matrix is used as the candidate prediction box matrix, that is, the label box is used as the input of the pose loss module.

[0102] Finally, the face pose model is generated based on the category loss function, the standard box loss function and the pose loss function.

[0103] In the embodiment of the present application, the standard picture and the initial face pose model are obtained, the standard picture carries a standard logical value matrix, a standard prediction box matrix and a standard global degree of freedom matrix, the standard picture and the initial face pose model are further used to generate an initial logical value matrix, an initial prediction box matrix and an initial local degree of freedom matrix of the standard picture, a category loss function is generated according to the standard logical value matrix and the initial logical value matrix, a standard box loss function is generated according to the standard prediction box matrix and the initial prediction box matrix, a candidate prediction box matrix is generated based on the standard prediction box matrix and the initial prediction box matrix, further, a pose loss function is generated according to the standard global degree of freedom matrix, the candidate prediction box matrix and the initial local degree of freedom matrix, and finally a face pose model is generated based on the category loss function, the standard box loss function and the pose loss function. By using the above method, the problem of training not converging in the training process of the single-stage face detection method due to the initial prediction box being randomly initialized can be avoided, and the training speed of the single-stage face detection method in the training process is improved.

[0104] Please refer to Figure 5 , Figure 5Fig. 1 is a structural schematic diagram of a data processing apparatus provided in an embodiment of the present application. The data processing apparatus can be a computer program (including program code) running in a computer device, for example, the data processing apparatus is an application software; the apparatus can be used to execute corresponding steps in the method provided in the embodiments of the present application. As shown in Fig. 1, the data processing apparatus 1 provided in the embodiments of the present application can include a data acquisition unit 11, an initial matrix generation unit 12, a first loss function generation unit 13, a candidate box generation unit 14, a second loss function generation unit 15, and a model generation unit 16. Figure 5

[0105] The data acquisition unit 11 is configured to acquire a standard picture and an initial face pose model; the standard picture carries a standard logical value matrix, a standard prediction box matrix, and a standard global degree of freedom matrix.

[0106] The initial matrix generation unit 12 is configured to generate an initial logical value matrix, an initial prediction box matrix, and an initial local degree of freedom matrix of the standard picture from the standard picture and the initial face pose model.

[0107] The first loss function generation unit 13 is configured to generate a category loss function according to the standard logical value matrix and the initial logical value matrix, and generate a standard box loss function according to the standard prediction box matrix and the initial prediction box matrix.

[0108] The candidate box generation unit 14 is configured to generate a candidate prediction box matrix based on the standard prediction box matrix and the initial prediction box matrix.

[0109] The second loss function generation unit 15 is configured to generate a pose loss function according to the standard global degree of freedom matrix, the candidate prediction box matrix, and the initial local degree of freedom matrix.

[0110] The model generation unit 16 is configured to generate a face pose model based on the category loss function, the standard box loss function, and the pose loss function; the face pose model is used to predict pose information of a face in a picture.

[0111] Please refer to Figure 5 , the data processing apparatus 1 provided in the embodiments of the present application can further include a standard matrix generation unit 17.

[0112] The standard matrix generation unit 17 is configured to mark a face in the standard picture to generate a standard logical value matrix, a standard prediction box matrix, and a standard degree of freedom matrix of the standard picture.

[0113] In a possible implementation, the initial matrix generation unit 12 is specifically configured to:

[0114] ​The feature extraction module in the initial human face posture model is used for performing feature extraction on the standard picture to generate an initial logical value matrix, an initial prediction box matrix and an initial degree of freedom matrix of the standard picture;

[0115] The label selection module in the initial human face posture model is used for performing screening processing on the initial degree of freedom matrix to generate an initial local degree of freedom matrix.

[0116] See Figure 5 The candidate box generation unit 14 in the embodiment of the present application can further include a confidence state determination subunit 141 and a candidate box generation subunit 142.

[0117] The confidence state determination subunit 141 is configured to perform screening processing on the initial prediction box matrix by the label selection module in the initial human face posture model to generate an initial screening prediction box matrix, and perform screening processing on the standard prediction box matrix to generate a standard screening prediction box matrix.

[0118] Based on the standard screening prediction box matrix and the initial screening prediction box matrix, a first prediction box loss value at a first time point is obtained, and a second prediction box loss value at a target time point is obtained; the target time point is a current training time point of the initial human face posture model, and the first time point is a training time point of a previous round of training of the current training time point.

[0119] The candidate box generation subunit 142 is configured to determine a confidence state of the target time point according to the first prediction box loss value and the second prediction box loss value, and determine a candidate prediction box matrix according to the confidence state.

[0120] In a feasible implementation, the candidate box generation subunit 142 is specifically configured to:

[0121] Obtain a ratio between the second prediction box loss value and the first prediction box loss value.

[0122] If the ratio is greater than a ratio threshold value, the confidence state of the target time point is fail, and the standard screening prediction box matrix is determined as the candidate prediction box matrix.

[0123] If the ratio is less than or equal to the ratio threshold value, the confidence state of the target time point is pass, and the initial screening prediction box matrix is determined as the candidate prediction box matrix.

[0124] In a feasible implementation, the second loss function generation unit 15 is specifically configured to:

[0125] Based on the candidate prediction box matrix, the initial local degree of freedom matrix is converted into an initial global degree of freedom matrix.

[0126] generate a pose loss function according to the initial global degree of freedom matrix and the standard global degree of freedom matrix.

[0127] In an implementation, the model generating unit 16 is specifically configured to:

[0128] adjust parameters of the initial face pose model according to the category loss function, the standard bounding box loss function and the pose loss function;

[0129] determine the initial face pose model with the adjusted parameters as the face pose model when the parameters of the initial face pose model satisfy a convergence condition.

[0130] In the embodiments of the present application, the standard picture carries a standard logical value matrix, a standard prediction bounding box matrix and a standard global degree of freedom matrix by obtaining the standard picture and the initial face pose model, and the initial logical value matrix, the initial prediction bounding box matrix and the initial local degree of freedom matrix of the standard picture are further generated according to the standard picture and the initial face pose model. The category loss function is generated according to the standard logical value matrix and the initial logical value matrix, the standard bounding box loss function is generated according to the standard prediction bounding box matrix and the initial prediction bounding box matrix, the candidate prediction bounding box matrix is generated based on the standard prediction bounding box matrix and the initial prediction bounding box matrix, and further, the pose loss function is generated according to the standard global degree of freedom matrix, the candidate prediction bounding box matrix and the initial local degree of freedom matrix. Finally, the face pose model is generated based on the category loss function, the standard bounding box loss function and the pose loss function. By using the above method, the problem that the training does not converge in the training process of the single-stage face detection method due to the random initialization of the initial prediction bounding box can be avoided, and the training speed of the single-stage face detection method in the training process is improved.

[0131] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of a computer device provided by the embodiments of the present application. As shown in Figure 6As shown, the computer device 1000 can include at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is configured to realize the connection and communication between the components. The user interface 1003 can include a display screen, and the optional user interface 1003 can further include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a random access memory (RAM), and can also be a non-volatile memory (NVM), such as at least one disk memory. The memory 1005 can also be at least one storage device located away from the aforementioned processor 1001. For example, Figure 6 As shown, the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a data processing application program.

[0132] In Figure 6 As shown in the computer device 1000, the network interface 1004 can provide network communication functions, and the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the data processing application program stored in the memory 1005 to realize the aforementioned Figures 2-4 The description of the data processing method in any of the corresponding embodiments is not repeated here.

[0133] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the aforementioned Figures 2-4 The description of the data processing method in any of the corresponding embodiments is also not repeated here. Figure 5 The description of the data processing device in any of the corresponding embodiments is not repeated here. In addition, the description of the beneficial effects of using the same method is also not repeated here.

[0134] In addition, it should be noted that the embodiments of the present application further provide a computer readable storage medium, and the computer readable storage medium stores the computer program executed by the aforementioned data processing device, and the computer program includes program instructions. When the processor executes the program instructions, the aforementioned Figures 2-4The description of the data processing method in any of the corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated here. For technical details not disclosed in the computer-readable storage medium embodiments involved in the present application, please refer to the description of the method embodiments of the present application. As an example, the program instructions can be deployed to execute on one computing device, or on multiple computing devices located in one place, or on multiple computing devices distributed in multiple places and interconnected through a communication network, which can constitute a blockchain system.

[0135] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The above-mentioned program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the above-mentioned embodiment methods. Among them, the above-mentioned computer-readable storage medium can be an internal storage unit of the above-mentioned data processing apparatus or the above-mentioned device, such as the hard disk or the memory of an electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The above-mentioned computer-readable storage medium can also include a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory, etc. Further, the computer-readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0136] The terms "first", "second", etc. in the claims and specification and drawings of the present application are used for distinguishing between similar objects, not for describing a specific sequential or chronological order. Moreover, the terms "comprising", "including", "containing", and "having" and their variations, are intended to be inclusive, and do not exclude additional non-recited members, steps, or units. For example, a process, method, system, product, or apparatus that comprises a list of steps or units is not necessarily limited to the listed steps or units, but can include additional or other steps or units not expressly listed or inherent to such process, method, system, product, or apparatus. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely examples from a great array of embodiments that can be claimed under this application. In this application, the use of the singular includes the plural, the use of "or" means "and / or", and the use of "one" means "one or more". The use of "including" means "including but not limited to".

[0137] Those of skill in the art would understand that information and signals can be represented using any of a variety of technologies and techniques. For the purposes of this disclosure, the terms "information" and "signal" can be used interchangeably. The terms "coupled," "connected," and "responsive" as well as variations thereof, are intended to encompass an electrical connection, a magnetic connection, a wireless connection, a physical or mechanical connection, or any combination thereof, and do not require direct electrical, magnetic, or physical contact between the connected items. The terms "first", "second", etc. in the claims and specification and drawings of the present application are used for distinguishing between similar objects, not for describing a specific sequential or chronological order. Moreover, the terms "comprising", "including", "containing", and "having" and their variations, are intended to be inclusive, and do not exclude additional non-recited members, steps, or units. For example, a process, method, system, product, or apparatus that comprises a list of steps or units is not necessarily limited to the listed steps or units, but can include additional or other steps or units not expressly listed or inherent to such process, method, system, product, or apparatus. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely examples from a great array of embodiments that can be claimed under this application. In this application, the use of the singular includes the plural, the use of "or" means "and / or", and the use of "one" means "one or more". The use of "including" means "including but not limited to".

[0138] The various functional units in the embodiments of the present application can be integrated in one processing unit, or each functional unit can exist physically, or two or more functional units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional units.

[0139] The above disclosure is merely preferred embodiments of the present application and thus cannot be used to limit the scope of the present application. Any equivalent changes made in the claims and specification of the present application are still within the scope of the present application.

Claims

1. A data processing method, characterized in that, include: Obtain a standard image and an initial face pose model; the standard image carries a standard logistic matrix, a standard prediction box matrix, and a standard global degree of freedom matrix; The standard image is used to generate an initial logistic value matrix, an initial prediction box matrix, and an initial local degree of freedom matrix. Generate a category loss function based on the standard log value matrix and the initial log value matrix; generate a standard box loss function based on the standard prediction box matrix and the initial prediction box matrix. The initial prediction box matrix is ​​filtered to generate an initial filtered prediction box matrix through the label selection module in the initial face pose model, and the standard prediction box matrix is ​​filtered to generate a standard filtered prediction box matrix. Based on the standard filtering prediction box matrix and the initial filtering prediction box matrix, the first prediction box loss value at the first time point is obtained, and the second prediction box loss value at the target time point is obtained; the target time point is the current training time of the initial face pose model, and the first time point is the training time of the previous round of training of the current training time point. Obtain the ratio between the loss value of the second predicted box and the loss value of the first predicted box; If the ratio is greater than the ratio threshold, the confidence state at the target time is not passed, and the standard screening prediction box matrix is ​​determined as the candidate prediction box matrix. If the ratio is less than or equal to the ratio threshold, the confidence state at the target time is passed, and the initial screening prediction box matrix is ​​determined as the candidate prediction box matrix. Generate a pose loss function based on the standard global degree-of-freedom matrix, the candidate prediction box matrix, and the initial local degree-of-freedom matrix; Based on the aforementioned category loss function, standard bounding box loss function, and pose loss function, a face pose model is generated; the face pose model is used to predict the pose information of faces in images.

2. The method according to claim 1, characterized in that, Also includes: The faces in the standard image are labeled to generate the standard logical value matrix, standard prediction box matrix, and standard global degree of freedom matrix of the standard image.

3. The method according to claim 1, characterized in that, The process of generating the initial logistic matrix, initial prediction box matrix, and initial local degrees of freedom matrix of the standard image using the standard image and the initial face pose model includes: The feature extraction module in the initial face pose model extracts features from the standard image to generate the initial logical value matrix, initial prediction box matrix, and initial degree of freedom matrix of the standard image. The initial degree-of-freedom matrix is ​​filtered and processed by the label selection module in the initial face pose model to generate the initial local degree-of-freedom matrix.

4. The method according to claim 1, characterized in that, The generation of the pose loss function based on the standard global degree-of-freedom matrix, the candidate prediction box matrix, and the initial local degree-of-freedom matrix includes: Based on the candidate prediction box matrix, the initial local degree of freedom matrix is ​​converted into the initial global degree of freedom matrix; A pose loss function is generated based on the initial global degree-of-freedom matrix and the standard global degree-of-freedom matrix.

5. The method according to claim 1, characterized in that, The process of generating a face pose model based on the category loss function, the standard bounding box loss function, and the pose loss function includes: The parameters of the initial face pose model are adjusted according to the category loss function, the standard bounding box loss function, and the pose loss function. When the parameters of the initial face pose model after parameter adjustment meet the convergence condition, the initial face pose model after parameter adjustment is determined as a face pose model.

6. A data processing apparatus, characterized in that, include: The data acquisition unit is used to acquire standard images and an initial face pose model; the standard images carry a standard logistic matrix, a standard prediction box matrix, and a standard global degree of freedom matrix. The initial matrix generation unit is used to generate the initial logical value matrix, the initial prediction box matrix, and the initial local degree of freedom matrix of the standard image based on the standard image and the initial face pose model. The first loss function generation unit is used to generate a category loss function based on the standard log value matrix and the initial log value matrix, and to generate a standard box loss function based on the standard prediction box matrix and the initial prediction box matrix. The candidate box generation unit is used to filter the initial prediction box matrix to generate an initial filtered prediction box matrix and filter the standard prediction box matrix to generate a standard filtered prediction box matrix through the label selection module in the initial face pose model. The candidate box generation unit is further configured to obtain a first prediction box loss value at a first time step and a second prediction box loss value at a target time step based on the standard filtering prediction box matrix and the initial filtering prediction box matrix; the target time step is the current training time of the initial face pose model, and the first time step is the training time of the previous round of training of the current training time. The candidate box generation unit is further configured to obtain the ratio between the second predicted box loss value and the first predicted box loss value; The candidate box generation unit is further configured to determine the standard screening prediction box matrix as a candidate prediction box matrix if the ratio is greater than the ratio threshold and the confidence state at the target time is not passed. The candidate box generation unit is further configured to determine the initial screening prediction box matrix as the candidate prediction box matrix if the ratio is less than or equal to the ratio threshold and the confidence state at the target time is passed. The second loss function generation unit is used to generate the attitude loss function based on the standard global degree of freedom matrix, the candidate prediction box matrix and the initial local degree of freedom matrix; The model generation unit is used to generate a face pose model based on the category loss function, the standard bounding box loss function, and the pose loss function; the face pose model is used to predict the pose information of a face in an image.

7. The apparatus according to claim 6, characterized in that, Also includes: The standard matrix generation unit is used to mark the faces in the standard image to generate the standard logical value matrix, standard prediction box matrix, and standard global degree of freedom matrix of the standard image.

8. The apparatus according to claim 6, characterized in that, The initial matrix generation unit is specifically used for: The feature extraction module in the initial face pose model extracts features from the standard image to generate an initial logical value matrix, an initial prediction box matrix, and an initial degree of freedom matrix. The label selection module in the initial face pose model then filters the initial degree of freedom matrix to generate an initial local degree of freedom matrix.

9. The apparatus according to claim 6, characterized in that, The second loss function generation unit is specifically used for: Based on the candidate prediction box matrix, the initial local degree of freedom matrix is ​​converted into the initial global degree of freedom matrix; A pose loss function is generated based on the initial global degree-of-freedom matrix and the standard global degree-of-freedom matrix.

10. The apparatus according to claim 6, characterized in that, The model generation unit is specifically used for: The parameters of the initial face pose model are adjusted according to the category loss function, the standard bounding box loss function, and the pose loss function. When the parameters of the initial face pose model after parameter adjustment meet the convergence condition, the initial face pose model after parameter adjustment is determined as a face pose model.

11. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the method of any one of claims 1-5.

Images