Method, apparatus, device and storage medium for generating three-dimensional dynamic portrait

Abstract

The application discloses a method, device, equipment and storage medium for generating a three-dimensional dynamic portrait, and belongs to the technical field of visual identification. The method comprises the following steps: receiving two-dimensional position coordinates of facial feature points and body part feature points, and two-dimensional-to-three-dimensional conversion coefficients of a face image and a body image; determining three-dimensional position coordinates of facial reference points based on the two-dimensional-to-three-dimensional conversion coefficients of the face image and the two-dimensional position coordinates of the facial feature points; determining three-dimensional position coordinates of body part reference points based on the two-dimensional-to-three-dimensional conversion coefficients of the body image and the two-dimensional position coordinates of the body part feature points; determining a face contour curve in a three-dimensional animation model based on the three-dimensional position coordinates of the facial reference points; determining a body part contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the body part reference points; and displaying a three-dimensional dynamic portrait of a call character based on the face contour curve and the body part contour curve, thereby improving the interactivity of in-vehicle voice calls.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of visual recognition, and particularly relate to a method, device and equipment for generating a three-dimensional dynamic portrait and a storage medium. BACKGROUND

[0002] Currently, a vehicle telephone voice assistant generally only transmits voice, so the interactivity of in-vehicle and out-of-vehicle calls is poor. If a real-time video is directly connected, the requirements for the vehicle-mounted device are relatively high, and a large amount of information flow is required, so the cost is relatively high, and the large amount of information flow may affect other more important communications. SUMMARY

[0003] The present application provides a method, device and equipment for generating a three-dimensional dynamic portrait and a storage medium, which can be used to improve the interactive effect of in-vehicle voice calls. The technical solution is as follows:

[0004] In one aspect, the present application provides a method for generating a three-dimensional dynamic portrait, the method comprising:

[0005] receiving two-dimensional position coordinates of a first number of facial feature points, two-dimensional position coordinates of a second number of body part feature points, a two-dimensional to three-dimensional conversion coefficient of a face image and a two-dimensional to three-dimensional conversion coefficient of a body image, the two-dimensional position coordinates of the facial feature points being obtained by performing facial recognition on a face image of a call person by an out-of-vehicle terminal, the two-dimensional position coordinates of the body feature points being obtained by performing body part recognition on a body image of the call person by the out-of-vehicle terminal, the face image and the body image being obtained by a camera installed on the out-of-vehicle terminal;

[0006] determining three-dimensional position coordinates of facial reference points in a three-dimensional animation model based on the two-dimensional to three-dimensional conversion coefficient of the face image and the two-dimensional position coordinates of the first number of facial feature points, and determining three-dimensional position coordinates of body part reference points in the three-dimensional animation model based on the two-dimensional to three-dimensional conversion coefficient of the body image and the two-dimensional position coordinates of the second number of body part feature points;

[0007] determining a face contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the facial reference points, and determining a body part contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the body part reference points;

[0008] displaying a three-dimensional dynamic portrait of the call person based on the face contour curve in the three-dimensional animation model and the body part contour curve in the three-dimensional animation model.

[0009] In another aspect, a device for generating a three-dimensional dynamic portrait is provided, the device comprising:

[0010] receive a first quantity of two-dimensional position coordinates of facial feature points, a second quantity of two-dimensional position coordinates of body part feature points, a two-dimensional-to-three-dimensional conversion coefficient of a face image, and a two-dimensional-to-three-dimensional conversion coefficient of a body image, the two-dimensional position coordinates of the facial feature points being obtained by performing face recognition on the face image of a talking person by an off-vehicle terminal, the two-dimensional position coordinates of the body part feature points being obtained by performing body part recognition on a body image of the talking person by the off-vehicle terminal, the face image and the body image being obtained by a camera installed on the off-vehicle terminal;

[0011] a first determination module configured to determine three-dimensional position coordinates of a facial reference point in a three-dimensional animation model based on the two-dimensional-to-three-dimensional conversion coefficient of the face image and the first quantity of two-dimensional position coordinates of facial feature points, and determine three-dimensional position coordinates of a body part reference point in the three-dimensional animation model based on the two-dimensional-to-three-dimensional conversion coefficient of the body image and the second quantity of two-dimensional position coordinates of body part feature points;

[0012] a second determination module configured to determine a face contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the facial reference point, and determine a body part contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the body part reference point;

[0013] a display module configured to display a three-dimensional dynamic portrait of the talking person based on the face contour curve in the three-dimensional animation model and the body part contour curve in the three-dimensional animation model.

[0014] In another aspect, a computer device is provided, which includes a processor and a memory, and the memory stores at least one computer program, which is loaded and executed by the processor, so that the computer device implements the method for generating a three-dimensional dynamic portrait according to any one of the above aspects.

[0015] In another aspect, a computer readable storage medium is also provided, which stores at least one computer program, which is loaded and executed by a processor, so that a computer implements the method for generating a three-dimensional dynamic portrait according to any one of the above aspects.

[0016] In another aspect, a computer program product or a computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for generating a three-dimensional dynamic portrait according to any one of the above aspects.

[0017] The technical scheme provided in the application brings at least the following beneficial effects:

[0018] The application determines the three-dimensional position coordinates of the face reference points and the body part reference points in the three-dimensional animation model by receiving the two-dimensional position coordinates of the face feature points, the two-dimensional position coordinates of the body part feature points, the two-dimensional-to-three-dimensional conversion coefficient of the face image and the two-dimensional-to-three-dimensional conversion coefficient of the body image from the vehicle exterior camera terminal, and further determines the face contour curve and the body part contour curve in the three-dimensional animation model, so as to generate and display the three-dimensional dynamic portrait of the call person, and reflect the face state and the body state of the call person in real time, thereby improving the interactive effect of the in-vehicle and out-vehicle voice call. Moreover, since only the two-dimensional position coordinates of the face feature points, the two-dimensional position coordinates of the body part feature points and the corresponding two-dimensional-to-three-dimensional conversion coefficient need to be transmitted, the real-time performance of the transmission is improved and the requirement on the transmission equipment is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical scheme in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0020] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the application;

[0021] Figure 2 is a method flowchart for generating a three-dimensional dynamic portrait provided by an embodiment of the application;

[0022] Figure 3 is a process diagram for obtaining the two-dimensional position coordinates of the face feature points and the two-dimensional position coordinates of the body part feature points provided by an embodiment of the application;

[0023] Figure 4 is a method flowchart for obtaining the two-dimensional-to-three-dimensional conversion coefficient of the face image provided by an embodiment of the application;

[0024] Figure 5 is a method flowchart for obtaining the two-dimensional-to-three-dimensional conversion coefficient of the body image provided by an embodiment of the application;

[0025] Figure 6 is a process description diagram for vehicle end display of the three-dimensional dynamic portrait provided by an embodiment of the application;

[0026] Figure 7 is a system framework diagram for generating a three-dimensional dynamic portrait provided by an embodiment of the application;

[0027] Figure 8 is a structural schematic diagram of a device for generating a three-dimensional dynamic portrait provided by an embodiment of the present application;

[0028] Figure 9 is a structural schematic diagram of a device for generating a three-dimensional dynamic portrait provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0030] The present application provides a method for generating a three-dimensional dynamic portrait, please refer to Figure 1 which shows a schematic diagram of a method implementation environment provided by an embodiment of the present application. The implementation environment can include: a vehicle 11, a vehicle central control system 12 and an off-vehicle terminal 13.

[0031] Optionally, the vehicle central control system 12 receives the two-dimensional position coordinates of the first number of facial feature points, the two-dimensional position coordinates of the second number of body part feature points, the two-dimensional-to-three-dimensional conversion coefficient of the face image and the two-dimensional-to-three-dimensional conversion coefficient of the body image sent by the off-vehicle terminal 13. The two-dimensional position coordinates of the facial feature points are obtained by the off-vehicle terminal 13 performing facial recognition on the face image of the talking person. The two-dimensional position coordinates of the body feature points are obtained by the off-vehicle terminal 13 performing body part recognition on the body image of the talking person. The face image and the body image are obtained by the camera installed on the off-vehicle terminal 13.

[0032] The vehicle central control system 12 determines the three-dimensional position coordinates of the facial reference points in the three-dimensional animation model based on the two-dimensional-to-three-dimensional conversion coefficient of the face image and the two-dimensional position coordinates of the first number of facial feature points. The vehicle central control system 12 determines the three-dimensional position coordinates of the body part reference points in the three-dimensional animation model based on the two-dimensional-to-three-dimensional conversion coefficient of the body image and the two-dimensional position coordinates of the second number of body part feature points. Based on the three-dimensional position coordinates of the facial reference points, the vehicle central control system 12 determines the face contour curve in the three-dimensional animation model. Based on the three-dimensional position coordinates of the body part reference points, the vehicle central control system 12 determines the body part contour curve in the three-dimensional animation model. Based on the face contour curve in the three-dimensional animation model and the body part contour curve in the three-dimensional animation model, the vehicle 11 displays the three-dimensional dynamic portrait of the talking person.

[0033] The vehicle central control system 12 can store the two-dimensional position coordinates of the facial feature points, the two-dimensional position coordinates of the body part feature points, the two-dimensional to three-dimensional conversion coefficients of the face image, and the two-dimensional to three-dimensional conversion coefficients of the body image transmitted by the terminal 13 outside the vehicle, and display a three-dimensional dynamic portrait of the caller on the vehicle 11. For example, the vehicle 11 and the vehicle central control system 12 are connected by a wired or wireless network, and the vehicle central control system 12 obtains the two-dimensional position coordinates of the facial feature points, the two-dimensional position coordinates of the body part feature points, the two-dimensional to three-dimensional conversion coefficients of the face image, and the two-dimensional to three-dimensional conversion coefficients of the body image from the terminal 13 outside the vehicle through the wired or wireless network.

[0034] Based on the above Figure 1 The implementation environment is shown, and the embodiment of the application provides a method for generating a three-dimensional dynamic portrait, as shown in the method Figure 2 The method is applied to the vehicle central control system, for example, and the method includes steps 201-204.

[0035] In step 201, the two-dimensional position coordinates of the first number of facial feature points, the two-dimensional position coordinates of the second number of body part feature points, the two-dimensional to three-dimensional conversion coefficients of the face image, and the two-dimensional to three-dimensional conversion coefficients of the body image are received. The two-dimensional position coordinates of the facial feature points are obtained by face recognition of the face image of the caller through a terminal such as a mobile phone outside the vehicle, the two-dimensional position coordinates of the body feature points are obtained by body part recognition of the body image of the caller through the mobile phone, and the face image and the body image are obtained by a camera installed on the mobile phone.

[0036] For example, when the person inside the vehicle and the person outside the vehicle make a voice call through a mobile phone, the face image and the body image are obtained by a camera installed on the mobile phone, including: obtaining the face image and the body image of the caller inside the vehicle through the camera installed on the mobile phone of the caller inside the vehicle; obtaining the face image and the body image of the caller outside the vehicle through the camera installed on the mobile phone of the caller outside the vehicle. The camera installed on the mobile phone includes a front camera and a rear camera.

[0037] In a possible implementation, after obtaining the face image and the body image of the caller, the face image is recognized by the mobile phone to obtain the first number of facial feature points, and the body image is recognized by the mobile phone to obtain the second number of body part feature points. Optionally, the face image is recognized by the mobile phone, that is, the face image is recognized by a facial feature point recognition algorithm; the body image is recognized by the mobile phone, that is, the body image is recognized by a body feature point recognition algorithm.

[0038] The embodiments of the present application do not limit the face feature point recognition algorithm, and the face feature point recognition algorithm is constructed in an exemplary manner, including but not limited to: obtaining a face training image and a face reference image corresponding to the face training image, the face reference image including a first number of pre-labeled face feature points; importing the face training image into an initial face recognition model to obtain the first number of face feature points; determining a loss value based on the pre-labeled face feature points on the face reference image and the face feature points on the face training image, adjusting the initial face recognition model based on the loss value to obtain a target face recognition model, and the algorithm of the target face recognition model can perform face recognition processing on the face image.

[0039] The embodiments of the present application do not limit the first number, and the first number can be set and adjusted according to actual needs. The present application also does not limit the initial face recognition model, and the initial face recognition model can be constructed based on the pre-labeled face feature points in the face reference image, or the initial face recognition model can be adjusted according to actual needs.

[0040] The face feature points in the obtained face training image are compared with the positions of the face feature points in the face reference image to determine the loss value generated by the initial face recognition model, and the initial face recognition model is adjusted based on the loss value, so that the positions of the face feature points in the face training image obtained after the face recognition model is processed next time are more coincident with the positions of the face feature points in the face reference image, until a preset number of times of adjusting the initial face recognition model is reached, and the face recognition model when the preset number of times of adjusting the initial face recognition model is reached is taken as the target face recognition model.

[0041] The embodiments of the present application do not limit the preset number of times of adjusting the face recognition model, which can be set based on experience or adjusted according to actual conditions.

[0042] In one possible implementation, after the algorithm corresponding to the target face recognition model is determined, the face image is subjected to face recognition by a mobile phone to obtain the first number of face feature points, including: importing the obtained face image into the target face recognition model, and performing face recognition processing on the face image by the target face recognition model to obtain the first number of face feature points in the face image.

[0043] The embodiments of the present application do not limit the body feature point recognition algorithm, and the manner of constructing the body feature point recognition algorithm includes but is not limited to: obtaining a body training image and a body reference image corresponding to the body training image, the body reference image including a second number of pre-labeled body part feature points; importing the face training image into an initial face recognition model to obtain the second number of body part feature points; determining a loss value based on the pre-labeled body feature points on the body reference image and the body feature points on the body training image, adjusting the initial body recognition model based on the loss value to obtain a target body recognition model, and the algorithm of the target body recognition model can perform body part recognition processing on the body image.

[0044] The embodiments of the present application do not limit the second number, and the second number can be set and adjusted according to actual needs. The present application also does not limit the initial body recognition model, and the initial body recognition model can be constructed based on the pre-labeled body part feature points in the body reference image, or the initial body recognition model can be adjusted according to actual needs.

[0045] In a possible implementation, after the initial body recognition model is constructed, the second number of body part feature points on the body training image is determined based on the initial body recognition model, including: importing the body training image into the initial body recognition model, and obtaining the second number of body part feature points in the body training image after the body training image is processed by the initial body recognition model for body part recognition.

[0046] The obtained body part feature points in the body training image are compared with the positions of the body part feature points in the body reference image to determine the loss value generated by the initial body recognition model, and the initial body recognition model is adjusted based on the loss value, so that the positions of the body part feature points in the body training image obtained after the next processing by the body recognition model tend to coincide with the positions of the body part feature points in the body reference image, until a preset number of times of adjusting the initial body recognition model is reached, and the body recognition model at the time when the preset number of times of adjusting the initial body recognition model is reached is taken as the target body recognition model.

[0047] The embodiments of the present application do not limit the number of times of adjusting the body recognition model, which can be set based on experience or adjusted according to the coincidence degree of the positions of the body part feature points in the body training image and the body reference image. After adjustment, the target body recognition model is finally obtained.

[0048] Optionally, after determining the algorithm corresponding to the target body recognition model, the body part recognition of the body image is performed through the mobile phone to obtain the second number of body part feature points, including: importing the obtained body image into the target body recognition model, performing body part recognition processing on the body image through the target body recognition model, and obtaining the second number of body part feature points in the body image.

[0049] Exemplarily, after obtaining the first number of face feature points and the second number of body part feature points, the two-dimensional position coordinates of the face feature points are determined based on the face feature points, and the two-dimensional position coordinates of the body part feature points are determined based on the body part feature points. The selection of the origin is not limited by the embodiments of the present application, and exemplarily, the center point of the two-dimensional image can be selected as the coordinate origin of the two-dimensional image, or the coordinate origin can be adjusted according to the actual situation. Optionally, the process of obtaining the two-dimensional position coordinates of the face feature points and the body part feature points is as shown in Figure 3 Exemplarily, after obtaining the first number of face feature points and the second number of body part feature points, the two-dimensional position coordinates of the face feature points are determined based on the face feature points, and the two-dimensional position coordinates of the body part feature points are determined based on the body part feature points. The selection of the origin is not limited by the embodiments of the present application, and exemplarily, the center point of the two-dimensional image can be selected as the coordinate origin of the two-dimensional image, or the coordinate origin can be adjusted according to the actual situation. Optionally, the process of obtaining the two-dimensional position coordinates of the face feature points and the body part feature points is as shown in

[0050] In one possible implementation, after the mobile phone obtains the two-dimensional position coordinates of the first number of face feature points and the two-dimensional position coordinates of the second number of body part feature points, at least one of wireless network or Bluetooth transmission is used to transmit the two-dimensional position coordinates of the first number of face feature points and the two-dimensional position coordinates of the second number of body part feature points to the vehicle. The vehicle receives the two-dimensional position coordinates of the first number of face feature points and the two-dimensional position coordinates of the second number of body part feature points, which are used for subsequent control of the three-dimensional animation model by the vehicle.

[0051] The face image and the body image of the calling person are obtained through the mobile phone, and after face recognition and body part recognition are performed on the face image and the body image, the two-dimensional position coordinates of the face feature points and the two-dimensional position coordinates of the body part feature points are transmitted to the vehicle. The vehicle only receives the two-dimensional position coordinates of the face feature points and the two-dimensional position coordinates of the body part feature points obtained by processing, instead of the face image and the body image, which speeds up the data transmission and reduces the requirements on the transmission equipment.

[0052] In step 202, the three-dimensional position coordinates of the face reference points in the three-dimensional animation model are determined based on the two-dimensional to three-dimensional conversion coefficients of the face image and the two-dimensional position coordinates of the first number of face feature points, and the three-dimensional position coordinates of the body part reference points in the three-dimensional animation model are determined based on the two-dimensional to three-dimensional conversion coefficients of the body image and the two-dimensional position coordinates of the second number of body part feature points.

[0053] In a possible implementation, the manner of determining the two-dimensional-to-three-dimensional conversion coefficient of the face image and the two-dimensional-to-three-dimensional conversion coefficient of the body image includes but is not limited to: determining the two-dimensional-to-three-dimensional conversion coefficient of the face image based on the internal parameters of the camera, the two-dimensional position coordinates of the facial feature points, and the three-dimensional position coordinates of the labeled facial reference points in the three-dimensional animation model; and determining the two-dimensional-to-three-dimensional conversion coefficient of the body image based on the internal parameters of the camera, the two-dimensional position coordinates of the body part feature points, and the three-dimensional position coordinates of the labeled body part reference points in the three-dimensional animation model.

[0054] Exemplarily, the internal parameters of the camera include: the focal length of the camera in the x-axis direction, the focal length of the camera in the y-axis direction, the horizontal coordinate of the principal point of the camera projected on the face image, and the vertical coordinate of the principal point of the camera projected on the face image. In a possible implementation, the internal parameters of the camera can be determined by a camera calibration method, and the process of camera calibration includes: collecting at least a reference number of calibration images at different angles, and the images need to contain objects of different depths. The corner points in each calibration image are extracted by using a function of OpenCV (Open Source Computer Vision Library). The function of extracting the corner points in the calibration image is not limited in the embodiment of the application. For example, if the calibration board used for calibration is a checkerboard calibration board, the cv2.findChessboardCorners function can be used to extract the corner points. After the corner points are extracted, the internal parameters of the camera are calculated based on the extracted corner points by using a function of OpenCV, for example, the cv2.calibrateCamera function can be used to calculate the internal parameters of the camera.

[0055] Optionally, the embodiment of the application can select the key points of the face in the three-dimensional model as the labeled facial reference points, and select the key points of the body part in the three-dimensional animation model as the labeled body part reference points. The selection of the origin is not limited in the embodiment of the application, and the center point of the three-dimensional animation model can be selected as the coordinate origin, or the coordinate origin can be adjusted according to the actual situation. After the origin of the three-dimensional animation model is determined, the three-dimensional position coordinates of the facial reference points can be determined based on the positions of the facial reference points, and the three-dimensional position coordinates of the body part reference points can be determined based on the positions of the body part reference points.

[0056] In a possible implementation, after obtaining the internal parameters of the camera, the two-dimensional position coordinates of the facial feature points, and the three-dimensional position coordinates of the labeled facial reference points in the three-dimensional animation model, the two-dimensional-to-three-dimensional conversion coefficient of the face image is determined based on the internal parameters of the camera, the two-dimensional position coordinates of the facial feature points, and the three-dimensional position coordinates of the labeled facial reference points in the three-dimensional animation model, including: bringing the internal parameters of the camera, the two-dimensional position coordinates of the facial feature points, and the three-dimensional position coordinates of the labeled facial reference points into a two-dimensional-to-three-dimensional conversion formula to obtain the two-dimensional-to-three-dimensional conversion coefficient of the face image; wherein the two-dimensional-to-three-dimensional conversion formula includes:

[0057]

[0058] is the horizontal coordinate of the facial feature point, is the vertical coordinate of the facial feature point, s is a scaling parameter, is the focal length of the camera in the x-axis direction, is the focal length of the camera in the y-axis direction, is the horizontal coordinate of the principal point of the camera on the face image, is the vertical coordinate of the principal point of the camera on the face image, is the horizontal coordinate of the facial reference point, is the vertical coordinate of the facial reference point, is the height coordinate of the facial reference point, is the two-dimensional-to-three-dimensional conversion coefficient of the face image.

[0059] Exemplarily, the inverse matrix of the internal parameters of the camera is multiplied on the left of the equation, the inverse matrix of the three-dimensional position coordinates of the facial reference points is multiplied on the right, and s is divided to obtain the following equation:

[0060] 1 / s

[0061] Optionally, in the case where the camera does not have magnification or reduction, the scaling parameter s is set to 1. Exemplarily, the obtained horizontal coordinates of the facial feature points are brought into , the obtained vertical coordinates of the facial feature points are brought into , the horizontal coordinates of the labeled facial reference points are brought into , the vertical coordinates of the labeled facial reference points are brought into , the height coordinates of the labeled facial reference points are brought into , the focal length of the camera in the x-axis direction is brought into , the focal length of the camera in the y-axis direction is brought into , the horizontal coordinate of the principal point of the camera on the face image is brought into , and the vertical coordinate of the principal point of the camera on the face image is brought into , the conversion coefficient of the two-dimensional to three-dimensional of the face image is calculated .

[0062] In a possible implementation, the inverse matrix of the internal parameter of the camera and the inverse matrix of the three-dimensional position coordinates of the face reference points can be solved by a pending coefficient method, an adjoint matrix method or an elementary transformation method.

[0063] Exemplarily, a method flow of acquiring the conversion coefficient of the two-dimensional to three-dimensional of the face image is shown in Figure 4 FIG. 4, wherein the internal parameter of the camera is determined by camera calibration using the camera 401 installed on the mobile phone 402, and then the conversion coefficient of the two-dimensional to three-dimensional of the face image is obtained by the internal parameter of the camera 402, the two-dimensional position coordinates of the face feature points 403 and the three-dimensional position coordinates of the face reference points marked in the three-dimensional animation model 404.

[0064] In a possible implementation, after the conversion coefficient of the two-dimensional to three-dimensional of the face image is determined, the three-dimensional position coordinates of the face reference points in the three-dimensional animation model are determined based on the conversion coefficient of the two-dimensional to three-dimensional of the face image and the two-dimensional position coordinates of the first number of face feature points, including: the conversion coefficient of the two-dimensional to three-dimensional of the face image, the two-dimensional position coordinates of the face feature points and the internal parameter of the camera are brought into the conversion formula of the two-dimensional to three-dimensional, and the three-dimensional position coordinates of the face reference points in the three-dimensional animation model are calculated.

[0065] In a possible implementation, after the internal parameter of the camera, the two-dimensional position coordinates of the body part feature points and the three-dimensional position coordinates of the face reference points marked in the three-dimensional animation model are acquired, the conversion coefficient of the two-dimensional to three-dimensional of the face image is determined based on the internal parameter of the camera, the two-dimensional position coordinates of the face feature points and the three-dimensional position coordinates of the body part reference points marked in the three-dimensional animation model, including: the internal parameter of the camera, the two-dimensional position coordinates of the body part feature points and the three-dimensional position coordinates of the body part reference points marked are brought into the conversion formula of the two-dimensional to three-dimensional, and the conversion coefficient of the two-dimensional to three-dimensional of the body image is obtained;

[0066] The conversion formula of the two-dimensional to three-dimensional includes:

[0067]

[0068] is the horizontal coordinate of the body part feature point, is the vertical coordinate of the body part feature point, and s is a scaling parameter, is the focal length of the camera in the x-axis direction, is the focal length of the camera in the y-axis direction, is the horizontal coordinate of the principal point of the camera on the body image, is the vertical coordinate of the principal point of the camera on the body image, a horizontal coordinate of a reference point of a body part, a vertical coordinate of a reference point of a body part, an altitude coordinate of a reference point of a body part, a conversion coefficient from two dimensions to three dimensions of a body image.

[0069] Exemplarily, multiplying the inverse matrix of the internal parameters of the camera on the left and multiplying the inverse matrix of the three-dimensional position coordinates of the reference points of the body part on the right of both sides of the equation, the following equation is obtained:

[0070] 1 / s

[0071] In a possible implementation, the scaling parameter s is set to 1 when the camera does not have magnification or reduction. Exemplarily, the horizontal coordinate of the acquired feature point of the body part is brought into , the vertical coordinate of the acquired feature point of the body part is brought into . The horizontal coordinate of the labeled reference point of the body part is brought into , the vertical coordinate of the labeled reference point of the body part is brought into , the altitude coordinate of the labeled reference point of the body part is brought into . The focal length of the camera in the x-axis direction is brought into , the focal length of the camera in the y-axis direction is brought into , the horizontal coordinate of the principal point of the camera on the face image is brought into , and the vertical coordinate of the principal point of the camera on the face image is brought into , and the conversion coefficient from two dimensions to three dimensions of the body image is calculated as .

[0072] Optionally, the inverse matrix of the three-dimensional position coordinates of the reference points of the body part can be solved by the method of undetermined coefficients, the method of adjoint matrix or the method of elementary transformation.

[0073] In a possible implementation, the method flow of acquiring the conversion coefficient from two dimensions to three dimensions of the body image is as shown in Figure 5 , wherein the conversion coefficient from two dimensions to three dimensions of the body image 504 is obtained through the internal parameters of the camera 501, the two-dimensional position coordinates of the feature points of the body part 502 and the three-dimensional position coordinates of the labeled reference points of the body part in the three-dimensional animation model 503.

[0074] Exemplarily, after the two-dimensional to three-dimensional conversion coefficients of the body image are determined, the three-dimensional position coordinates of the body part reference points in the three-dimensional animation model are determined based on the two-dimensional to three-dimensional conversion coefficients of the body image and the two-dimensional position coordinates of the first number of body part feature points, including: bringing the two-dimensional to three-dimensional conversion coefficients of the face image, the two-dimensional position coordinates of the body part feature points and the internal parameters of the camera into a two-dimensional to three-dimensional conversion formula to calculate the three-dimensional position coordinates of the body part reference points in the three-dimensional animation model.

[0075] In a possible implementation, after the mobile phone obtains the three-dimensional position coordinates of the face reference points and the three-dimensional position coordinates of the body part reference points, the three-dimensional position coordinates of the face reference points and the three-dimensional position coordinates of the body part reference points are transmitted to the vehicle by at least one of wireless network transmission or Bluetooth transmission, and the vehicle receives the three-dimensional position coordinates of the face reference points and the three-dimensional position coordinates of the body part reference points for subsequent control of the three-dimensional animation model by the vehicle.

[0076] The three-dimensional position coordinates of the face reference points and the three-dimensional position coordinates of the body part reference points are obtained by the mobile phone and transmitted to the vehicle, and the vehicle only receives the three-dimensional position coordinates of the face reference points and the three-dimensional position coordinates of the body part reference points instead of the three-dimensional dynamic portrait, which accelerates the speed of data transmission for displaying the three-dimensional dynamic portrait and reduces the requirements on the transmission equipment.

[0077] In step 203, the face contour curve in the three-dimensional animation model is determined based on the three-dimensional position coordinates of the face reference points, and the body part contour curve in the three-dimensional animation model is determined based on the three-dimensional position coordinates of the body part reference points.

[0078] In a possible implementation, after the three-dimensional position coordinates of the face reference points are determined, the face contour curve in the three-dimensional animation model is determined based on the three-dimensional position coordinates of the face reference points, including: determining the control point coordinates of a first Bezier curve corresponding to the face reference points based on the three-dimensional position coordinates of the face reference points; determining the first Bezier curve based on the control point coordinates of the first Bezier curve, and taking the first Bezier curve as the face contour curve in the three-dimensional animation model. Wherein, the function of the first Bezier curve is as follows, taking four control points as an example:

[0079]

[0080] (t) is the function of the first Bezier curve, t is the independent variable of the function of the first Bezier curve, is the abscissa of the face reference point, is the ordinate of the face reference point, is the height coordinate of the face reference point, is the horizontal coordinate of the control point of the first Bezier curve, is the vertical coordinate of the control point of the first Bezier curve, is the height coordinate of the control point of the first Bezier curve, i is equal to 1, 2, 3 or 4.

[0081] Exemplarily, the horizontal coordinate of the facial reference point is substituted into the vertical coordinate of the facial reference point is substituted into the height coordinate of the facial reference point is substituted into the coordinates of the control point of the first Bezier curve can be calculated, , , , , , and , based on the coordinates of the control point of the first Bezier curve, the first Bezier curve is determined the first Bezier curve is taken as the face contour curve in the three-dimensional animation model.

[0082] In a possible implementation, after the three-dimensional position coordinates of the body part reference points are determined, based on the three-dimensional position coordinates of the body part reference points, a body part contour curve in the three-dimensional animation model is determined, including: determining the three-dimensional position coordinates of the body part reference points as the control point coordinates of a second Bezier curve corresponding to the body part reference points; based on the control point coordinates of the second Bezier curve, determining the second Bezier curve, and taking the second Bezier curve as the body part contour curve in the three-dimensional animation model.

[0083] wherein, the function of the second Bezier curve is as follows, taking four control points as an example:

[0084]

[0085] (t) is the function of the second Bezier curve, t is the independent variable of the function of the second Bezier curve, is the horizontal coordinate of the body part reference point, is the vertical coordinate of the body part reference point, is the height coordinate of the body part reference point, is the horizontal coordinate of the control point of the second Bezier curve, is the vertical coordinate of the control point of the second Bezier curve, is the height coordinate of the control point of the second Bezier curve, wherein j is equal to 5, 6, 7 or 8.

[0086] Exemplarily, the horizontal coordinate of the body part reference point is substituted into Substitute the ordinate of the body part reference point into Substitute the height coordinates of the body part reference points into The coordinates of the control points of the second Bézier curve can be calculated. , ), ( , ), ( , )and( , The second Bézier curve is determined based on the coordinates of its control points. The second Bézier curve is used as the contour curve of the body parts in the 3D animation model.

[0087] In step 204, a three-dimensional dynamic portrait of the person in the call is displayed based on the facial contour curve and body part contour curve in the three-dimensional animation model.

[0088] In one possible implementation, after determining the facial contour curve and the body part contour curve in the 3D animation model, the 3D dynamic portrait of the person in the call is determined based on the facial contour curve and the body part contour curve in the 3D animation model, including: combining the facial contour curve and the body part contour curve in the 3D animation model according to the 3D animation model to obtain the 3D dynamic portrait of the person in the call.

[0089] After obtaining the 3D dynamic image of the person in the call, the 3D dynamic image is displayed on a display screen installed on the vehicle. This application embodiment does not limit the location of the display screen on the vehicle; for example, it can be installed in the location of the vehicle's instrument panel display system.

[0090] Combining the above methods and processes, with Figure 6 The following is an example illustration of a process for generating a three-dimensional dynamic human image for display on a vehicle, provided by an embodiment of this application. Specifically, by using the two-dimensional position coordinates 601 of facial and body feature points and the two-dimensional-to-three-dimensional conversion coefficients 602 of the face and body images, a 3D (three-dimensional) pose is reconstructed, and a three-dimensional dynamic human image 603 of the person making the call is displayed on the vehicle terminal.

[0091] By installing a display screen in the vehicle's instrument panel, a three-dimensional dynamic image of the person making the call can be displayed, reflecting their facial and physical state in real time, thus improving the interactivity of voice calls inside and outside the vehicle.

[0092] This embodiment of the application determines the 3D coordinates of facial and body reference points in a 3D animation model by receiving the 2D coordinates of facial feature points and body feature points from an external vehicle camera terminal, along with the 2D-to-3D conversion coefficients of the face and body images. This allows for the determination of the facial and body contour curves in the 3D animation model, thereby generating and displaying a 3D dynamic portrait of the person making the call. This reflects the facial and body states of the person in real time, improving the interactive effect of voice calls both inside and outside the vehicle. Furthermore, because only the 2D coordinates of facial and body feature points and the corresponding 2D-to-3D conversion coefficients need to be transmitted, the real-time performance of the transmission is improved, and the requirements for the transmission equipment are reduced.

[0093] Combining the above methods and processes, with Figure 7 The system framework diagram for generating a three-dimensional dynamic human image provided in this application embodiment is illustrated below. The system uses a camera 701 installed on a mobile phone to perform facial recognition on the face image of the person making the call, and body part recognition on the body image of the person making the call, obtaining two-dimensional position coordinates 702 of facial feature points and body part feature points, and two-dimensional to three-dimensional conversion coefficients 703. The remote terminal sends the facial feature points, body part feature points 702, and two-dimensional to three-dimensional conversion coefficients 703 to the in-vehicle communication system 704 on the vehicle side through data fusion, and the three-dimensional dynamic human image of the person making the call is displayed in the vehicle 705.

[0094] See Figure 8 This application provides an apparatus for generating a three-dimensional dynamic human image, the apparatus comprising:

[0095] The receiving module 801 is used to receive the two-dimensional position coordinates of a first number of facial feature points, the two-dimensional position coordinates of a second number of body part feature points, the two-dimensional to three-dimensional conversion coefficients of the face image and the two-dimensional to three-dimensional conversion coefficients of the body image. The two-dimensional position coordinates of the facial feature points are obtained by facial recognition of the face image of the person making the call through an external terminal. The two-dimensional position coordinates of the body feature points are obtained by body part recognition of the body image of the person making the call through a mobile phone. The face image and body image are obtained by a camera installed on the mobile phone.

[0096] The first determining module 802 is used to determine the three-dimensional position coordinates of facial reference points in a three-dimensional animation model based on the two-dimensional to three-dimensional conversion coefficients of the face image and the two-dimensional position coordinates of a first number of facial feature points, and to determine the three-dimensional position coordinates of body part reference points in a three-dimensional animation model based on the two-dimensional to three-dimensional conversion coefficients of the body image and the two-dimensional position coordinates of a second number of body part feature points.

[0097] The second determining module 803 is configured to determine a face contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the face reference points, and determine a body part contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the body part reference points.

[0098] The display module 804 is configured to display a three-dimensional dynamic portrait of the call character based on the face contour curve in the three-dimensional animation model and the body part contour curve in the three-dimensional animation model.

[0099] In a possible implementation, the apparatus further includes: a third determining module configured to determine a two-dimensional-to-three-dimensional conversion coefficient of the face image based on the internal parameters of the camera, the two-dimensional position coordinates of the face feature points, and the three-dimensional position coordinates of the labeled face reference points in the three-dimensional animation model; and a fourth determining module configured to determine a two-dimensional-to-three-dimensional conversion coefficient of the body image based on the internal parameters of the camera, the two-dimensional position coordinates of the body part feature points, and the three-dimensional position coordinates of the labeled body part reference points in the three-dimensional animation model.

[0100] In a possible implementation, the third determining module is configured to input the internal parameters of the camera, the two-dimensional position coordinates of the face feature points, and the three-dimensional position coordinates of the labeled face reference points into a two-dimensional-to-three-dimensional conversion formula to obtain the two-dimensional-to-three-dimensional conversion coefficient of the face image; and the two-dimensional-to-three-dimensional conversion formula includes:

[0101]

[0102] wherein x is a horizontal coordinate of the face feature point, wherein y is a vertical coordinate of the face feature point, and s is a scaling parameter, wherein f x is a focal length of the camera in the x-axis direction, wherein f y is a focal length of the camera in the y-axis direction, wherein x 0 is a horizontal coordinate of a principal point of the camera on the face image, wherein y 0 is a vertical coordinate of the principal point of the camera on the face image, wherein x r is a horizontal coordinate of the face reference point, wherein y r is a vertical coordinate of the face reference point, wherein h r is a height coordinate of the face reference point, wherein the two-dimensional-to-three-dimensional conversion coefficient of the face image is obtained.

[0103] In a possible implementation, the fourth determining module is configured to input the internal parameters of the camera, the two-dimensional position coordinates of the body part feature points, and the three-dimensional position coordinates of the labeled body part reference points into a two-dimensional-to-three-dimensional conversion formula to obtain the two-dimensional-to-three-dimensional conversion coefficient of the body image; and the two-dimensional-to-three-dimensional conversion formula includes:

[0104]

[0105] is a horizontal coordinate of a feature point of a body part, is a vertical coordinate of a feature point of a body part, s is a scaling parameter, is a focal length of a camera in an x-axis direction, is a focal length of a camera in a y-axis direction, is a horizontal coordinate of a principal point of a camera on a body image, is a vertical coordinate of a principal point of a camera on a body image, is a horizontal coordinate of a reference point of a body part, is a vertical coordinate of a reference point of a body part, is a height coordinate of a reference point of a body part, is a two-dimensional to three-dimensional conversion coefficient of a body image.

[0106] In a possible implementation, the second determining module 803 is configured to determine, based on the three-dimensional position coordinates of the face reference points, control point coordinates of a first Bezier curve corresponding to the face reference points; determine the first Bezier curve based on the control point coordinates of the first Bezier curve, and take the first Bezier curve as a face contour curve in the three-dimensional animation model.

[0107] In a possible implementation, the second determining module 803 is configured to determine, based on the three-dimensional position coordinates of the body part reference points, control point coordinates of a second Bezier curve corresponding to the body part reference points; determine the second Bezier curve based on the control point coordinates of the second Bezier curve, and take the second Bezier curve as a body part contour curve in the three-dimensional animation model.

[0108] The device determines the three-dimensional position coordinates of the face reference points and the body part reference points in the three-dimensional animation model based on the two-dimensional position coordinates of the face feature points, the two-dimensional position coordinates of the body part feature points, the two-dimensional to three-dimensional conversion coefficient of the face image, and the two-dimensional to three-dimensional conversion coefficient of the body image received from the vehicle external camera terminal, and further determines the face contour curve and the body part contour curve in the three-dimensional animation model, so as to generate a three-dimensional dynamic portrait of a call person and display, reflect the face state and the body state of the call person in real time, and improve the interactive effect of the vehicle internal and external voice call. Moreover, only the two-dimensional position coordinates of the face feature points, the two-dimensional position coordinates of the body part feature points, and the corresponding two-dimensional to three-dimensional conversion coefficients need to be transmitted, so that the real-time performance of transmission is improved and the requirement on the transmission equipment is reduced.

[0109] It should be noted that the apparatus provided by the above embodiments is only exemplified by the above division of functional modules when realizing its functions. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0110] Figure 9 is a device structure schematic diagram for generating a three-dimensional dynamic portrait provided by an embodiment of the present application. The device can be a terminal, for example, can be: a vehicle-mounted computing device, a smart phone, a tablet computer, a player, a notebook computer or a desktop computer. The terminal can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, and other names.

[0111] Generally, the terminal includes a processor 901 and a memory 902.

[0112] The processor 901 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 901 can also include a main processor and a coprocessor, the main processor is a processor for processing data in the wake-up state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 901 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0113] The memory 902 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 902 can also include high-speed random access memory and can include non-volatile memory, such as one or more magnetic disk storage devices, optical storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, the non-transitory computer-readable storage medium of the memory 902 is used to store at least one instruction for execution by the processor 901 to enable the terminal to implement the method for diagnosing traffic faults provided by the method embodiments of the present application.

[0114] In some embodiments, the terminal can also optionally include a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902, and the peripheral device interface 903 can be connected by a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 903 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 908.

[0115] The peripheral device interface 903 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 can be implemented on a separate chip or circuit board, and the present embodiment does not limit this.

[0116] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 904 can also include NFC (Near Field Communication) related circuitry, which is not limited by the present application.

[0117] The display screen 905 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 is further configured to capture touch signals on or above the surface of the display screen 905. The touch signals can be input to the processor 901 as control signals for processing. In this case, the display screen 905 can also be configured to provide virtual buttons and / or virtual keyboard, also known as soft buttons and / or soft keyboard. In some embodiments, the display screen 905 can be one, disposed on the front panel of the terminal; in other embodiments, the display screen 905 can be at least two, respectively disposed on different surfaces of the terminal or in a folding design; in other embodiments, the display screen 905 can be a flexible display screen, disposed on a curved surface or a folding surface of the terminal. Even, the display screen 905 can also be disposed in an irregular shape, i.e., a special-shaped screen. The display screen 905 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc.

[0118] The camera assembly 906 is configured to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera assembly 906 can further include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0119] The audio circuit 907 can include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor 901 for processing, or input to the radio frequency circuit 904 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, arranged at different parts of the terminal. The microphone can also be an array microphone or an omnidirectional collection type microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker can be a traditional diaphragm speaker, or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only can it convert electrical signals into sound waves that humans can hear, but it can also convert electrical signals into sound waves that humans cannot hear for ranging purposes. In some embodiments, the audio circuit 907 can also include a headphone jack.

[0120] The power supply 908 is used to supply power to various components in the terminal. The power supply 908 can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply 908 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0121] In some embodiments, the terminal also includes one or more sensors 909. The one or more sensors 909 include but are not limited to: an acceleration sensor 910, a gyroscope sensor 911, a pressure sensor 912, an optical sensor 913, and a proximity sensor 914.

[0122] The acceleration sensor 910 can detect the acceleration in three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 910 can be used to detect the components of gravitational acceleration in three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 910. The acceleration sensor 910 can also be used for game or user motion data collection.

[0123] The gyroscope sensor 911 can detect the body orientation and rotation angle of the terminal. The gyroscope sensor 911 can work with the acceleration sensor 910 to collect 3D actions of the user on the terminal. The processor 901 can realize the following functions according to the data collected by the gyroscope sensor 911: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0124] The pressure sensor 912 can be disposed at the side frame of the terminal and / or the lower layer of the display screen 905. When the pressure sensor 912 is disposed at the side frame of the terminal, the holding signal of the user to the terminal can be detected, and the left-hand or right-hand recognition or shortcut operation can be performed by the processor 901 according to the holding signal collected by the pressure sensor 912. When the pressure sensor 912 is disposed at the lower layer of the display screen 905, the operable control on the UI interface can be controlled by the processor 901 according to the pressure operation of the user to the display screen 905. The operable control includes at least one of the button control, the scroll bar control, the icon control, and the menu control.

[0125] The optical sensor 913 is configured to collect the ambient light intensity. In an embodiment, the processor 901 can control the display brightness of the display screen 905 according to the ambient light intensity collected by the optical sensor 913. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; and when the ambient light intensity is low, the display brightness of the display screen 905 is decreased. In another embodiment, the processor 901 can also dynamically adjust the shooting parameter of the camera assembly 906 according to the ambient light intensity collected by the optical sensor 913.

[0126] The proximity sensor 914, also referred to as the distance sensor, is usually disposed at the front panel of the terminal. The proximity sensor 914 is configured to collect the distance between the user and the front of the terminal. In an embodiment, when the proximity sensor 914 detects that the distance between the user and the front of the terminal gradually decreases, the display screen 905 is switched from the bright screen state to the screen-off state by the processor 901; and when the proximity sensor 914 detects that the distance between the user and the front of the terminal gradually increases, the display screen 905 is switched from the screen-off state to the bright screen state by the processor 901.

[0127] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown in the figures, or combine certain components, or adopt a different component arrangement. Figure 9 The structure shown in the above embodiments does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown in the figures, or combine certain components, or adopt a different component arrangement.

[0128] In an exemplary embodiment, a computer device is also provided, which includes a processor and a memory having at least one computer program stored therein. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the above-described methods for diagnosing a flow fault.

[0129] In an exemplary embodiment, a computer readable storage medium is also provided, which has at least one computer program stored therein. The at least one computer program is loaded and executed by the processor of the computer device to enable the computer to implement any of the above-described methods for diagnosing a flow fault.

[0130] In a possible implementation manner, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0131] In an example embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs any one of the above-mentioned fault diagnosis methods of traffic.

[0132] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0133] It should be understood that "multiple" referred to herein refers to two or more. "And / or", which describes the association relationship of associated objects, means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0134] It should be noted that the terms "first", "second", and the like (if any) in the specification and claims of the present application are used to distinguish similar objects, and do not necessarily have to describe a specific order or chronological order. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following example embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0135] The above is only an example embodiment of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of generating a three-dimensional dynamic portrait, characterized by, The method comprises: receiving two-dimensional position coordinates of a first number of facial feature points and two-dimensional position coordinates of a second number of body part feature points, the two-dimensional position coordinates of the facial feature points being obtained by performing facial recognition on a face image of a person in conversation by an off-vehicle terminal, and the two-dimensional position coordinates of the body part feature points being obtained by performing body part recognition on a body image of the person in conversation by the off-vehicle terminal; inputting internal parameters of a camera, the two-dimensional position coordinates of the facial feature points, and three-dimensional position coordinates of facial reference points marked in a three-dimensional animation model into a two-dimensional-to-three-dimensional conversion formula to obtain two-dimensional-to-three-dimensional conversion coefficients of the face image; the two-dimensional-to-three-dimensional conversion formula comprises: The is a horizontal coordinate of the face feature point, and the is a vertical coordinate of the face feature point, and the s is a scaling parameter, and the is a focal length of the camera in the x-axis direction, and the is a focal length of the camera in the y-axis direction, and the is a horizontal coordinate of the principal point of the camera on the face image, and the is a vertical coordinate of the principal point of the camera on the face image, and the is a horizontal coordinate of the face reference point, and the is a vertical coordinate of the face reference point, and the is a height coordinate of the face reference point, and the is a two-dimensional to three-dimensional conversion coefficient of the face image; inputting the internal parameters of the camera, the two-dimensional position coordinates of the body part feature points, and three-dimensional position coordinates of body part reference points marked in the three-dimensional animation model into the two-dimensional-to-three-dimensional conversion formula to obtain two-dimensional-to-three-dimensional conversion coefficients of the body image; the two-dimensional-to-three-dimensional conversion formula comprises: The is a horizontal coordinate of the body part feature point, and the is a vertical coordinate of the body part feature point, and the s is a scaling parameter, and the is a focal length of the camera in the x-axis direction, and the is a focal length of the camera in the y-axis direction, and the is a horizontal coordinate of the principal point of the camera on the body image, and the is a vertical coordinate of the principal point of the camera on the body image, and the is a horizontal coordinate of the body part reference point, and the is a vertical coordinate of the body part reference point, and the is a height coordinate of the body part reference point, and the is a two-dimensional to three-dimensional conversion coefficient of the body image; determining three-dimensional position coordinates of the facial reference points in the three-dimensional animation model based on the two-dimensional-to-three-dimensional conversion coefficients of the face image and the two-dimensional position coordinates of the first number of facial feature points, and determining three-dimensional position coordinates of the body part reference points in the three-dimensional animation model based on the two-dimensional-to-three-dimensional conversion coefficients of the body image and the two-dimensional position coordinates of the second number of body part feature points; determining a face contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the facial reference points, and determining a body part contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the body part reference points; displaying a three-dimensional dynamic portrait of the person in conversation based on the face contour curve in the three-dimensional animation model and the body part contour curve in the three-dimensional animation model.

2. The method of claim 1, wherein, The method comprises: determining control point coordinates of a first Bezier curve corresponding to the facial reference points based on the three-dimensional position coordinates of the facial reference points; determining the first Bezier curve based on the control point coordinates of the first Bezier curve, and taking the first Bezier curve as the face contour curve in the three-dimensional animation model.

3. The method of claim 1, wherein, The method comprises: determining control point coordinates of a second Bezier curve corresponding to the body part reference points based on the three-dimensional position coordinates of the body part reference points; determining the second Bezier curve based on the control point coordinates of the second Bezier curve, and taking the second Bezier curve as the body part contour curve in the three-dimensional animation model.

4. An apparatus for generating a three-dimensional dynamic portrait, the apparatus comprising: The device comprises: a receiving module configured to receive two-dimensional position coordinates of a first number of facial feature points and two-dimensional position coordinates of a second number of body part feature points, the two-dimensional position coordinates of the facial feature points being obtained by performing facial recognition on a face image of a person in conversation by an off-vehicle terminal, and the two-dimensional position coordinates of the body part feature points being obtained by performing body part recognition on a body image of the person in conversation by the off-vehicle terminal; The internal parameters of the camera, the two-dimensional position coordinates of the face feature points, and the three-dimensional position coordinates of the face reference points marked in the three-dimensional animation model are brought into a two-dimensional to three-dimensional conversion formula to obtain two-dimensional to three-dimensional conversion coefficients of the face image; The two-dimensional to three-dimensional conversion formula comprises: The is a horizontal coordinate of the face feature point, and the is a vertical coordinate of the face feature point, and the s is a scaling parameter, and the is a focal length of the camera in the x-axis direction, and the is a focal length of the camera in the y-axis direction, and the is a horizontal coordinate of the principal point of the camera on the face image, and the is a vertical coordinate of the principal point of the camera on the face image, and the is a horizontal coordinate of the face reference point, and the is a vertical coordinate of the face reference point, and the is a height coordinate of the face reference point, and the is a two-dimensional to three-dimensional conversion coefficient of the face image. The internal parameters of the camera, the two-dimensional position coordinates of the body part feature points, and the three-dimensional position coordinates of the body part reference points marked in the three-dimensional animation model are brought into a two-dimensional to three-dimensional conversion formula to obtain two-dimensional to three-dimensional conversion coefficients of the body image; The two-dimensional to three-dimensional conversion formula comprises: The is a horizontal coordinate of the body part feature point, and the is a vertical coordinate of the body part feature point, and the s is a scaling parameter, and the is a focal length of the camera in the x-axis direction, and the is a focal length of the camera in the y-axis direction, and the is a horizontal coordinate of the principal point of the camera on the body image, and the is a vertical coordinate of the principal point of the camera on the body image, and the is a horizontal coordinate of the body part reference point, and the is a vertical coordinate of the body part reference point, and the is a height coordinate of the body part reference point, and the is a two-dimensional to three-dimensional conversion coefficient of the body image; The first determination module is configured to determine the three-dimensional position coordinates of the face reference points in the three-dimensional animation model based on the two-dimensional to three-dimensional conversion coefficients of the face image and the two-dimensional position coordinates of the first number of face feature points, and determine the three-dimensional position coordinates of the body part reference points in the three-dimensional animation model based on the two-dimensional to three-dimensional conversion coefficients of the body image and the two-dimensional position coordinates of the second number of body part feature points; The second determination module is configured to determine a face contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the face reference points, and determine a body part contour curve in the three-dimensional animation model based on the three-dimensional position coordinates of the body part reference points; The display module is configured to display a three-dimensional dynamic portrait of the call character based on the face contour curve in the three-dimensional animation model and the body part contour curve in the three-dimensional animation model.

5. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one computer program, which is loaded and executed by the processor, so that the computer device implements the method for generating a three-dimensional dynamic portrait according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, which is loaded and executed by the processor, so that the computer implements the method for generating a three-dimensional dynamic portrait according to any one of claims 1 to 3.

Images