Digital human image generation method and device and electronic equipment

Abstract

The invention provides a digital human image generation method and device and electronic equipment, and the method comprises the steps: obtaining expression data corresponding to at least one user and cabin passenger data under the condition that the face posture of the user is detected to meet a preset target face posture; outputting an expression cue word for the expression data, and outputting an attribute cue word for the cabin passenger data; and generating a digital human image corresponding to the user through the expression cue word, the attribute cue word and pre-acquired user photo information. Namely, in the embodiment of the invention, the dynamic and personalized digital human image is generated by detecting the face posture and expression of the user in real time and combining the cabin passenger data, and the intuition and emotion resonance of the user experience are improved through real-time generation and intelligent adjustment.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of robotics technology, and in particular to a method, device and electronic device for generating a digital human image. Background Art

[0002] With the rapid development of smart cars, the user experience in smart cockpits has gradually become the focus of major car manufacturers. As an important part of the smart cockpit, the cockpit digital human is designed to interact with users through virtual images and provide navigation, entertainment, information query and other services. These digital human images not only enhance the sense of technology in the driving experience, but also provide users with a more intuitive and humane way of interaction.

[0003] However, although digital humans in cockpits have great potential in improving user experience, related technical solutions still have many limitations.

[0004] At present, the cockpit digital human images of mainstream car manufacturers are usually pre-stored by users on the car computer. Although this pre-defined digital human image performs well in terms of switching speed and stability, its inherent disadvantages cannot be ignored. Due to the limited number and types of pre-defined digital humans, it is difficult to meet the increasingly diverse and personalized needs of users. As users' requirements for the aesthetics and personalization of the in-car environment increase, the single and fixed digital human image style options appear too limited and lack novelty and appeal. Summary of the invention

[0005] The embodiments of the present application provide a method, device, system, electronic device and computer-readable storage medium for generating a digital human image.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a digital human image, the method comprising:

[0007] When it is detected that the user's facial posture meets the preset target facial posture, acquiring expression data corresponding to at least one user and cabin passenger data;

[0008] Outputting expression prompt words for the expression data, and outputting attribute prompt words for the cabin passenger data;

[0009] The digital human image corresponding to the user is generated through the expression prompt word, the attribute prompt word and the pre-acquired user photo information.

[0010] Optionally, before the step of acquiring expression data corresponding to at least one user and cabin passenger data when it is detected that the user's facial posture meets a preset target facial posture, the method includes:

[0011] Capture user gestures;

[0012] When it is detected that the user's gesture posture meets a preset target gesture posture, obtaining a user's face area;

[0013] A user's facial gesture is detected in the facial area.

[0014] Optionally, detecting a user's facial gesture in the facial area includes:

[0015] Acquire facial key point data, wherein the key point data includes left eye coordinates, right eye coordinates, nose coordinates, left mouth corner coordinates, and right mouth corner coordinates;

[0016] Obtaining a vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate;

[0017] When it is detected that the vertical distance is less than a preset threshold, it is determined that the user's facial posture meets a preset target facial posture.

[0018] Optionally, obtaining the vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate comprises:

[0019] Determine the coordinates of the foot point of the nose coordinate on the line between the left eye coordinate and the right eye coordinate;

[0020] Obtain the vertical distance between the perpendicular foot point coordinates and the nose coordinates.

[0021] Optionally, outputting expression prompt words for the expression data includes:

[0022] Identifying, with respect to the expression data, an expression category corresponding to the expression data;

[0023] The expression type is converted into an expression prompt word, wherein the expression prompt word is used to describe the expression corresponding to the user.

[0024] Optionally, the outputting of attribute prompt words for the cabin passenger data includes:

[0025] Identifying attribute information corresponding to each user in the cabin passenger data according to the cabin passenger data, wherein the attribute information includes age information corresponding to each user;

[0026] The age information is converted into an attribute prompt word, wherein the attribute prompt word is used to describe the corresponding age of the user.

[0027] Optionally, generating a digital human image corresponding to the user through the expression prompt word, the attribute prompt word and pre-acquired user photo information includes:

[0028] Extracting facial features from pre-acquired user photo information;

[0029] The facial features are combined with preset timbre information to adjust the parameters of the preset stable diffusion model to obtain a target stable diffusion model;

[0030] The expression prompt word, the attribute prompt word and the pre-acquired user photo information target are input into a stable diffusion model to generate a digital human image corresponding to the user.

[0031] In a second aspect, an embodiment of the present application provides a digital human image generation device, the device comprising:

[0032] An acquisition module, configured to acquire expression data corresponding to at least one user and cabin passenger data when detecting that the user's facial posture meets a preset target facial posture;

[0033] An output module, used for outputting expression prompt words for the expression data, and outputting attribute prompt words for the cabin passenger data;

[0034] The generating module is used to generate a digital human image corresponding to the user through the expression prompt word, the attribute prompt word and the pre-acquired user photo information.

[0035] Optionally, the device further comprises:

[0036] A capture module, used to capture user gestures;

[0037] A second acquisition module is used to acquire a user face area when it is detected that the user's gesture posture meets a preset target gesture posture;

[0038] The detection module is used to detect the user's facial posture in the face area.

[0039] Optionally, the detection module includes:

[0040] The first detection submodule is used to obtain key point data of the face, wherein the key point data includes left eye coordinates, right eye coordinates, nose coordinates, left mouth corner coordinates and right mouth corner coordinates;

[0041] A second detection submodule, used for acquiring a vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate;

[0042] The third detection submodule is used to determine that the user's facial posture meets a preset target facial posture when it is detected that the vertical distance is less than a preset threshold.

[0043] Optionally, the second detection submodule includes:

[0044] A first detection unit, used to determine the coordinates of the foot point of the nose coordinate on the line between the left eye coordinate and the right eye coordinate;

[0045] The second detection unit is used to obtain the vertical distance between the perpendicular foot point coordinates and the nose coordinates.

[0046] Optionally, the output module includes:

[0047] A first output submodule, configured to identify an expression category corresponding to the expression data based on the expression data;

[0048] The second output submodule is used to convert the expression type into an expression prompt word, wherein the expression prompt word is used to describe the expression corresponding to the user.

[0049] Optionally, the output module includes:

[0050] A third output submodule is used to identify attribute information corresponding to each user in the cabin passenger data according to the cabin passenger data, wherein the attribute information includes age information corresponding to each user;

[0051] The fourth output submodule is used to convert the age information into an attribute prompt word, wherein the attribute prompt word is used to describe the corresponding age of the user.

[0052] Optionally, the generating module includes:

[0053] The first generation submodule is used to extract facial features from the pre-acquired user photo information;

[0054] The second generation submodule is used to adjust the parameters of the preset stable diffusion model by combining the facial features with the preset timbre information to obtain a target stable diffusion model;

[0055] The third generation submodule is used to input the expression prompt word, the attribute prompt word and the pre-acquired user photo information target into the stable diffusion model to generate a digital human image corresponding to the user.

[0056] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement any of the above-described methods for generating a digital human image.

[0057] In a fourth aspect, an embodiment of the present application further provides a storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute any of the digital human image generation methods described above.

[0058] In the embodiment of the present application, when it is detected that the user's facial posture meets the preset target facial posture, the expression data and cabin passenger data corresponding to at least one user are obtained; the expression prompt word is output for the expression data, and the attribute prompt word is output for the cabin passenger data; the digital human image corresponding to the user is generated through the expression prompt word, the attribute prompt word and the pre-acquired user photo information. That is, in the embodiment of the present application, a dynamic and personalized digital human image is generated by real-time detection of the user's facial posture and expression, combined with the cabin passenger data, that is, a dynamically changing digital human image can be generated by real-time detection of the user's facial posture and expression, meeting the user's demand for personalization and dynamism, and is no longer limited to predefined digital human images. Users can generate unique digital human images according to their own expressions and cabin environment, which increases the diversity of choices, and also improves the intuitiveness and emotional resonance of the user experience through real-time generation and intelligent adjustment. In addition, the technology also demonstrates its wide potential in multi-scenario applications, bringing the dual advantages of technological advancement and wide application to smart cockpits and other virtual image application fields.

[0059] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0061] Figure 1 It is a flowchart of the steps of a method for generating a digital human image provided by an embodiment of the present application;

[0062] Figure 2 is a device block diagram of a digital human image generation device provided in an embodiment of the present application;

[0063] Figure 3 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0064] Figure 4 This is a schematic diagram of key points of a face provided in an embodiment of the present application;

[0065] Figure 5 This is a schematic diagram of the starting stage of a digital human image provided by an embodiment of the present application;

[0066] Figure 6 This is a schematic diagram of the digital human image generation stage provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0068] Figure 1 is a flowchart of a method for generating a digital human image provided by an embodiment of the present application. Figure 1 As shown, the method may include:

[0069] Step 101, when it is detected that the user's facial posture meets the preset target facial posture, acquiring expression data corresponding to at least one user and cabin passenger data;

[0070] Further, before step 101, before the step of acquiring expression data corresponding to at least one user and cabin passenger data when detecting that the user's facial posture meets a preset target facial posture, the method includes:

[0071] Capture user gestures;

[0072] When it is detected that the user's gesture posture meets a preset target gesture posture, obtaining a user's face area;

[0073] A user's facial gesture is detected in the facial area.

[0074] It should be noted that in the embodiments of the present application, reference is made to Figure 5 Before generating a digital human image, that is, replicating the digital human image, first, trigger the image replication with an air gesture, that is, use a dynamic gesture to click the image replication function button to trigger the image replication function of the cockpit ceiling screen or the front car screen. At this time, the cockpit AI assistant will ask whether to trigger the image replication function. If a user makes a static gesture of OK at this time, the system will lock the user.

[0075] Secondly, when the car's RGB camera takes a photo, try to keep your face facing the camera during this process.

[0076] Next, the face is detected or located, and the area where the user's face is located, that is, the user's face area, can be located through a face detection algorithm.

[0077] Then, the facial key points can be predicted and the user's facial posture can be locked through the facial key point detection algorithm. The schematic diagram of the facial key points is as follows: Figure 4 As shown, the set face posture determination rules are used to determine whether the user has a qualified photo posture. If the face is not facing the camera, the AI ​​assistant will remind the user to take the photo again. If the face is facing the camera, a high-quality photo of the person can be obtained, and the assistant will ask whether to perform the image replication function.

[0078] Specifically, detecting the user's facial posture in the face area may be performed by the following steps:

[0079] Among them, in the above steps, detecting the user's facial posture in the face area includes:

[0080] Acquire facial key point data, wherein the key point data includes left eye coordinates, right eye coordinates, nose coordinates, left mouth corner coordinates, and right mouth corner coordinates;

[0081] Obtaining a vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate;

[0082] When it is detected that the vertical distance is less than a preset threshold, it is determined that the user's facial posture meets a preset target facial posture.

[0083] Further, the obtaining of the vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate comprises:

[0084] Determine the coordinates of the foot point of the nose coordinate on the line between the left eye coordinate and the right eye coordinate;

[0085] Obtain the vertical distance between the perpendicular foot point coordinates and the nose coordinates.

[0086] First, the user's facial posture can be judged from the front and side.

[0087] Obtain the key point data of the face, including the coordinates of the left eye, right eye, nose, left corner of the mouth and right corner of the mouth, that is, the coordinates of the left eye, right eye, nose, left corner of the mouth and right corner of the mouth; calculate the vertical distance from the nose coordinates (nose tip point) to the line between the two eyes, and use this distance as the basis for judging the profile face.

[0088] Specifically, the coordinates of the foot of the perpendicular point can be determined through three points (left eye, right eye, and nose tip), the distance from the nose tip to the foot of the perpendicular point can be calculated, and the distance can be normalized and compared with the ratio of the distance between the eyes to determine whether the face is sideways.

[0089] If the ratio is greater than a preset threshold (eg, 0.5), it is determined to be a profile face, otherwise it is determined to be a front face.

[0090] The above steps can be expressed in formula form as follows:

[0091] First, the key points in the face are defined, that is, the coordinates of the left eye corner key point p1 are (x1, y1), the coordinates of the right eye corner key point p2 are (x2, y2), and the coordinates of the nose tip key point p3 are (x3, y3).

[0092] To calculate the midpoints of the two eyes, first calculate the coordinates of the perpendicular point A:

[0093] The foot point A=(a x ,a y )satisfy:

[0094] (x2-x1)(x3-a x )+(y2-y1)(y3-a y )=0(Formula 1)

[0095]

[0096] a y =k*a x +b(Formula 3)

[0097] Among them, in the above formulas 1, 2, and 3, k and b are the slope and intercept of the straight line equation y=kx+b determined by the coordinates of the left eye corner and the right eye corner.

[0098] Calculate the normalized vertical distance ratio:

[0099]

[0100] If dis front >0.5, it is judged as a profile face, otherwise it is judged as a frontal face.

[0101] Secondly, the user's facial posture can be judged by looking up or down.

[0102] A new straight line equation is established through the foot point of the perpendicular line between the nose tip and the center of the two eyes. The intersection of the two equations is obtained through the straight line equation of the line connecting the corners of the mouth. The ratio of the distance from the nose tip to the foot point of the perpendicular to the distance from the intersection to the foot point of the perpendicular is calculated to determine whether the face is looking up or down. If the ratio is not within the preset range (for example, between 0.2 and 0.7), it is determined to be looking up or down.

[0103] The above steps can be expressed in formula form as follows:

[0104] First, the key points in the face are defined. The coordinates of the left corner of the mouth key point p4 are (x4, y4), and the coordinates of the right corner of the mouth key point p5 are (x5, y5);

[0105] The equation of the straight line passing through the nose tip point and the perpendicular foot point A is y=k1x+c1, where k1 and c1 are calculated from the coordinates of the nose tip point p3 and the perpendicular foot point A.

[0106] Using the straight line equation y=k2x+c2 of the mouth corner points p4 and p5, calculate the coordinates of the intersection point B of the two straight lines:

[0107]

[0108] y B =k1x B +c1(Formula 6)

[0109] Calculate the normalized ratio dis related to looking up or looking down bow :

[0110]

[0111] If dis bow If it is not within the range [0.2,0.7], it is judged as looking up or down.

[0112] After obtaining high-quality photos of people, you need to go through the steps of image reproduction.

[0113] Step 102, outputting expression prompt words for the expression data, and outputting attribute prompt words for the cabin passenger data;

[0114] Furthermore, the outputting of expression prompt words for the expression data includes:

[0115] Identifying, with respect to the expression data, an expression category corresponding to the expression data;

[0116] The expression type is converted into an expression prompt word, wherein the expression prompt word is used to describe the expression corresponding to the user.

[0117] Furthermore, the outputting of attribute prompt words for the cabin passenger data includes:

[0118] Identifying attribute information corresponding to each user in the cabin passenger data according to the cabin passenger data, wherein the attribute information includes age information corresponding to each user;

[0119] The age information is converted into an attribute prompt word, wherein the attribute prompt word is used to describe the corresponding age of the user.

[0120] It should be noted that in the embodiments of the present application, reference is made to Figure 6 , shows a process of converting user photos into character images, which involves the recognition and generation of expressions and ages.

[0121] Among them, the expression data includes the user's expression information, and the cabin passenger data includes the user's age, gender and other attribute information.

[0122] The expression data is processed by the "expression detection and recognition network" to output the expression category, and the expression category information is converted into the corresponding prompt describing the expression.

[0123] The cabin passenger data is processed by the "U-Net semantic segmentation network" to output age attribute categories, and the age attribute category information is converted into prompts describing age.

[0124] Step 103: Generate a digital human image corresponding to the user through the expression prompt word, the attribute prompt word and the pre-acquired user photo information.

[0125] Furthermore, generating a digital human image corresponding to the user through the expression prompt word, the attribute prompt word and the pre-acquired user photo information includes:

[0126] Extracting facial features from pre-acquired user photo information;

[0127] The facial features are combined with preset timbre information to adjust the parameters of the preset stable diffusion model to obtain a target stable diffusion model;

[0128] The expression prompt word, the attribute prompt word and the pre-acquired user photo information target are input into a stable diffusion model to generate a digital human image corresponding to the user.

[0129] The pre-acquired high-quality user photos are input into the IP-Adapter together with the generated expression and age prompt for further processing.

[0130] IP-Adapter combines the generated Prompt with the photo information and passes it to the Stable Diffusion model to generate the final character image.

[0131] The logic of the entire process is to identify the user's expression and age attributes, generate a corresponding text description (Prompt), combine it with high-quality user photos, and finally generate an image with specific expression and age characteristics.

[0132] It should be noted that in the embodiment of the present application, for the semantic segmentation task of the above age attribute, a lightweight segmentation network U-Net is used to train the verified data. For the above gesture recognition and expression recognition tasks, hand detection and face detection use a lightweight detection network based on MobileNet V4, and hand recognition and expression recognition use a lightweight classification network based on MobileNet V4.

[0133] The hand detection and recognition network consists of a hand detection network based on MobileNet V4 and a gesture classification network based on MobileNet V4.

[0134] The core of MobileNet V4 (MNv4) lies in the two major modules of Universal Inverted Bottleneck (UIB) and Mobile MQA, combined with an optimized neural network architecture search (NAS) formula.

[0135] MNv4 introduces the Universal Inverted Neck (UIB) based on MobileNet's classic components such as separable depthwise convolution (DW) and pointwise (PW) expansion and projection inverted bottleneck blocks. This structure is quite simple, introducing two optional depthwise convolutions (DW) in the inverted bottleneck block, one before the expansion layer and the other between the expansion layer and the projection layer. The presence or absence of these two depthwise convolutions is part of the neural architecture search (NAS) optimization process, which ultimately generates a novel network architecture. Although this modification seems simple, the author cleverly unified several important modules, including the original inverted neck structure, ConvNext, and FFN in ViT. In addition, UIB also introduces a new variant: ExtraDW.

[0136] The inverted neck structure is flexibly constructed during the neural network architecture search process, eliminating the need to manually formulate scaling rules. In addition, combined with the SuperNet-based neural network architecture search algorithm, this approach achieves parameter sharing (>95%) across different instantiations, making NAS extremely efficient.

[0137] Complementing UIB is Mobile MQA, a groundbreaking attention block designed specifically for accelerators, achieving up to 39% inference acceleration.

[0138] Taking a 480*288*3 RGB image as an input, the hand and face detection network based on Mobilenetv4 small performs feature extraction. The model has two output task heads, one corresponding to the Gaussian heat map of the center point of the gesture target box, and the other corresponding to the upper left and lower right deviations of the target box relative to the box center. Similarly, taking a 224*224*3 RGB image as an input, the gesture classification and expression classification network based on Mobilenetv4 performs gesture feature and facial expression feature extraction.

[0139] The core methods used in this module include Stable Diffusion and Ip-Adapter. Stable Diffusion is a generative model. Currently, the most advanced models in the image and video fields are based on the modification of the Stable Diffusion Model. The Diffusion Model is inspired by the diffusion process in physics. The whole process can be divided into two stages: the diffusion process and the reverse process. Given a real image sample X0, diffusion adds Gaussian noise to it through T cumulative times in the diffusion process to obtain X1, X2,…, X T The size of each step is determined by a series of Gaussian distributions with a variance hyperparameter Since each moment t of the diffusion process is only related to the moment t-1, it can also be regarded as a Markov process:

[0140]

[0141] In this process, as increases, gets closer and closer to pure noise. When is completely Gaussian noise. The reverse process of the diffusion model is the denoising inference process, which is achieved by sampling in random Gaussian noise, that is, using a neural network to predict the added noise, and using the actual noise added in the forward process as supervision, iteratively implementing the denoising and reconstruction task step by step.

[0142] Stable Diffusion is an improvement on the Diffusion model. Its core part is to first use an autoencoder to perceptually compress the image, and then diffuse it from the compressed latent space, thereby significantly reducing the computational complexity of the Diffusion model. It is worth mentioning that in the Stable Diffusion model (SD model), a cross-attention mechanism can be used to inject information of other modalities (such as text information) into the reverse diffusion process, thereby achieving precise control of the generated image. The present invention uses the SD model to achieve personalized and rapid generation of digital human characters.

[0143] In order to make the generated digital human closer to the user's desired image, the image replication function module adopts IP-Adapter technology. IP-Adapter is a technology used to fine-tune the SD model. It improves the performance of SD in specific tasks by adjusting a small-scale model. IP-Adapter is widely used to fine-tune the SD model to achieve the generation of specific human images. It separates the cross-attention layer of text features and image features by decoupling the cross-attention mechanism, guides the image generation process, and thus achieves the generation of specific images. Specifically, IP-Adapter technology can adapt to specific task requirements by adjusting a small number of parameters based on the SD model. For example, when generating a specific digital human image, IP-Adapter extracts facial features based on the photos uploaded by the user, and fine-tunes the SD model based on the timbre information specified by the user, so that the generated digital human image is more in line with the user's personalized needs. This process makes the image generation process more accurate and flexible by separating the cross-attention mechanism of text and image features, thereby achieving high-quality and personalized digital human images.

[0144] In the embodiment of the present application, when it is detected that the user's facial posture meets the preset target facial posture, the expression data and cabin passenger data corresponding to at least one user are obtained; the expression prompt word is output for the expression data, and the attribute prompt word is output for the cabin passenger data; the digital human image corresponding to the user is generated through the expression prompt word, the attribute prompt word and the pre-acquired user photo information. That is, in the embodiment of the present application, a dynamic and personalized digital human image is generated by real-time detection of the user's facial posture and expression, combined with the cabin passenger data, that is, a dynamically changing digital human image can be generated by real-time detection of the user's facial posture and expression, meeting the user's demand for personalization and dynamism, and is no longer limited to predefined digital human images. Users can generate unique digital human images according to their own expressions and cabin environment, which increases the diversity of choices, and also improves the intuitiveness and emotional resonance of the user experience through real-time generation and intelligent adjustment. In addition, the technology also demonstrates its wide potential in multi-scenario applications, bringing the dual advantages of technological advancement and wide application to smart cockpits and other virtual image application fields.

[0145] Corresponding to the method provided in the above-mentioned embodiment of the digital human image generation method of the present application, see Figure 2 The present application also provides a device block diagram of a digital human image generation device. In this embodiment, the device includes:

[0146] An acquisition module 201 is used to acquire expression data corresponding to at least one user and cabin passenger data when it is detected that the user's facial posture meets a preset target facial posture;

[0147] An output module 202, configured to output expression prompt words for the expression data, and output attribute prompt words for the cabin passenger data;

[0148] The generation module 203 is used to generate a digital human image corresponding to the user through the expression prompt word, the attribute prompt word and the pre-acquired user photo information.

[0149] Optionally, the device further comprises:

[0150] A capture module, used to capture user gestures;

[0151] A second acquisition module is used to acquire a user face area when it is detected that the user's gesture posture meets a preset target gesture posture;

[0152] The detection module is used to detect the user's facial posture in the face area.

[0153] Optionally, the detection module includes:

[0154] The first detection submodule is used to obtain key point data of the face, wherein the key point data includes left eye coordinates, right eye coordinates, nose coordinates, left mouth corner coordinates and right mouth corner coordinates;

[0155] A second detection submodule, used for acquiring a vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate;

[0156] The third detection submodule is used to determine that the user's facial posture meets a preset target facial posture when it is detected that the vertical distance is less than a preset threshold.

[0157] Optionally, the second detection submodule includes:

[0158] A first detection unit, used to determine the coordinates of the foot point of the nose coordinate on the line between the left eye coordinate and the right eye coordinate;

[0159] The second detection unit is used to obtain the vertical distance between the perpendicular foot point coordinates and the nose coordinates.

[0160] Optionally, the output module includes:

[0161] A first output submodule, configured to identify an expression category corresponding to the expression data based on the expression data;

[0162] The second output submodule is used to convert the expression type into an expression prompt word, wherein the expression prompt word is used to describe the expression corresponding to the user.

[0163] Optionally, the output module includes:

[0164] A third output submodule is used to identify attribute information corresponding to each user in the cabin passenger data according to the cabin passenger data, wherein the attribute information includes age information corresponding to each user;

[0165] The fourth output submodule is used to convert the age information into an attribute prompt word, wherein the attribute prompt word is used to describe the corresponding age of the user.

[0166] Optionally, the generating module includes:

[0167] The first generation submodule is used to extract facial features from the pre-acquired user photo information;

[0168] The second generation submodule is used to adjust the parameters of the preset stable diffusion model by combining the facial features with the preset timbre information to obtain a target stable diffusion model;

[0169] The third generation submodule is used to input the expression prompt word, the attribute prompt word and the pre-acquired user photo information target into the stable diffusion model to generate a digital human image corresponding to the user.

[0170] In summary, the digital human image generation device provided in the embodiment of the present application generates a dynamic and personalized digital human image by detecting the user's facial posture and expression in real time and combining the cabin passenger data. That is, it can generate a dynamically changing digital human image by detecting the user's facial posture and expression in real time to meet the user's needs for personalization and dynamism. Moreover, it is no longer limited to predefined digital human images. Users can generate unique digital human images based on their own expressions and cabin environment, which increases the diversity of choices. It also improves the intuitiveness and emotional resonance of the user experience through real-time generation and intelligent adjustment. In addition, the technology also demonstrates its broad potential in multi-scenario applications, bringing the dual advantages of technological advancement and wide application to smart cockpits and other virtual image application fields.

[0171] Figure 3 This is a structural diagram of an electronic device M00 provided in an embodiment of the present application. In the diagram, the electronic device M00 includes a processor M01 and a memory M02. The electronic device includes: a processor; a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement any of the digital human image generation methods described above, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0172] In an embodiment of the present application, the memory M02 can be used to store software programs and various data. The memory M02 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory M02 may include a volatile memory or a non-volatile memory, or the memory x09 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory M02 in the embodiment of the present application includes but is not limited to these and any other suitable types of memories.

[0173] The processor M01 may include one or more processing units; optionally, the processor M01 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor M01.

[0174] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned digital human image generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0175] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0176] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned digital human image generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0177] It should be understood that the chip involved in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0178] An embodiment of the present application further provides a storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the digital human image generation methods described above.

[0179] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned digital human image generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0180] An embodiment of the present application also provides a vehicle, which includes the digital human image generating device as described above.

[0181] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0182] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0183] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for generating a digital human image, characterized in that: The method comprises: When it is detected that the user's facial posture meets the preset target facial posture, acquiring expression data corresponding to at least one user and cabin passenger data; Outputting expression prompt words for the expression data, and outputting attribute prompt words for the cabin passenger data; The digital human image corresponding to the user is generated by using the expression prompt word, the attribute prompt word and the pre-acquired user photo information.

2. The method according to claim 1, characterized in that Before the step of acquiring expression data corresponding to at least one user and cabin passenger data when detecting that the user's facial posture meets a preset target facial posture, the method includes: Capture user gestures; When it is detected that the user's gesture posture meets a preset target gesture posture, obtaining a user's face area; A user's facial gesture is detected in the facial area.

3. The method according to claim 2, characterized in that Detecting the user's facial gesture in the face area includes: Acquire facial key point data, wherein the key point data includes left eye coordinates, right eye coordinates, nose coordinates, left mouth corner coordinates, and right mouth corner coordinates; Obtaining a vertical distance between the nose coordinate and a line connecting the left eye coordinate and the right eye coordinate; When it is detected that the vertical distance is less than a preset threshold, it is determined that the user's facial posture meets a preset target facial posture.

4. According to claim 3, the obtaining of the vertical distance between the nose coordinate and the line connecting the left eye coordinate and the right eye coordinate comprises: Determine the coordinates of the foot point of the nose coordinate on the line between the left eye coordinate and the right eye coordinate; Obtain the vertical distance between the perpendicular foot point coordinates and the nose coordinates.

5. The method according to claim 1, characterized in that Outputting expression prompt words for the expression data includes: Identifying, with respect to the expression data, an expression category corresponding to the expression data; The expression type is converted into an expression prompt word, wherein the expression prompt word is used to describe the expression corresponding to the user.

6. The method according to claim 1, characterized in that The output attribute prompt words for the cabin passenger data include: Identifying attribute information corresponding to each user in the cabin passenger data according to the cabin passenger data, wherein the attribute information includes age information corresponding to each user; The age information is converted into an attribute prompt word, wherein the attribute prompt word is used to describe the corresponding age of the user.

7. The method according to claim 1, characterized in that The generating of the digital human image corresponding to the user by using the expression prompt word, the attribute prompt word and the pre-acquired user photo information comprises: Extracting facial features from pre-acquired user photo information; The facial features are combined with preset timbre information to adjust the parameters of the preset stable diffusion model to obtain a target stable diffusion model; The expression prompt word, the attribute prompt word and the pre-acquired user photo information target are input into a stable diffusion model to generate a digital human image corresponding to the user.

8. A digital human image generation device, characterized in that: The device comprises: An acquisition module, configured to acquire expression data corresponding to at least one user and cabin passenger data when detecting that the user's facial posture meets a preset target facial posture; An output module, used for outputting expression prompt words for the expression data, and outputting attribute prompt words for the cabin passenger data; The generating module is used to generate a digital human image corresponding to the user through the expression prompt word, the attribute prompt word and the pre-acquired user photo information.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for generating a digital human image as described in any one of claims 1 to 7.

10. A computer storage medium, a readable storage medium, for storing a program, characterized in that: When the program is executed by a processor, the method for generating a digital human image as claimed in any one of claims 1 to 7 is implemented.

Images