Voice assistant image display method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a voice assistant image display method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: under the condition that a voice instruction is monitored, obtaining user appearance characteristics of a target user sending the voice instruction; the user appearance features are extracted based on target image data of the target user; and performing fusion processing on the basis of the appearance user characteristics of the target user and the target virtual image to obtain a voice assistant image, and displaying the voice assistant image. By adopting the method, the voice assistant image can be enriched, and the display flexibility of the voice assistant image is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment, readable storage medium and program product for displaying a voice assistant image. Background Art

[0002] With the rapid development of science and technology, cars are no longer just a means of transportation, but can also provide emotional value. Among them, voice assistants are one of the common functions of cars. Voice assistants can not only provide vehicle control functions based on voice interaction, but also provide emotional companionship during driving.

[0003] In traditional technology, when the car's voice assistant is turned on, when the user interacts with the voice assistant by voice, the car's display screen will display a preset voice assistant image to make the voice assistant more vivid and improve the user interaction experience. However, the current voice assistant image is pre-set in the car's system, with a single image and poor display flexibility. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, readable storage medium and program product for displaying the image of a voice assistant, which can enrich the image of the voice assistant and improve the flexibility of the display of the voice assistant image, in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for displaying a voice assistant image, the method comprising:

[0006] In the case of detecting a voice command, obtaining a user appearance feature of a target user who issues the voice command; the user appearance feature is extracted based on target image data of the target user;

[0007] Based on the on-screen user characteristics of the target user and the target virtual image, a fusion process is performed to obtain a voice assistant image, and the voice assistant image is displayed.

[0008] In one embodiment, the method further comprises:

[0009] Acquire user image data in a target area, and extract user appearance features of each user in the target area based on the user image data;

[0010] In the case where a voice command is detected, obtaining the user appearance characteristics of the target user who issued the voice command includes:

[0011] In the case of detecting a voice command, determining the location information of the target user;

[0012] According to the location information of the target user, the user appearance feature of the target user is identified from among the user appearance features included in the user image data.

[0013] In one embodiment, when a voice command is detected, determining the location information of the target user includes:

[0014] Collecting voice data of users in the target area by means of a sound pickup component;

[0015] Identifying whether the user's voice data contains a voice command;

[0016] When the voice data of a target user among the users includes a voice command, the position information of the target user is determined according to the position information of a sound pickup component corresponding to the voice data.

[0017] In one embodiment, before the fusion processing based on the on-screen user characteristics of the target user and the target virtual image to obtain the voice assistant image, the method further includes:

[0018] Based on the target image data, extracting user attribute features of the target user;

[0019] Among a plurality of pre-stored candidate virtual images, a target virtual image matching the user attribute characteristics is determined.

[0020] In one embodiment, before the fusion processing based on the on-screen user characteristics of the target user and the target virtual image to obtain the voice assistant image, the method further includes:

[0021] Based on the target image data, extracting user attribute features of the target user;

[0022] Inputting the user attribute characteristics into a preference prediction model to obtain the preference characteristics of the target user;

[0023] A target virtual image matching the preference characteristics of the target user is downloaded from the server.

[0024] In one embodiment, the user appearance features include at least facial features and clothing features; the fusion processing based on the user appearance features of the target user and the target virtual image to obtain the voice assistant image includes:

[0025] Based on the facial features of the target user, adjusting the facial image of the target virtual image to obtain a facial image matching the facial features;

[0026] Based on the clothing features of the target user, adjusting the body image of the target avatar to obtain a body image that matches the clothing features;

[0027] A voice assistant image is generated based on the facial image and the body image.

[0028] In a second aspect, the present application provides a device for displaying a voice assistant image, the device comprising:

[0029] An acquisition module is used to acquire the user appearance features of the target user who issued the voice command when a voice command is detected; the user appearance features are extracted based on the target image data of the target user;

[0030] The display module is used to perform fusion processing based on the on-screen user characteristics of the target user and the target virtual image to obtain a voice assistant image and display the voice assistant image.

[0031] In one embodiment, the device further comprises:

[0032] A first extraction module, configured to obtain user image data in a target area, and extract user appearance features of each user in the target area based on the user image data;

[0033] The acquisition module is specifically used for:

[0034] In the case of detecting a voice command, determining the location information of the target user;

[0035] According to the location information of the target user, the user appearance feature of the target user is identified from among the user appearance features included in the user image data.

[0036] In one embodiment, the acquisition module is specifically used to:

[0037] Collecting voice data of users in the target area by means of a sound pickup component;

[0038] Identifying whether the user's voice data contains a voice command;

[0039] When the voice data of a target user among the users includes a voice command, the position information of the target user is determined according to the position information of a sound pickup component corresponding to the voice data.

[0040] In one embodiment, the device further comprises:

[0041] A second extraction module, used for extracting user attribute features of the target user based on the target image data;

[0042] The first determination module is used to determine a target virtual image that matches the user attribute characteristics from a plurality of pre-stored candidate virtual images.

[0043] In one embodiment, the device further comprises:

[0044] A third extraction module, used to extract user attribute features of the target user based on the target image data;

[0045] A second determination module is used to input the user attribute characteristics into a preference prediction model to obtain the preference characteristics of the target user;

[0046] The download module is used to download a target virtual image matching the preference characteristics of the target user from a server.

[0047] In one embodiment, the user appearance features include at least facial features and clothing features; the display module is specifically used to:

[0048] Based on the facial features of the target user, adjusting the facial image of the target virtual image to obtain a facial image matching the facial features;

[0049] Based on the clothing features of the target user, adjusting the body image of the target avatar to obtain a body image that matches the clothing features;

[0050] A voice assistant image is generated based on the facial image and the body image.

[0051] According to a third aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0052] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0053] According to a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0054] The above-mentioned method, device, computer equipment, computer-readable storage medium and computer program product for displaying the voice assistant image can obtain the target image data of the target user corresponding to the voice command when the voice command is detected; the target user is the user in the target area, and then the user appearance characteristics of the target user are extracted based on the target image data, and then the appearance characteristics of the target user and the target virtual image are fused to obtain the voice assistant image, and the voice assistant image is displayed in the display component of the target area. In this solution, the voice assistant image can be generated according to the characteristics of the initiator of the voice command and combined with the target virtual image. The voice assistant image is flexible and changeable, which effectively improves the flexibility and fun of the display. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A diagram showing an application environment of a method for displaying a voice assistant image in one embodiment;

[0057] Figure 2 is a flowchart of a method for displaying a voice assistant image in one embodiment;

[0058] Figure 3 A flowchart of an example of a method for displaying a voice assistant image in one embodiment;

[0059] Figure 4 is a structural block diagram of a display device for a voice assistant image in one embodiment;

[0060] FIG. 5 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] The embodiment of the present application provides a method for displaying a voice assistant image. The present embodiment uses the method applied to a terminal as an example. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this solution, the terminal can be a vehicle, such as a car, an electric car, etc., or a mobile phone, a tablet computer, a ship, etc. It can be understood that the method can be applied to any terminal with a voice assistant function, and the embodiment of the present application does not limit it. Figure 1 As shown, a schematic diagram of an application scenario provided by an embodiment of the present application includes an image acquisition unit, a sound detection unit, a cockpit domain controller (including a sound processing unit and an image processing unit), an integrated processing unit, and a display unit. Among them, the image acquisition unit can be a high-definition camera, an OMS (Occupant Monitor System) camera, etc., which is used to collect image data of users in the target area. The sound detection unit can be a microphone, which can collect sound data of users in the target area. The sound processing unit and the image processing unit can be used for sound data and image data, respectively; the integrated processing unit is used to perform integrated processing on the processing results of the sound data and the image data to generate a voice assistant image. The display unit can be a display screen, an instrument panel, etc., which is used to display the voice assistant image.

[0063] In one example, the terminal may be a vehicle. Among them, the sound detection unit may be a microphone arranged next to each seat of the vehicle, used to obtain the voice of users in different seats and send it to the sound processing unit. The image acquisition unit may be an OMS camera arranged inside the vehicle, used to obtain the image data of the user in the vehicle and send it to the image processing unit. The sound processing unit may be located in the cockpit domain controller in the vehicle, judge and process the received user voice, judge which seat the user who issued the voice command is, and send the seat information to the comprehensive processing unit. The image processing unit may be located in the cockpit domain controller in the vehicle, used to perform image analysis on the received image of the user in the vehicle, and determine the user attribute characteristics and user appearance characteristics of each user in each seat in the image. The comprehensive processing unit may be located in the cockpit domain controller in the vehicle, used to comprehensively analyze and judge the results of the sound processing unit and the image processing unit, that is, to generate a voice assistant image according to the user attribute characteristics, the user appearance characteristics and the target virtual image. The display unit may be arranged in the vehicle to display the image information of the voice image.

[0064] In this embodiment, Figure 2 As shown, the method comprises the following steps:

[0065] Step 202: when a voice command is detected, obtaining the user appearance features of the target user who issues the voice command.

[0066] Wherein, the user appearance feature is extracted based on the target image data of the target user;

[0067] In an embodiment of the present application, the terminal can monitor in real time whether there is a user making a sound at the moment. When the user's sound data is monitored, it can further identify whether the sound data contains a voice command. If a voice command is included, the target user who issued the voice command can be located, and the user appearance features of the target user who issued the voice command can be obtained based on the location information of the target user. If no voice command is included, no processing is performed. Among them, the voice command is any command that can interact with the voice assistant, such as a wake-up command of a voice assistant, or other control commands of a voice assistant. Optionally, the target user can be a user in the same space as the terminal. For example, the target user can be a passenger and the terminal can be a vehicle.

[0068] In one example, when the terminal detects the sound data in the cabin, it can identify whether the sound data contains voice commands. If it contains voice commands, the target user who issued the voice command can be located, and then the user appearance features of the target user can be obtained. Among them, the user appearance features may include but are not limited to facial features, clothing features, hairstyle features, etc. Facial features are visual features related to the user's face, and may include at least one of face shape, whether there is a beard, etc.; clothing features may include at least one of clothing color, style (such as T-shirt, jacket, dress), accessories, etc. Hairstyle features may include at least one of hair color, length, hairstyle, etc.

[0069] Step 204: A fusion process is performed based on the target user's appearance and appearance user characteristics and the target virtual image to obtain a voice assistant image, and the voice assistant image is displayed.

[0070] In an embodiment of the present application, after the terminal obtains the user appearance characteristics of the target user, it can obtain the target virtual image. The target virtual image can be pre-stored locally, can be obtained through a wireless communication network, or can be randomly generated in real time. The terminal can fuse the target user's appearance characteristics and the target virtual image to obtain a voice assistant image, and then display the voice assistant image through a display unit. The display unit may include but is not limited to one or more of an instrument panel, a central control screen, and a rear screen.

[0071] In this solution, the voice assistant image can be generated based on the appearance characteristics of the initiator of the voice command and combined with the target virtual image. The voice assistant image is flexible and changeable, which effectively improves the flexibility and fun of the display.

[0072] Optionally, the method also includes: acquiring user image data within the target area, and extracting user appearance features of each user within the target area based on the user image data; accordingly, when a voice command is detected, acquiring user appearance features of the target user who issues the voice command, including: when a voice command is detected, determining location information of the target user; and identifying user appearance features of the target user from among the user appearance features contained in the user image data based on the location information of the target user.

[0073] Among them, the target area is a designated area that needs to be monitored, such as the seating area of ​​​​the passengers in the cabin, a specific area in the vehicle, etc.

[0074] In an embodiment of the present application, the terminal can obtain the image data (i.e., user image data) of all users in the target area in real time through a camera, and then analyze the user image data through face recognition technology and a deep learning model to obtain an image analysis result. The image analysis result may include the user appearance features of each user extracted from the user image data, and the location information of each user. In addition, it may also include user attribute features of each user. Among them, the location information may represent the seat position of the user in the cabin. For example, the location information may be the co-pilot seat, the left rear seat, the middle rear seat, etc.

[0075] When a voice command is detected, the terminal can first determine the location information of the target user based on the sound data containing the voice command, and then, based on the location information of the target user, identify the target user at the corresponding location in the recognition results of the user image data, and then obtain the user appearance characteristics of the target user.

[0076] In one example, the OMS camera in the car can obtain image information of all users in the car in real time and send the image information to the image processing unit for analysis and processing. The image processing unit uses face recognition technology and deep learning models to determine the user's gender and age information based on the image of each user, and extracts the user's appearance features (such as the color and pattern / design of the top), and then sends this information to the comprehensive processing unit.

[0077] In this solution, image recognition technology can be used to accurately identify the user's appearance features, and then based on the location information of the target user who issues the voice command, the target user can be accurately located among multiple users, and then the user appearance features of the target user can be obtained, so that the image of the voice assistant matches the appearance of the user who issues the voice command, effectively improving the flexibility and fun of voice interaction.

[0078] Optionally, when a voice command is detected, the location information of the target user is determined, including: collecting voice data of users in the target area through a sound pickup component; identifying whether the voice data of the users contains voice commands; and when the voice data of the target user among the users contains voice commands, determining the location information of the target user based on the location information of the sound pickup component corresponding to the sound data.

[0079] In an embodiment of the present application, the terminal can collect the sound data of users in the target area through a sound pickup component, and then identify whether the user's sound data contains a voice command through a sound processing unit. If the sound data of the target user among the users contains a voice command, the location information of the target user can be determined based on the location information of the sound pickup component that sends the sound data. Optionally, a sound pickup positioning algorithm can also be used in the present application to locate the position of the target user. Any algorithm with a positioning function can be applied to the embodiment of the present application, and the embodiment of the present application is not limited. Among them, the position information can represent the seat position of the user in the cabin. For example, the position information can be the co-pilot seat, the left rear seat, the middle rear seat, etc.

[0080] In one example, microphones can be installed next to each seat in the cabin, and the acquired sound data can be sent to the sound processing unit for analysis and processing. When a user in the car issues a voice command, the sound decibel values ​​acquired by each microphone are compared to determine the position of the microphone with the loudest volume, which is the position of the target user who issued the voice command. In particular, if the sound decibel value acquired by the rear microphone is greater than that of the front row, and the sound decibel value acquired by the microphones of the left and right doors of the rear row is less than the preset threshold (less than 5db), the middle user in the rear row is judged to be the target user. And the position of the target user is sent to the integrated processing unit. Alternatively, the sound data can also be content analyzed. If the sound data contains the content of the preset voice command, the position of the microphone that collects the sound data is the position of the target user who issued the voice command.

[0081] In this solution, the recognition accuracy of the target user is improved through dual confirmation of sound and image, and more accurate voice image feedback is given to ensure that the image of the voice assistant matches the appearance of the user who issues the voice command, effectively improving the flexibility and fun of voice interaction.

[0082] Optionally, before obtaining the voice assistant image, the method further includes: extracting user attribute characteristics of the target user based on the target image data; and determining a target virtual image that matches the user attribute characteristics from a plurality of pre-stored candidate virtual images.

[0083] In an embodiment of the present application, the terminal can obtain all user image data in the target area in real time through a camera, and then send it to the image processing unit. The image processing unit can analyze the user image data through face recognition technology and a deep learning model to obtain an image analysis result. The image analysis result may include the user appearance features of each user extracted from the user image data, the location information of each user, and the user attribute features of each user. Among them, the user attribute features may include the user's age and gender information. The image processing unit may send the image analysis result to the comprehensive processing unit. The comprehensive processing unit may receive the location information of the target user sent by the sound processing unit. The location information may characterize the user's seat position in the cabin. For example, the location information may be the co-pilot seat, the left rear seat, the middle rear seat, etc.

[0084] The comprehensive processing unit can determine the target user corresponding to the position information in the image analysis result according to the position information sent by the sound processing unit, and then obtain the user attribute characteristics of the target user. The comprehensive processing unit can store multiple candidate virtual images, and according to the pre-stored correspondence between the user attribute characteristics and the virtual image, determine the candidate virtual image corresponding to the user attribute characteristics among the multiple candidate virtual images as the target virtual image.

[0085] In one example, the age group of the target user can be determined, and then the first type of the target user can be determined, such as children under 12 years old, teenagers between 12 and 18 years old, young and middle-aged people between 19 and 50 years old, and the elderly over 50 years old. The gender of the target user can also be determined as the second type, and then the first type and the second type are combined into the user type of the target user, such as male children, female children, male teenagers, female teenagers, young and middle-aged men, young and middle-aged women, male elderly, and female elderly. Then, according to the correspondence between the user type and the virtual image, a candidate virtual image corresponding to the user attribute characteristics is determined from multiple candidate virtual images as the target virtual image.

[0086] In this solution, a virtual image that matches the user can be determined so that the virtual image matches the user's interests, thereby improving the user experience.

[0087] Optionally, before obtaining the voice assistant image, the method further includes: extracting user attribute characteristics of the target user based on the target image data; inputting the user attribute characteristics into a preference prediction model to obtain the preference characteristics of the target user; and downloading a target virtual image that matches the preference characteristics of the target user from a server.

[0088] In an embodiment of the present application, the terminal can obtain all user image data in the target area in real time through a camera, and then send it to the image processing unit. The image processing unit can analyze the user image data through face recognition technology and deep learning models to obtain image analysis results. The image analysis results may include user appearance features of each user extracted from the user image data, location information of each user, and user attribute features of each user. Among them, the user attribute features may include the user's age and gender information. The image processing unit may send the image analysis results to the comprehensive processing unit. The comprehensive processing unit may receive the location information of the target user sent by the sound processing unit. The location information may characterize the user's seat position in the cabin. For example, the location information may be the co-pilot seat, the left rear seat, the middle rear seat, etc.

[0089] The comprehensive processing unit can determine the target user corresponding to the location information in the image analysis result based on the location information sent by the sound processing unit, and then obtain the user attribute characteristics of the target user. The comprehensive processing unit can input the user attribute characteristics into the preference prediction model to obtain the preference characteristics of the target user. The preference characteristics are used to characterize the type and / or style of the virtual image that the user is predicted to be interested in, and can also include the type of special effects applied to the virtual image. The types of virtual images include animals, cartoons, characters, etc., and the special effects types can include makeup special effects, action special effects, expression special effects, etc. of the virtual image.

[0090] The integrated processing unit may send a virtual image acquisition request to the server, the virtual image acquisition request includes a preference feature, the server determines the corresponding target virtual image according to the preference feature, and then sends the data of the target virtual image to the integrated processing unit. The integrated processing unit receives the data of the target virtual image sent by the server.

[0091] In this solution, a target virtual image that matches the user's preferences can be obtained, so that the virtual image matches the user's preferences and improves the user experience. In addition, the data of the virtual image can be stored in the cloud or backend server, making the update and expansion of the virtual image more convenient and timely, with a wider range of options, and saving terminal resources.

[0092] Optionally, the user appearance features include at least facial features and clothing features; the voice assistant image is obtained by fusing the target user's appearance user features and the target virtual image, including: adjusting the facial image of the target virtual image based on the target user's facial features to obtain a facial image that matches the facial features; adjusting the body image of the target virtual image based on the target user's clothing features to obtain a body image that matches the clothing features; and generating the voice assistant image based on the facial image and the body image.

[0093] In an embodiment of the present application, the user appearance features include at least facial features and clothing features. The comprehensive processing unit can match and change the facial image of the target virtual image based on the facial features of the target user to obtain a facial image that matches the facial features. At the same time, the body image of the target virtual image can be matched and changed based on the clothing features of the target user to obtain a body image that matches the clothing features. Then, the terminal can generate a voice assistant image based on the facial image and the body image. In one example, the image information and color of the target user's top obtained by the image processing unit can be drawn onto the top of the target virtual image.

[0094] It can be understood that the user appearance characteristics may also include other characteristics such as accessory characteristics. Accordingly, the comprehensive processing unit can change, increase or reduce the accessories of the target virtual image based on the accessory characteristics of the target user, so that the target virtual image is consistent with the accessories of the target user.

[0095] In this solution, a voice assistant image that is similar to the user's appearance can be generated, making the voice assistant image changeable and different for each person, thereby improving the flexibility and fun of the voice assistant display and enhancing the user experience.

[0096] like Figure 3 As shown, the embodiment of the present application provides an example of a method for displaying a voice assistant image, comprising the following steps:

[0097] Step 301: The sound pickup component collects the sound data of the user in the car and sends it to the sound processing unit.

[0098] Step 302: The sound processing unit identifies the voice command contained in the user's sound data, and determines the location information of the target user according to the location information of the sound pickup component corresponding to the sound data, and sends it to the integrated processing unit.

[0099] Step 303: The OMS camera sends the user image data in the vehicle to the image processing unit.

[0100] Step 304 : the image processing unit extracts user information of each user in the vehicle based on the user image data, and sends the information to the integrated processing unit.

[0101] The user information includes the user appearance characteristics and user attribute characteristics of each user, and also includes the location information of each user. The user appearance characteristics at least include facial features and clothing features; the user attribute characteristics at least include gender and age.

[0102] Step 305: The comprehensive processing unit identifies the user appearance features and user attribute features of the target user from among the user appearance features according to the location information of the target user.

[0103] Step 306 : The comprehensive processing unit determines a target virtual image that matches the user's attribute characteristics from among the multiple candidate virtual images stored in advance.

[0104] Step 307, the comprehensive processing unit adjusts the facial image of the target virtual image based on the facial features of the target user to obtain a facial image that matches the facial features;

[0105] Step 308, the comprehensive processing unit adjusts the body image of the target avatar based on the clothing characteristics of the target user to obtain a body image that matches the clothing characteristics;

[0106] Step 309: The integrated processing unit generates a voice assistant image based on the facial image and the body image.

[0107] Step 310: the display unit displays the voice assistant image.

[0108] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiment of the present application also provides a device for displaying a voice assistant image for implementing the above-mentioned method for displaying a voice assistant image. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of the display device for one or more voice assistant images provided below can refer to the limitations of the method for displaying a voice assistant image above, and will not be repeated here.

[0110] In an exemplary embodiment, Figure 4 As shown, a display device for a voice assistant image is provided, including: an acquisition module 410 and a display module 420, wherein:

[0111] The acquisition module 410 is used to acquire the user appearance features of the target user who issued the voice command when the voice command is detected; the user appearance features are extracted based on the target image data of the target user;

[0112] The display module 420 is used to perform fusion processing based on the appearance and appearance user characteristics of the target user and the target virtual image to obtain a voice assistant image and display the voice assistant image.

[0113] In one embodiment, the device further comprises:

[0114] A first extraction module, used to obtain user image data in the target area, and extract user appearance features of each user in the target area based on the user image data;

[0115] The acquisition module 410 is specifically used for:

[0116] When a voice command is detected, the location information of the target user is determined;

[0117] According to the location information of the target user, the user appearance feature of the target user is identified from among the user appearance features included in the user image data.

[0118] In one embodiment, the acquisition module 410 is specifically configured to:

[0119] Collecting voice data of users in the target area through the sound pickup component;

[0120] Identify whether the user's voice data contains voice commands;

[0121] When the voice data of a target user among the users includes a voice command, the position information of the target user is determined based on the position information of a sound collecting member corresponding to the voice data.

[0122] In one embodiment, the device further comprises:

[0123] A second extraction module is used to extract user attribute features of a target user based on the target image data;

[0124] The first determination module is used to determine a target virtual image that matches the user's attribute characteristics from a plurality of pre-stored candidate virtual images.

[0125] In one embodiment, the device further comprises:

[0126] A third extraction module is used to extract user attribute features of a target user based on the target image data;

[0127] The second determination module is used to input the user attribute characteristics into the preference prediction model to obtain the preference characteristics of the target user;

[0128] The download module is used to download a target virtual image matching the preference characteristics of the target user from the server.

[0129] In one embodiment, the user appearance features include at least facial features and clothing features; the display module 420 is specifically used to:

[0130] Based on the facial features of the target user, the facial image of the target avatar is adjusted to obtain a facial image that matches the facial features;

[0131] Based on the clothing characteristics of the target user, the body image of the target avatar is adjusted to obtain a body image that matches the clothing characteristics;

[0132] Generate a voice assistant image based on facial image and body image.

[0133] Each module in the display device of the voice assistant image can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0134] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG5. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for displaying the image of a voice assistant is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0135] Those skilled in the art will understand that the structure shown in FIG. 5 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method for displaying a voice assistant image when executing the computer program.

[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a method for displaying a voice assistant image are implemented.

[0138] In one embodiment, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of a method for displaying a voice assistant image.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0140] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0141] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0142] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for displaying a voice assistant image, characterized in that: The method comprises: In the case of detecting a voice command, obtaining a user appearance feature of a target user who issues the voice command; the user appearance feature is extracted based on target image data of the target user; Based on the on-screen user characteristics of the target user and the target virtual image, a fusion process is performed to obtain a voice assistant image, and the voice assistant image is displayed.

2. The method according to claim 1, characterized in that The method further comprises: Acquire user image data in a target area, and extract user appearance features of each user in the target area based on the user image data; In the case where a voice command is detected, obtaining the user appearance characteristics of the target user who issued the voice command includes: In the case of detecting a voice command, determining the location information of the target user; According to the location information of the target user, the user appearance feature of the target user is identified from among the user appearance features included in the user image data.

3. The method according to claim 2, characterized in that The step of determining the location information of the target user when a voice command is detected includes: Collecting voice data of users in the target area by means of a sound pickup component; Identifying whether the user's voice data contains a voice command; When the voice data of a target user among the users includes a voice command, the position information of the target user is determined according to the position information of a sound pickup component corresponding to the voice data.

4. The method according to claim 1, characterized in that Before the fusion processing based on the on-off user characteristics of the target user and the target virtual image to obtain the voice assistant image, the method further includes: Based on the target image data, extracting user attribute features of the target user; Among a plurality of pre-stored candidate virtual images, a target virtual image matching the user attribute characteristics is determined.

5. The method according to claim 1, characterized in that Before the fusion processing based on the on-off user characteristics of the target user and the target virtual image to obtain the voice assistant image, the method further includes: Based on the target image data, extracting user attribute features of the target user; Inputting the user attribute characteristics into a preference prediction model to obtain the preference characteristics of the target user; A target virtual image matching the preference characteristics of the target user is downloaded from the server.

6. The method according to claim 1, characterized in that The user appearance features include at least facial features and clothing features; The step of fusing the target user's external and internal user characteristics with the target virtual image to obtain a voice assistant image includes: Based on the facial features of the target user, adjusting the facial image of the target virtual image to obtain a facial image matching the facial features; Based on the clothing features of the target user, adjusting the body image of the target avatar to obtain a body image that matches the clothing features; A voice assistant image is generated based on the facial image and the body image.

7. A display device for a voice assistant image, characterized in that: The device comprises: An acquisition module is used to acquire the user appearance features of the target user who issued the voice command when a voice command is detected; the user appearance features are extracted based on the target image data of the target user; The display module is used to perform fusion processing based on the on-screen user characteristics of the target user and the target virtual image to obtain a voice assistant image and display the voice assistant image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images