Apparatus and method for changing figure of digital assistant, and vehicle infotainment system

Abstract

To provide a control device for changing the figure of a digital assistant.SOLUTION: A control device provided herein comprises an image analysis module configured to determine figure features of at least one person on the basis of image data related to the at least one person, and a figure image generation module configured to change the figure of the digital assistant so that the changed figure of the digital assistant matches the figure features of the at least one person.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to in-vehicle infotainment systems, and more particularly to costume-changing for vehicle digital assistants. [Background technology]

[0002] Taking into account consumer interests and business needs, an increasing number of vehicles are now being equipped with in-vehicle digital assistants. As a type of service platform for automakers, digital assistants can encourage vehicles to interact naturally with users, including drivers and passengers, thereby establishing customer relationships and improving user experiences. Digital assistants are typically presented on the vehicle's display screen as virtual animated character avatars. To further increase user engagement, it is common for these digital assistants to appear in different costumes and outfits, further increasing user engagement and willingness to use the service.

[0003] Currently, dressing up animated characters is primarily performed based on three-dimensional scenes outside the vehicle, e.g., spatial and temporal fences. For example, when entering a specific area or at a specific time, such as a holiday, the animated character is dressed in appropriate clothing. However, such dressing up is primarily performed by selecting and coordinating outfits from an online store or an internal database in the vehicle. For example, according to a user's preferences or behavior, a clothing file with a matching costume is selected from a pre-established clothing database, and the clothing file is loaded using an animation generation tool or mode to achieve dressing up of the animated character. However, due to the limited number of clothing samples in the database or human artistic creativity, it is clear that a perfectly matching outfit may not be found in the database, or the resulting outfit may not be appealing to the user. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention proposes ways to make dressing up a digital assistant more in line with the image of a real person, increasing the fun of dressing up and making dressing up independent of existing clothing materials, for example, in an online store or an internal database in the car. [Means for solving the problem]

[0005] Thus, according to one aspect of the present invention, there is provided an apparatus for dressing up a digital assistant, the apparatus including: an image analysis module that determines image characteristics of at least one person based on image data relating to the at least one person; and an image generation module that dresses up the digital assistant so that an image of the dressed digital assistant matches the image characteristics of the at least one person.

[0006] According to another aspect of the present invention, there is provided a control device for dressing up a digital assistant, the control device including: a feature extraction module that extracts one or more environmental features of a vehicle's current environment; a language generation module configured to generate a clothing description language based on the environmental features; a file generation module configured to generate a clothing file based on the clothing description language; and an image generation module configured to read the clothing file for dressing up the digital assistant. According to this aspect, an artificial intelligence (AI) model can be automatically driven to generate art materials according to the content features of an in-vehicle scene or an application, thereby realizing a way to dress up the digital assistant on the spot. The image and coordination of the digital assistant avatar (Avatar) generated in real time can be matched to the environmental features of the vehicle at the time, making the dressing up of the digital assistant avatar (Avatar) more lifelike. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating the configuration of a digital assistant dress-up device according to an example of the present invention. [Figure 2] 10 is a diagram illustrating the configuration of a digital assistant dress-up device according to another example of the present invention. [Figure 3] 1 illustrates a configuration of a host system according to an example of the present invention. [Figure 4] 1 is a flowchart of a digital assistant dress-up process according to an example of the present invention. [Figure 5] 10 is a flowchart of a digital assistant dress-up process according to another example of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] Before describing any embodiment of the present invention in detail, it should be understood that the use of the present invention is not limited to the details of the architecture set forth in the following description or illustrated in the drawings. The present invention may have other embodiments and the real-time customization aspect of the in-car assistant according to the present invention may be practiced or implemented in various forms.

[0009] According to one embodiment of the present invention, an in-vehicle digital assistant avatar can mimic the costumes of people inside the vehicle, such as the driver or other passengers, without relying on existing databases or materials, which can also increase entertainment value. Typically, a vehicle cabin is equipped with an in-vehicle camera that captures images of the interior scene, including image frames or video streams of the driver and other passengers. A host system inside the vehicle communicates with the camera to capture these image frames or videos, hereinafter referred to as Img. According to one embodiment of the present invention, the host system includes a dress-up control device. As shown in FIG. 1, the dress-up control device 100 includes an image analysis module 101 and an image generation module 102. The image analysis module 101 receives image data Img from the camera and recognizes different relevant people therein, such as the driver or passengers, to obtain image data of marked people, hereinafter referred to as FigureImg. For example, an image of the driver (FigureImg1) and images of the other passengers (FigureImg2, FigureImg3, etc.) can be generated. For the recognized person image data FigureImg, the image analysis module 101 performs further analysis to determine the image features FigureFeature of the person. The image features here may be the costume features cF of the person, such as (1) whether or not the person is wearing a hat and its style (cF 11 ), color (cF 12 ), icon(cF 13 ) and (2) whether or not to wear a scarf or tie and its style (cF 21 ), color (cF 21 ), icon(cF 23) etc., showing the neck costume characteristics cF2, (3) style (cF 31 ), texture (cF 32 ), material (cF 33 ), color (cF 34 ), icon(cF 35 ) and (4) whether or not to wear glasses and the style of glasses (cF 41 ), color (cF 42 ), etc. It should be noted that the costume features cF obtained by the image analysis module 101 are not limited to the above examples, and may include some of the features, or may further include other features that reflect the person's costume features, but are not necessarily limited to the above specific examples. The image analysis module 101 can determine these costume features cF by employing any image analysis technique known in the art. For example, if a driver is driving wearing a baseball cap, sunglasses, and a black leather jacket, the analysis module 101 can analyze the driver's image features FigureImg1 to determine that the costume features include the following:

[0010] cF1:cF 11 ="baseball cap", cF 12 = "white", cF 13 = "None" cF3:cF 31 = "Jacket", cF 32 ="NULL", cF 33 ="NULL", cF 34 ="black", cF 35 = None (where "NULL" indicates that the image analysis module 101 is unable to determine relevant features from the image) cF4:cF 41 = "Sunglasses", cF 42 = dark color.

[0011] The image generation module 102 changes the appearance of the in-vehicle digital assistant avatar so that the image of the changed digital assistant avatar matches the image characteristics of the person being analyzed. For example, in the above example of a driver wearing sunglasses and a black leather jacket, the image generation module 102 generates a digital assistant avatar by dressing the avatar in a baseball cap, sunglasses, and a black top, accurately imitating the driver's appearance. It should be noted that the image of the digital assistant avatar according to an embodiment of the present invention is not static but can change dynamically. For example, if the driver removes his hat, sunglasses, and leather jacket and is wearing only a shirt, the image analysis module 101 can determine that the driver's image has changed by analyzing one or more consecutively received image frames or video streams related to the driver, and can therefore determine or update the changed image characteristics (FigureImg') of the driver. As a result, the image generation module 102 further changes the digital assistant avatar's clothes to reflect the change in the driver's image; for example, in this example, the avatar is also wearing only a shirt, which matches the driver's image characteristics.

[0012] For example, the image generation module 102 can be implemented by a conventional animation generation tool or digital human production tool such as MJ+SD, Unreal Engine, Blender / C4D / MAYA, SadTalker, etc. These tools can load clothing files in standard formats to perform avatar clothing operations. Of course, these tools are also typically used to create or modify other images of the avatar, such as changing appearance features. The clothing files loaded into these tools typically include standard feature elements, hereinafter referred to as sE, which are hereinafter defined as sE={sE1, sE2, sE3...sE m ,}, which define a plurality of different types of clothing, including style, texture, material, color, icon, etc. Therefore, as an example, as shown in FIG. 1, the dress-up control device 100 further includes a file generation module 103, which receives image feature data cF from the image analysis module 101 and prepares values ​​of each of the standard feature elements sE based on the received image feature data cF, for example, costume feature data cF3:cF 31 = "Jacket", cF 32 ="NULL", cF 33 ="NULL", cF 34 ="black", cF 35 For = None, the file generation module 103 determines the standard elements of the Avatar's clothing as follows:

[0013] sE 衣服 = sE1 = "Jacket", sE4 = "Black", sE5 = None. For texture and material features not provided by the image analysis module (i.e., cF 32 ="NULL", cF 33="NULL"), the file generation module 103 can set sE2="smooth" and sE3="leathery" based on factors such as the user's wearing characteristics. Here, the file generation module 103 can determine feature values ​​that are not provided in the image feature cF but are essential for standard feature elements based on factors such as any conventional habits. It should be noted here that the image analysis module 101 can determine the texture feature cF 32 For example, if it can be determined that the shirt the driver is wearing is a "checkered" pattern, the analysis module 101 may determine that cF 32 is set to "check", and accordingly, the file generation module 103 can set sE3="check". After determining all feature elements sE, the file generation module 103 can form a clothing file ClothFile according to a standard file format, for example, BVH, and provide the clothing file to the image generation module 102. Thus, the image generation module 102 can read the file ClothFile to change the avatar's clothes.

[0014] According to another embodiment of the present invention, the image feature FigureFeature may further include the appearance feature of a person. For example, the image analysis module 101 can analyze the image FigureImg1 of the driver to determine the appearance feature of the driver, such as hairstyle, face shape, etc., denoted as aF, where aF={aF1, aF2, ...}, where aF irepresents one element belonging to the appearance feature. As a result, the image generation module 102 can generate or change the appearance of the digital assistant avatar (Avatar) based on the appearance feature aF. For example, if the driver has removed his / her baseball cap and glasses and is wearing only a shirt, the image analysis module 101 can determine elements such as his / her hairstyle and face shape, and the digital assistant avatar (Avatar) generated by the image generation module 102 will also change accordingly to match the recognized driver's hairstyle, face shape, etc.

[0015] In the above example, the in-vehicle infotainment system may be configured such that the camera only focuses on the driver, i.e., creates an image of the digital avatar based only on the captured image of the driver. However, in another example of the present invention, the vehicle host system may instruct the control device 100 to dress up the digital assistant based on images of other passengers in the vehicle. To this end, the analysis module 101 analyzes the image Img transmitted from the camera to determine whether there are passengers elsewhere in the vehicle. For example, if the analysis module 101 detects that there is a passenger in the passenger seat, the analysis module 101 extracts an image of the passenger, hereinafter referred to as FigureImg2, from the image Img. Next, as described above, the analysis module 101 further analyzes FigureImg2 to determine image features FigureFeature2 of the passenger, including costume features and / or appearance features. Accordingly, the image generation module 102 generates a digital assistant avatar based on the passenger's image, so that the image of the assistant presented on the screen matches the passenger.

[0016] According to another embodiment of the present invention, to further enhance the enjoyment of the experience, the control device 100 can also selectively change the assistant avatar so that the dressed assistant matches the driver or passenger, respectively. For example, the principle of selective change may be to change the person currently speaking. According to this example, the analysis module 101 receives one or more images Img transmitted from the camera and determines who is currently speaking by performing image processing on the received one or more images. Here, the speaker can be determined by adopting image recognition technology known in the prior art, for example, by analyzing changes in the mouth shape of the same person in consecutive image frames, or by determining the mouth shape of a person in one image.

[0017] According to this embodiment, the image analysis module 101 receives image data Img from the camera and recognizes that in addition to the driver 1, there is also a passenger, e.g., passenger 2, in the current vehicle, and therefore extracts images of the driver 1 and passenger 2, hereinafter referred to as FigureImg1 and FigureImg2. For each recognized person, the image analysis module 101 further determines image features FigureFeature1 and FigureFeature2 of the driver 1 and passenger 2, such as determining their respective costume features and appearance features. The file generation module 103 can generate costume files ClothFile1 and ClothesFile2, respectively, based on the image features FigureFeature1 and FigureFeature2 of the driver 1 and passenger 2.

[0018] The image analysis module 101 also determines who is currently speaking based on the acquired image Img or image sequence. For example, if it determines that the driver is currently speaking, the image analysis module 101 instructs the image generation module 102 to load the driver's clothing file ClothFile1, and at the same time, adjusts the appearance of the assistant avatar (Avatar) to match the appearance characteristics of the driver. As a result, the digital assistant is presented as the image of the driver. Thereafter, if the image analysis module 101 determines from the received Img that passenger 2 is currently speaking, the image analysis module 101 instructs the image generation module 102 to load passenger 2's clothing file ClothFile2, and at the same time, adjusts the appearance of the assistant avatar (Avatar) to match the appearance characteristics of passenger 2. As a result, the digital assistant is presented as the image of passenger 2.

[0019] In another example of the present invention, the digital assistant avatar can also be adapted to match the image of a person outside the vehicle in real time. Typically, a passenger's mobile phone inside the vehicle can be connected to the vehicle infotainment system via a wired or wireless link. When a passenger makes a video call with a remote person outside the vehicle, the control device 100 can acquire an image (Img') of the person outside the vehicle and determine the image features (FigureFeature'') of the person. Based on this, the image generation module 102 can create a digital assistant that matches the FigureFeature''. When the video call is completed, the digital assistant can be reverted to a default avatar image, such as the driver's image. In the above-described embodiment, the digital assistant is adapted to match the image of a person inside the vehicle in real time. However, if the vehicle does not have an in-vehicle camera or is prohibited from capturing in-vehicle scenes, another example of the present invention can select a costume that matches the current environment or application program of the vehicle, thereby making the image of the adapted digital assistant avatar more suited to the current environment. Such a vehicle environment may be the state of use of the vehicle, such as the screen or image on the main control panel of a current in-vehicle infotainment system. In-vehicle infotainment systems typically have different control or application programs installed, which display different images during operation. For example, in a vehicle navigating, the vehicle may be driving along city streets, or when passing by a beach or lawn, the corresponding color of the sea or grassland will be displayed. At the same time, as time passes, the navigation screen also changes depending on whether it is day, evening, or twilight. The vehicle environment may also be an audio environment, such as music played by a vehicle multimedia player, such as a radio, or music selected by the user, or the voices of users talking, arguing, etc., inside the vehicle.Of course, the vehicle environment is not limited to the above examples, and may be any other factor that affects a person's mood or emotion, such as speed or weather.

[0020] FIG. 2 shows the configuration of a control device for dressing up a digital assistant avatar based on the vehicle environment according to another example of the present invention. Hereinafter, the graphical user interface (GUI) on the host system will be described as an example of the vehicle environment.

[0021] As shown, in addition to the file generation module 103 and the image generation module 102, the control device 200 further includes a feature extraction module 104 and a language generation module 105. The feature extraction module 104 extracts desktop features of the GUI of the host system, for example, capturing an image or image features of the GUI. As an example, if the current vehicle is in navigation mode, the captured image feature may be an image GUI mg of the navigation screen. Based on the captured image GUI mg, multiple feature elements F are determined. The feature elements F include, but are not limited to, time features F1, such as evening or a specific time of day; road features F2, such as an urban road, rural road, or highway currently being driven; color features F3 of the current screen; and driving destination F4. As an example, based on the current navigation screen, it is determined that the current time displayed on the screen is 5:35 PM. The dark yellow navigation background and the buildings on both sides of the road indicate that the user is currently on his way home from work. From this, the feature extraction module 104 can determine each feature element as F1=evening, F2=city road, F3=warm yellow, F4=after work by reading and analyzing the image features of the current GUI.

[0022] It should be noted that the extracted time feature may further include a date feature and a holiday attribute estimated from the date. For example, if the current date is May 1, it can be determined that today is Labor Day. Therefore, the feature extraction module 104 can combine the attributes of evening or a specific time, date, and holiday to form a final time feature F1, for example, F1 = evening + May 1 + holiday. Accordingly, the destination F4 is changed to "holiday." The above merely illustrates the extraction of four feature elements based on the image features of the GUI screen. Obviously, the present invention is not limited to this and may include more or fewer feature elements. Therefore, the number of feature elements is represented by N.

[0023] It should also be noted that some of the above feature elements F, for example, F1 (= evening), F2 (= city road), and F3 (= warm yellow), are visualized features displayed on the screen image GUImg. Therefore, these elements F can be determined from the screen image GUImg using image analysis techniques, but can also be determined based on data inside the host system. For example, it can be determined that it is evening based on the current system time, and the current road features can be determined based on map data. Here, any image analysis technique known in the prior art can be adopted to obtain the feature elements F. Thus, the feature extraction module 104 can obtain the feature elements F=[F1, F2, ..., F N ] is provided to the language generation module 105.

[0024] The language generation module 105 is configured to generate a clothing description language ClothDes based on the environmental features F, and in this example, can process the feature elements F of the extracted desktop image features to generate the clothing description language ClothDes by invoking a clothing description language model (CDLM) of a trained generative artificial intelligence (AI). In this example, before providing it to the language generation module 105, the feature extraction module 104 processes the feature elements [F1, F2, ..., FN ] can be processed as needed, for example, converted into a standard feature vector and provided as an input parameter to the clothing description language model CDLM. For ease of description, we still use the feature vector as [F1,F2,...,F N ] is expressed as

[0025] In this invention, the clothing description language ClothDes describes appropriate costume coordination advice for the current vehicle environment, and the clothing description language model CDLM is like a fashion consultant that integrates each feature element of the current environment to provide clothing description advice ClothDes. For example, for the above example, F1 = evening, F2 = city road, F3 = warm yellow, and F4 = after work, the clothing description language ClothDes may be "white-collar everyday clothes for commuting, with a warm yellow base."

[0026] It should be noted here that in the above example, the feature extraction module 104 divides the extracted image GUImg into each feature element [F1, F2, ..., F N ] and provided to the model CDLM, but in another example, the feature extraction module 104 can also provide the image features GUImg directly to the trained clothing description language model CDLM', which can process the image directly to form the clothing description language ClothDes.

[0027] According to this embodiment, the file generation module 103 receives the clothing description language ClothDes from the language generation module 105 and creates a clothing file ClothFile based on the clothing description language ClothDes. As described in the above example, the clothing file ClothFile may include multiple clothing elements sE conventionally used to define clothing, such as a style sE1, an icon or accessory sE2, a color scheme sE3, a material sE4, and a texture sE5, each of which represents a style element of the clothing. The style sE1 may include information such as suit, short sleeve, and denim, and may also include information such as collar type and sleeve type; the color scheme sE3 may include cool and warm colors (or specific colors) and shiny colors; the icon sE2 may be a pattern related to the environment, such as a shining moon at night; the material sE4 represents the material that makes up the clothing, i.e., the clothing fabric, such as natural fibers (cotton, linen, silk, wool, leather), synthetic fibers (nylon, polyester), and blends; and the texture sE5 represents the pattern on the clothing fabric, such as a checkered, striped, or solid pattern. The clothing file (ClothFile) referred to here can adopt a format commonly used in conventional technology, such as the BVH format. The generated clothing file (ClothFile) in a standard format includes clothing elements such as the overall style, shape, color, fabric, and accessories.

[0028] According to this embodiment of the present invention, the file generation module 103 processes the clothing description language ClothDes by calling a clothing resource generation model (CRGM) of trained generative artificial intelligence. Here, the file generation module 103 can preprocess the clothing description language ClothDes, for example, by adopting a method such as semantic segmentation to divide the clothing description language ClothDes into a predetermined number of vectors v1, v2, v3, .... Then, the model CRGM processes the vector v and generates clothing elements (sE1, sE2, sE3, ..., sE M), where M represents the number of elements for defining the outfit. Here, each element sE1, sE2, sE3, sE4, sE5, etc. output by the model CRGM can directly indicate the corresponding outfit element. For example, for the above example of ClothDes "a white-collar everyday outfit for work with a warm yellow base," the model CRGM outputs sE1 = casual two-piece, sE2 = NULL (no icon), sE3 = light blue, sE4 = cotton, and texture sE5 = polka dots. Thus, the file generation module 103 can create an outfit file ClothFile in a standard format, for example, BVH, based on the elements sE generated by the model CRGM, where the file contains the element information E.

[0029] The image generation module 102 reads the clothing file ClothFile generated by the file generation module 103 to change the clothes of the digital assistant avatar (Avatar), for example, changing the clothes of the digital assistant avatar (Avatar) from a style such as a suit for work to a casual costume for relaxing after work. Here, the image generation module 102 can change the clothes of the digital assistant avatar (Avatar) by executing a conventional application program and reading the clothing file ClothFile.

[0030] As can be seen from the above description, in the present invention, the clothes of the digital assistant avatar are generated in real time and are determined on the spot in real time by automatically driving generative artificial intelligence models such as CDLM and CRGM based on the current environment, without relying on any database or any existing clothing options. Therefore, the clothes can be adjusted appropriately according to significant changes in the environment, allowing the user to feel more considerate and experience better human-machine interaction.

[0031] According to an embodiment of the present invention, the clothing description language model (CDLM) and the file generation model (CRGM) can be generated by training an artificial intelligence (AI) neural network, which may be a convolutional neural network or another network constructed using a convolutional neural network, such as a multilayer perceptron (MLP). Here, the CDLM and CRGM can be determined using a conventional training method, for example, by training on a large number of samples collected in various different environments, such as image samples of different GUI screens under different conditions, such as different scenes and times, in-car audio samples such as music or broadcasts played by multimedia playback devices such as radios or other players, or passenger conversations. At the same time, for each sample, expert costume coordination advice, such as professional advice from an art and fashion consultant or a large number of user surveys, is used as the clothing description sample label (ClothDes_Label), to form a training dataset {DataSample, ClothDes_Label}, where DataSample represents environmental data acquired under various environments, such as image features and music clips. Thus, by learning {DataSample, ClothDes_Label} through data training, the model parameters of CDLM can be determined, and by learning {ClothDesSample, Cloth_Lable}, the model parameters of CRGM can be determined. Here, Cloth_Lable is the clothing coordination for each vehicle environment and can be manually specified during the sample collection process. Meanwhile, ClothDesSample represents a sample of clothing description language, which can be obtained from the training results of the model CDLM or from other methods such as manually generated language samples.In the sample training process, the CDLM and CRGM models can be trained by performing sample mixing on samples from various different environments, thereby achieving model versatility. Additionally, in the sample training process, the CDLM and CRGM models can be trained separately to make it easier to observe and evaluate CDLM's clothing description language, ClothDes. Naturally, a combined training method for the CDLM and CRGM models can also be adopted to reduce evaluation costs.

[0032] According to another embodiment of the present invention, during the training of the CDLM and CRGM models, image elements F and clothing elements sE can be weighted based on their importance so that the clothing worn by the assistant avatar can be matched with factors that have a significant influence. For example, in the above example, a pre-set weighting strategy can be used to assign higher weights to city roads F2 and colors F3, while appropriately reducing the weights of time F1, etc. Thus, the clothing description language and clothing file obtained based on the CDLM and CRGM models can better reflect the factors that have been considered.

[0033] According to the present invention, the trained CDLM and CRGM models can be used in various environments. For example, in another example of the present invention, the vehicle environment will be described as in-car music. Since a user's favorite music can usually reflect their mood or emotion, if the digital assistant avatar's outfit matches the music, the user can be provided with a better riding experience and interactive experience. Therefore, according to an embodiment of the present invention, real-time dress-up of the digital assistant avatar can be realized based on the music being played in the vehicle. For example, assume that a user launches a music app on the in-car host to play online music, or that the user's mobile phone is connected to the vehicle's host system via communication technology such as in-car Bluetooth®, and the user is playing music, e.g., rock music, on the mobile phone.

[0034] The feature extraction module 104 is configured to extract music style features, such as genre F1 (e.g., folk, pop, rock, rap, electronic music, ACG, classical, jazz), rhythm F1, scale F2, timbre F3, theme F4, etc., as features of the currently playing music. As an example, if a user is currently listening to "Jasmine," the feature extraction module 104 can extract feature elements of "Jasmine" through a music app and determine N elements, such as genre F1-"Folk Song." Before providing the feature elements to the language generation module 105, the feature extraction module 104 extracts N elements [F1, F2, ..., F] from the feature elements [F1, F2, ..., F]. N ] is converted into a standard feature vector and provided to the language generation module 105 for processing as input parameters for the large-scale model.

[0035] The language generation module 105 generates these musical style elements [F1, F2, ..., F N] to generate a clothing description language ClothDes. Similarly, the language generation module 105 calls the clothing description model CDLM to generate a clothing description language ClothDes by processing the extracted music feature elements F. As described above, the clothing description model CDLM is an AI model obtained by pre-training in different types of environments. As a result, after processing the currently playing song "Jasmine Flower" through the clothing description model CDLM, it is possible to generate a ClothDes such as "an outfit with a jasmine dress based on white."

[0036] As described above, the file generation module 103 preprocesses the clothing description language ClothDes, for example, by using semantic segmentation to divide the clothing description language ClothDes into a predetermined number of vectors v1, v2, v3, .... Then, it invokes the model CRGM to process the clothing description language ClothDes to generate a clothing file ClothFile. As described above, the file includes clothing elements such as style or style sE1, icon or accessory sE2, color scheme sE3, material sE4, and texture sE5. Still taking the song "Jasmine" as an example, the style sE1 = dress, accessory sE2 = bow, color sE3 = white, material sE4 = silk, and texture sE5 = solid color. Thus, the file generation module 103 can generate a clothing file ClothFile in, for example, a BVH format based on the elements sE generated by the model. As a result, the image generation module 102 reads the generated clothing file ClothFile and realizes the changing of the digital assistant avatar.

[0037] Similarly, in the case of a music environment, in the process of training the CDLM and CRGM models, weights are selected according to the importance of the elements, so that the clothes the assistant wears match the influencing factors with high importance. Here, according to a preset weight allocation strategy, for example, genre F1 and timbre F3 can be assigned higher weights, while the weights of other elements can be appropriately reduced.

[0038] It should be noted that in the above example, the GUI desktop background and music are used as examples to describe the embodiment of dressing up an avatar, but the present invention is not limited thereto, and dressing up can also be performed by comprehensively considering different environments, for example, by considering both the GUI desktop background and music. Therefore, according to one embodiment of the present invention, the feature extraction module 104 extracts each feature element, for example, image elements (F1, F2, ...F) based on the GUI desktop, based on the GUI desktop and music background, respectively. N ), and for ease of description, we use (F1',F2',...F N Then, the feature extraction module 104 extracts the music style elements extracted based on the music background, which are denoted as F1'+F1→F1'', F2'+F2→F2'', F3'+F3→F3'', ...F N '+F N →F N Then, we can combine elements of the two types: F1, F2, … F NThe image data is then subjected to vector transformation and provided to the file generation module 103 for further processing. After processing by the models CDLM and CRGM, a clothing file ClothFile is generated that comprehensively takes into account environmental factors, and the image generation module 102 then dresses up the avatar. It can be seen that the present invention can realize real-time generation of avatar outfits by automatically driving large-scale models to generate art materials according to the content characteristics of the scene and application, thereby reducing the manual design process.

[0039] In the above example of the present invention, the control device 100 is implemented in the form of modules. However, it should be noted that each module may be a hardware circuit, firmware, or a program module implemented by a software program, and the module's functions are implemented by a controller or other processor in the vehicle's host system. Therefore, according to another example of the present invention, as shown in FIG. 3, such a host system may include a graphical user interface (GUI) and / or a multimedia player, a speaker, or a microphone, a memory storing a computer-readable program, and at least one controller or processor, and the memory may also store models CDLM and CRGM. Hereinafter, referring to FIG. 4, still taking the GUI desktop as an example, the flow of customizing the digital assistant executed by the host system will be described.

[0040] In step 401, a feature of the desktop of the GUI is captured, for example, if the vehicle is currently in navigation mode, the captured image feature may be an image GUImg of the navigation screen on the desktop of the GUI.

[0041] In step 403, N feature elements F are determined based on the captured image GUImg, including a time feature F1 such as evening or a specific time of day, a road feature F2 such as an urban road feature, rural road, or highway currently being driven, a color feature F3 of the current screen, and a driving destination F4. If the current time displayed on the current screen is 17:35, the dark yellow navigation background and the buildings on both sides of the road the user is passing indicate that the user is currently on his way home from work. Therefore, each feature element can be determined as F1 = evening, F2 = urban road, F3 = warm yellow, F4 = home from work, etc.

[0042] In step 405, the environmental features [F1, F2, ..., F N ], we generate a clothing description language ClothDes, and in this example, we call the pre-trained clothing description language model CDLM to extract desktop image features [F1, F2, ..., F N ] to generate a clothing description language ClothDes. For example, in this example, the clothing description language ClothDes generated by the clothing description model CDLM may be "a white-collar everyday outfit for commuting, with a warm yellow base."

[0043] In step 407, a clothing file ClothFile is created based on the clothing description language ClothDes, where the clothing file ClothFile may include multiple clothing elements sE conventionally used to define clothing, such as a style or format sE1 representing the shape elements of clothing, an icon or accessory sE2, a color scheme sE3, a material sE4, and a texture sE5. According to an example of the present invention, the clothing description language ClothDes is preprocessed, for example, by adopting a method such as semantic segmentation to divide the clothing description language ClothDes into a predetermined number of multiple vectors v1, v2, v3, .... Then, the vector v is processed by invoking a trained artificial intelligence-based clothing generation model CRGM to generate clothing elements (sE1, sE2, sE3, ..., sE5) representing the clothing. M) and so on. For example, for the above example of "white-collar everyday clothes for commuting with a warm yellow base," the model AIGC outputs sE1 = casual pants, sE2 = NULL (no icon), sE3 = dark blue, sE4 = fur (determining that it is currently winter based on the time), and texture sE5 = plain pattern. Then, based on the elements sE generated by the model CRGM, a clothing file ClothFile is created in a standard format, for example, the BVH format, where the file contains the element information E.

[0044] In step 409, the clothing file ClothFile is read to change the clothes of the digital assistant avatar (Avatar), for example, changing the clothes of the digital assistant avatar (Avatar) from a style such as a suit and tie for work to a casual costume for relaxing after work. Here, the image generation module 102 can change the clothes of the digital assistant avatar (Avatar) by executing a conventional application program and reading the clothing file ClothFile.

[0045] FIG. 5 shows a control flow for changing the appearance of a digital assistant performed by an in-vehicle infotainment system according to another example of the present invention. First, in step 501, it is determined whether an image of the in-vehicle scene can be acquired. For example, this can be done by the in-vehicle infotainment system by determining whether an in-vehicle camera is installed and whether the camera is turned on. If an in-vehicle camera is not installed or the camera is not turned on, the control flow switches to the control flow shown in FIG. 4, thereby realizing changing the appearance of the assistant avatar (Avatar) based on the vehicle environment. In step 501, if it is determined that an image Img of the in-vehicle scene can be acquired, the process proceeds to step 503.

[0046] In step 503, based on the received image data Img, different relevant persons therein, such as the driver or passengers, can be recognized to obtain image data of the marked persons, hereinafter referred to as FigureImg, and generate, for example, an image of the driver FigureImg1.

[0047] In step 505, the image FigureImg1 of the driver is analyzed to determine its image features FigureFeature, including the driver's costume features cf, and optional appearance features aF.

[0048] Next, in step 507, the digital assistant avatar is re-dressed so that the image of the re-dressed digital assistant avatar matches the image characteristics of the driver. According to different implementations of the present invention, the re-dressing of the digital assistant by a person can be dynamic, and analysis of image frames or video streams of a person, such as the driver in this example, can determine that the person's image characteristics have changed, so that when the digital assistant is re-dressed, its image reflects such changes in the person's image characteristics. For example, if the driver currently wears a baseball cap, sunglasses, and a black leather jacket, when generating the digital assistant avatar, the avatar can be similarly dressed in a baseball cap, sunglasses, and a black top to accurately mimic the driver's appearance. In response, if the driver removes their hat, sunglasses, and leather jacket and is wearing only a shirt, the digital assistant avatar will be dressed to match the image characteristics of wearing only a shirt, and the avatar's appearance will be changed to a crew cut and square face that suits the driver.

[0049] In another example of the present invention, the in-vehicle infotainment system can dress up the digital assistant based on images of all passengers in the vehicle. To this end, in step 503, the image Img transmitted from the camera is analyzed to identify passengers in all locations in the vehicle, including the driver. If a passenger is detected, for example, in the passenger seat, an image of the passenger, hereinafter referred to as FigureImg2, is also extracted from the image Img. Next, in step 505, image features FigureFeature2 of the passenger, including costume features and / or appearance features, are determined by analyzing FigureImg2.

[0050] Next, in step 507, it is determined who is currently speaking based on the received image or images. Here, the speaker can be determined using image recognition technology known in the art. For example, the speaker can be determined by analyzing changes in the mouth shape of the same person in consecutive image frames, or by determining the mouth shape of a person in a single image. For example, if it is determined that the driver is currently speaking, the image of the digital assistant avatar (Avatar) is matched to the image feature FigureFeature1 of the driver. On the other hand, if it is determined that the person currently speaking is a passenger, the image of the digital assistant avatar (Avatar) is matched to the image feature FigureFeature2 of the passenger.

[0051] While the above describes different embodiments of the present invention using specific examples, those skilled in the art will further recognize that the various illustrative logic modules and method steps described with reference to the contents disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. For example, a control device for changing the appearance of a digital assistant avatar according to the present invention may be implemented as a processor or main controller and memory, with each module stored in the memory as a computer program, and the processor can execute these modules to implement the method of the present invention. Another embodiment of the present invention provides a machine-readable medium having machine-readable instructions stored therein, which, when executed by a processor, cause the processor to perform any of the methods described herein. These embodiments also fall within the scope of the present invention. [Explanation of symbols]

[0052] 100 control device 101 Image Analysis Module 102 Image Generation Module 103 File Generation Module 104 Feature Extraction Module 105 Language Generation Module 200 Control Equipment 401 Steps 403 Step 405 Steps 407 Step 409 Steps 501 steps 503 steps 505 steps 507 steps

Claims

1. A control device for dressing up a digital assistant, an image analysis module that determines image characteristics of at least one person based on image data relating to the at least one person; An image generation module that dresses the digital assistant so that the image of the dressed digital assistant matches the image characteristics of the at least one person; Control equipment including.

2. the image data comprises a video stream or one or more image frames collected at different times; The image analysis module further determines changes in image characteristics of the at least one person based on image frames or video streams relating to the at least one person; The control device of claim 1 , wherein the image generation module dresses the digital assistant so that an image of the dressed digital assistant reflects changes in image characteristics of the at least one person.

3. the image features include at least one of costume features and person appearance features; The control device further includes a file generation module configured to generate a first clothing file based on the costume characteristics, the first clothing file including a plurality of elements defining an outfit, the elements including at least one of a style, a texture, a material, a color, and an icon; The control device according to claim 1 or 2, wherein the image generation module reads the first clothing file for dressing up the digital assistant.

4. The control device of claim 1 or 2, wherein the image analysis module receives the image data captured by an in-vehicle camera, and the at least one person includes one or more of a driver and a passenger.

5. The image analysis module further recognizes a person currently speaking based on one or more of the image data, and outputs the image characteristics of the person currently speaking; The control device of claim 3, wherein the image generation module changes the appearance of the digital assistant so that the image of the dressed digital assistant matches the image characteristics of the person currently speaking.

6. an environment detection module that extracts one or more environmental features of the vehicle's current environment; a language generation module configured to generate an outfit description language based on the environmental features; the file generation module is further configured to generate a second clothing file based on the clothing description language; The control device according to claim 3 , wherein the image generation module is configured to selectively read the first clothing file and the second clothing file for dressing the digital assistant.

7. The control device of claim 6 , wherein the current environment includes a current host screen of a vehicle host, and the environmental features include image features extracted from the host screen of the vehicle.

8. 8. A control device according to claim 6 or 7, wherein the current environment comprises background music being played in the vehicle, and the environment characteristics comprise music style characteristics.

9. The control device of claim 8 , wherein the language generation module generates the clothing description language by processing the environmental features using a trained clothing description language model, the clothing description language describing clothing features that match the current environment.

10. The control device of claim 9 , wherein the clothing description language model assigns different weights to the environmental features when processing the environmental features.

11. 10. The control device of claim 9, wherein the file generation module generates the second clothing file by processing the clothing description language using a trained clothing generation model, and the second clothing file includes a plurality of elements defining clothing, including at least one of style, texture, material, color, and icon.

12. 1. A method for dressing up a digital assistant, comprising: determining image characteristics of the at least one person based on image data relating to the at least one person; Dressing up the digital assistant so that the image of the dressed digital assistant matches the image characteristics of the at least one person; A method comprising:

13. The image data comprises a video stream or one or more image frames collected at different times, and the method comprises: determining changes in image characteristics of the at least one person based on image frames or a video stream relating to the at least one person; 13. The method of claim 12, further comprising: dressing up the digital assistant so that an image of the dressed digital assistant reflects changes in image characteristics of the at least one person.

14. the image features include at least one of costume features and person appearance features; 14. The method of claim 12 or 13, wherein the method includes receiving the image data captured by an in-vehicle camera, and the at least one person includes one or more of a driver and a passenger.

15. A dress-up method for a digital assistant in an in-vehicle infotainment system, comprising: extracting one or more environmental features of the vehicle's current environment; generating a clothing description language based on the environmental features; generating a clothes file based on the clothes description language; Reading the clothing file to dress up the digital assistant presented in the graphical user interface; Dressing methods including.

16. The current environment is: the graphical user interface, and the environmental features include image features extracted from the graphical user interface; and / or 16. The method of claim 15, wherein the environmental features include music being played in a vehicle, and the environmental features include music style features.

17. A graphical user interface; A controller configured to execute the method according to any one of claims 12 to 16 in order to realize the customization of the digital assistant presented in the graphical user interface; In-vehicle infotainment systems, including:

18. A program product including machine-readable instructions, the instructions, when executed by one or more controllers, implementing the dress-up method of any one of claims 12 to 16.

19. A medium having machine-readable instructions stored thereon, the instructions, when executed by one or more controllers, implementing the dress-up method of any one of claims 12 to 16.

20. A control device for dressing up a digital assistant, an environment detection module that extracts one or more environmental features of the vehicle's current environment; a language generation module configured to generate an outfit description language based on the environmental features; a file generation module configured to generate a clothing file based on the clothing description language; An image generation module configured to read the clothing file for dressing the digital assistant; Control equipment including.

Images