Avatar generation based on vehicle sensors

Abstract

Based on vehicle sensor data, occupant-based digital avatars are rendered for use inside the vehicle. [Solution] The systems, methods, and other embodiments described herein relate to generating in-vehicle digital avatars of vehicle occupants based on vehicle sensor data. In one embodiment, the method includes extracting the physical characteristics of a vehicle occupant from vehicle capture images of the occupant and rendering a basic digital avatar of the occupant based on the extracted physical characteristics of the occupant. The method also includes extracting the characteristics of objects in the external environment from vehicle capture images of the objects. The method also includes rendering a presentation digital avatar similar to the occupant by transferring the characteristics of the objects onto the basic digital avatar of the occupant and animating the presentation digital avatar on a display device in the vehicle.

Description

Technical Field

[0001] The subject matter described herein generally relates to occupant-based digital avatars within a vehicle, and more specifically to rendering occupant-based digital avatars within a vehicle based on vehicle sensor data.

Background Art

[0002] A Human Machine Interface (HMI) is a vehicle component for a user to interact with the vehicle. Historically, occupants have interacted with vehicle systems via knobs, dials, switches and their equivalents. For example, to change the temperature of the heating system, an occupant moves a slider. To change radio stations, an occupant turns a dial. To activate a seat heater, a user presses a button on the dashboard. As time goes by, HMI has advanced technically and some of these functions have come to be controlled via touch sensor-based display devices. For example, through touch operations, an occupant can access a menu and change various settings of the heating system, including temperature, zone control, and the intensity at which the system fan operates. Through the HMI, an occupant can control additional, more recently developed systems. For example, a user can interact with a navigation application that provides navigation assistance and can also interact with a communication application through which a user can make and receive calls.

Summary of the Invention

[0003] In one embodiment, exemplary systems and methods relate to aspects of improving a human machine interface by generating a virtual assistant that is 1) visually similar to an occupant of a vehicle and 2) customized based on vehicle capture information about the surrounding environment of the vehicle.

[0004] In one embodiment, an avatar rendering system for generating avatars based on vehicle sensor data is disclosed. The avatar rendering system includes one or more processors and a memory communicably coupled to the one or more processors. The memory stores instructions, which, when executed by one or more processors, cause one or more processors to extract the physical characteristics of a vehicle occupant from vehicle-captured images of the occupant, and render a basic digital avatar of the occupant based on the extracted physical characteristics of the occupant. The memory also stores instructions, when executed by one or more processors, to cause one or more processors to extract the characteristics of objects in the external environment of the vehicle from vehicle-captured images of the objects. The memory also stores instructions, when executed by a processor, to render a presentation digital avatar similar to the occupant by transferring the characteristics of the objects onto the basic digital avatar of the occupant, and to animate the presentation digital avatar on a display device in the vehicle.

[0005] In one embodiment, a non-temporary computer-readable medium for generating avatars based on vehicle sensor data is disclosed, which, when executed by one or more processors, includes instructions causing one or more processors to perform one or more functions. The instructions include instructions to extract the physical characteristics of a vehicle occupant from vehicle capture images of the occupant, and to render a basic digital avatar of the occupant based on the extracted physical characteristics of the occupant. The instructions also store, when executed by one or more processors, instructions to cause one or more processors to extract the characteristics of objects in the external environment of the vehicle from vehicle capture images of the objects. The instructions also store, when executed by a processor, instructions to render a presentation digital avatar similar to the occupant by transferring the characteristics of the objects onto the basic digital avatar of the occupant, and to animate the presentation digital avatar on a display device of the vehicle.

[0006] In one embodiment, a method for generating a digital avatar based on vehicle sensor data is disclosed. In one embodiment, the method includes extracting the physical characteristics of a vehicle occupant from vehicle capture images of the occupant, and rendering a basic digital avatar of the occupant based on the extracted physical characteristics of the occupant. The method also includes extracting the characteristics of objects in the vehicle's external environment from vehicle capture images of the objects. The method also includes rendering a presentation digital avatar similar to the occupant by transferring the characteristics of the objects onto the basic digital avatar of the occupant, and animating the presentation digital avatar on the vehicle's display device. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 shows one embodiment of a vehicle in which the systems and methods disclosed herein are implemented. [Figure 2] Figure 2 shows one embodiment of an avatar rendering system related to the generation of a digital avatar based on vehicle sensor data, according to an embodiment disclosed herein. [Figure 3] Figure 3 shows one embodiment of the avatar rendering system of Figure 2 in a cloud computing environment, according to an embodiment disclosed herein. [Figure 4] Figure 4 shows a flowchart of one embodiment of a method related to the generation of a digital avatar based on vehicle sensor data, according to an embodiment disclosed herein. [Figure 5A] Figure 5A shows an example of generating a basic digital avatar according to an embodiment disclosed herein. [Figure 5B] Figure 5B shows an example of generating a basic digital avatar according to an embodiment disclosed herein. [Figure 6] Figure 6 shows a flowchart of one embodiment of a method related to generating a presentation digital avatar by transferring an object onto a basic digital avatar, according to an embodiment disclosed herein. [Figure 7] Figure 7 shows an example of generating a presentation digital avatar by transferring an object onto a basic digital avatar, according to an embodiment disclosed herein. [Figure 8] Figure 8 shows a flowchart of one embodiment of a method related to generating a presentation digital avatar by transferring an object theme onto a basic digital avatar, according to an embodiment disclosed herein. [Figure 9] Figure 9 shows an example of generating a presentation digital avatar by transferring an object theme onto a basic digital avatar, according to an embodiment disclosed herein. [Figure 10] Figure 10 shows one embodiment of a neural network-based avatar rendering system that generates a digital avatar for presentation based on vehicle sensor data, according to an embodiment disclosed herein. [Modes for carrying out the invention]

[0008] The accompanying drawings are incorporated herein and constitute part of this specification and illustrate various systems, methods, and other embodiments of this disclosure. It will be understood that element boundaries shown in the drawings (e.g., boxes, groups of boxes, or other shapes) represent one embodiment of the boundary. In some embodiments, one element may be designed as multiple elements, or multiple elements may be designed as a single element. In some embodiments, an element shown as an internal component of another element may be implemented as an external component, and an element shown as an external component of another element may be implemented as an internal component. Furthermore, elements may not be drawn to exact scale.

[0009] Systems, methods, and other embodiments relating to improving vehicle human-machine interfaces (HMIs) by rendering occupant-based avatars customized based on perceptual information collected by the vehicle are disclosed herein. As mentioned above, vehicle HMIs are becoming increasingly sophisticated. For example, some HMIs are touch-sensitive, and user commands are received via touch-sensitive infotainment displays. Furthermore, HMIs also accept other command methods. For example, a vehicle may include cameras to detect the occupant's physical gestures. As a result, the occupant controls the vehicle through gesture commands. In yet another example, a vehicle may include microphones to capture voice signals. Captured voice signals can be processed by vehicle systems and used to control other vehicle systems.

[0010] Some vehicles include a virtual assistant within the vehicle. For example, a cartoon character may be displayed on the infotainment display, providing guidance, instructions, and requested information to the occupants. For instance, in response to an occupant's request to "turn off the heater," the cartoon character might respond, "I have received your request, the heater has been turned off." This adds an engaging and fun element to the HMI. In this way, the cartoon character acts as a virtual assistant to the vehicle's occupants, performing certain tasks and / or providing safety information and / or requested information about the vehicle and its surroundings. While we have specifically mentioned the operation of certain virtual assistants, virtual assistants in vehicles can perform any task and operation.

[0011] However, these virtual assistants are generic and not user-specific. That is, this type of virtual assistant cannot be adapted to the specific occupants of a vehicle or to the current motif and style. Given their static and generic nature, the information provided by these virtual assistants may be downplayed or ignored. Therefore, the information and usefulness provided by these systems may not be properly considered by the vehicle's occupants. In one case, if safety information is downplayed or ignored, this could increase the potential risks to the occupants.

[0012] Furthermore, these avatars are not generated by taking into account the vast amount of information available from vehicle sensors. That is, vehicles include sensors capable of capturing a wealth of information from their surrounding environment. This information is used for a variety of purposes, including object detection, lane keeping assist, lane change warning, and other advanced driver assistance systems, but the collected perceptual information can also be used to generate engaging, interactive, and customizable avatars.

[0013] Specifically, the avatar rendering system described herein transfers elements and / or themes detected by vehicle environment sensors to occupant-based avatars within the vehicle. That is, the system uses information collected from in-vehicle sensors, such as images of one or more vehicle occupants, to generate avatars that resemble one of the occupants.

[0014] The system also collects information from external vehicle sensors, such as outward-facing cameras that capture images of the environment and individuals / objects within that environment. After generating a base digital avatar similar to the occupant, the system uses the information collected from the external sensors to modify the base digital avatar based on thematic and / or physical elements from individuals / objects in the environment. For example, the occupant instructs the system to modify the base digital avatar to incorporate elements of a Western movie poster that the occupant saw through the vehicle's windshield. Specifically, the characters in the movie poster might be wearing Western clothing, including a cowboy hat. Based on this request, the vehicle sensors capture an image of the movie poster, identify the poster's key visual features, and apply these visual features to the base digital avatar. In the specific example above, the occupant-based avatar would be modified to have Western clothing and a cowboy hat.

[0015] Thematic avatar generation involves not only transcribing visual thematic elements but also generating audio themes and animating avatars based on the identified themes. For example, a character in a Western film exhibits certain physical behaviors / characteristics and speaks in a specific manner (e.g., specific phrasing, accent, and vocabulary). In contrast, a character in a Victorian-era film exhibits other physical behaviors / characteristics and speaks in a different manner (e.g., different phrasing, accent, and vocabulary). In either of these examples, the avatar rendering system identifies these elements that align with the visual theme of the object and applies these elements (e.g., auditory, behavior, and other elements) to the crew-based generated avatar.

[0016] As another example, an occupant might see a pedestrian wearing sunglasses and an orange shirt and want to know what they (i.e., the vehicle occupant) would look like wearing the same sunglasses and shirt. In this example, the occupant could instruct the avatar rendering system to modify the avatar in the vehicle to include the sunglasses of the orange-shirted pedestrian captured by the environmental sensors. Upon receiving this instruction, the system modifies the avatar to include the sunglasses and shirt of the nearby pedestrian.

[0017] In this example, the avatar rendering system can incorporate a neural network or other machine learning system. Generally, a neural network is a computer architecture inspired by the structure of the human brain, having a network of interconnected nodes. Each node processes input data and uses mathematical operations to produce an output. As depicted in Figure 10, the nodes are arranged in layers, specifically in an input layer, a number of hidden layers, and an output layer. The input layer receives images of occupants from vehicle capture images, images of objects with elements or themes to be applied to the avatar, and a corpus of information that helps identify features and / or themes. Nodes in the hidden layers perform calculations and extract patterns from the input nodes. Nodes in the output layer generate occupant-based avatars incorporating features and / or elements of objects captured by vehicle sensors. Such a system is particularly well-suited to handling large and complex datasets, including content available via the internet.

[0018] In the context of this specification, generative neural networks can create avatars not by replicating existing content, but by creating new digital content based on patterns identified in large datasets, such as curated datasets or corpora of digital content available from unsupervised datasets like the internet. That is, generative neural network-based avatar rendering systems are trained on large datasets, which are specific curated datasets or unsupervised datasets like the internet. During training, the neural network-based avatar rendering system learns patterns in the dataset and uses such patterns to generate new outputs (e.g., avatars) in response to inputs (e.g., vehicle capture images and requests from occupants to apply specific themes).

[0019] Therefore, the avatar is generated based on information collected from in-vehicle sensors, and may have a theme for creating a highly entertaining and immersive experience. This can enhance the functionality of the generation system by introducing a new type of sensor information as the basis of the avatar, and the avatar can be generated using information on the road collected during movement. The system is improved in that it enhances the technical function by enabling the user to modify the avatar in the vehicle using elements and / or themes detected by the vehicle environment sensors. For example, the avatar in the vehicle can be modified to include the clothes worn by a pedestrian detected by the vehicle environment sensors while the vehicle is in motion. Therefore, the avatar rendering system described in this specification explains the improvement by using a new input form (for example, external vehicle sensor data) to generate an attractive in-vehicle virtual assistant. While an advanced driver assistance system uses vehicle sensors to provide guidance to the driver, this system uses vehicle sensors to generate an in-vehicle avatar. For this reason, this specification describes a system that integrates vehicle sensors and a generation neural network to generate a specific, customized, relevant digital assistant in the vehicle.

[0020] Since the avatar is unique to the occupant and customized by the occupant, the generated avatar can interact with the occupant in a more attractive way unique to the occupant, thereby increasing the possibility of recognition and interaction with the virtual assistant. Furthermore, the current system is improved in that it can customize the avatar in real time during driving.

[0021] Referring to the drawings, FIG. 1 is an example of a vehicle 100. As used herein, a "vehicle" is any form of transportation powered by a motor or other power source. In one or more embodiments, the vehicle 100 is an automobile. Although the description herein is directed to configurations related to automobiles, it will be understood that the embodiments are not limited to automobiles. In some embodiments, for example, the vehicle 100 is in the form of a robotic device or transportation vehicle that includes sensors for sensing aspects of the surrounding environment and thus benefits from the functions discussed in the specification related to the generation of an avatar based on sensor data collected by the vehicle.

[0022] The vehicle 100 includes various elements. It will be understood that in various embodiments, the vehicle 100 need not have all of the elements shown in FIG. 1. The vehicle 100 can have various combinations of the various elements shown in FIG. 1. Further, the vehicle 100 can have additional elements with respect to the elements shown in FIG. 1. In some configurations, the vehicle 100 may be implemented without using one or more of the elements shown in FIG. 1. Although the various elements are shown as being disposed within the vehicle 100 of FIG. 1, it will be understood that one or more of these elements can be disposed outside of the vehicle 100. Further, the elements shown may be physically located at a long distance apart. For example, as contemplated, one or more components of the disclosed system may be implemented within the vehicle, while additional components of the system are implemented in a cloud computing environment remote from the vehicle 100 or within another system.

[0023] Some of the possible elements of vehicle 100 are shown in Figure 1 and described in conjunction with the following figures. However, for the sake of brevity of explanation, the description of many of the elements in Figure 1 is provided after the discussion of Figures 2 to 10. In addition, it will be understood that, in order to make the figures concise and clear, reference numerals are repeated between different drawings where necessary to indicate corresponding or similar elements. In addition, this discussion outlines numerous specific details in order to provide a full understanding of the embodiments described herein. However, those skilled in the art will understand that the embodiments described herein may be carried out using various combinations of these elements. In any case, vehicle 100 includes an avatar rendering system 126, which is implemented to perform methods and other functions as disclosed herein relating to improving the human-machine interface by incorporating two-way communication with an occupant-based avatar that incorporates elements captured by the driver of the vehicle while it is traveling along the roadside.

[0024] As will be discussed in more detail later, the avatar rendering system 126 is implemented in various embodiments as a partial implementation within the vehicle 100 and as a cloud-based service. For example, in one approach, functionality related to at least one module of the avatar rendering system 126 is implemented within the vehicle 100, and further functionality is implemented within a cloud-based computing system. Thus, the avatar rendering system 126 includes a local instance in the vehicle 100 and a remote instance operating in a cloud-based environment.

[0025] Furthermore, the avatar rendering system 126, when provided within the vehicle 100, functions in cooperation with the communication system 127. In one embodiment, the communication system 127 communicates according to one or more communication standards. For example, the communication system 127 may include multiple various antennas / transmitters and / or other hardware elements to communicate at various frequencies and according to their respective protocols. In one configuration, the communication system 127 communicates via a communication protocol such as WiFi, dedicated short-range communication (DSRC), vehicle-to-infrastructure communication (V2I), vehicle-to-vehicle communication (V2V), or another suitable protocol for communication between the vehicle 100 and other entities in a cloud environment. Furthermore, in one configuration, the communication system 127 further communicates according to a protocol such as Global System for Mobile Communications (GSM), GSM Evolutionary High-Speed ​​Data Rate (EDGE), Long-Term Evolution (LTE), 5G, or another communication technology provided for the vehicle 100 to communicate with various remote devices (e.g., cloud-based servers). In any case, the avatar rendering system 126 can utilize various wireless communication technologies to provide communication to other entities, such as members of a cloud computing environment.

[0026] Referring to Figure 2, one embodiment of the avatar rendering system 126 of Figure 1 is further shown. The avatar rendering system 126 is shown as including a processor 101 from the vehicle 100 of Figure 1. Thus, the processor 101 may be part of the avatar rendering system 126, the avatar rendering system 126 may include a processor separate from the processor 101 of the vehicle 100, or the avatar rendering system 126 may access the processor 101 through a data bus or another communication path separated from the vehicle 100. In one embodiment, the avatar rendering system 126 includes a memory 232 that stores avatar modules 234 and display modules 236. The memory 232 is random access memory (RAM), read-only memory (ROM), a hard disk drive, flash memory, or another suitable memory for storing modules 234 and 236. Modules 234 and 236 are, for example, computer-readable instructions that, when executed by the processor 101, cause the processor 101 to perform various functions disclosed herein. In an alternative configuration, modules 234 and 236 are independent elements from memory 232, consisting, for example, of hardware elements. Thus, modules 234 and 236 are either application-specific integrated circuits (ASICs), hardware-based controllers, logic gate components, or other hardware-based solutions.

[0027] Furthermore, in one embodiment, the avatar rendering system 126 includes a data store 118. The avatar rendering system 126 is shown as including the data store 118 from the vehicle 100 in Figure 1. Thus, the data store 118 may be part of the avatar rendering system 126, the avatar rendering system 126 may include a data store separate from the data store 118 of the vehicle 100, or the avatar rendering system 126 may access the data store 118 through a data bus or another communication path separate from the vehicle 100. In one embodiment, the data store 118 is an electronic data structure stored in memory 232 or another data storage device, and consists of routines that can be executed by the processor 101 for analysis, provision, organization, etc. of the stored data. Thus, in one embodiment, the data store 118 stores data used by modules 234 and 236 when performing various functions.

[0028] In one embodiment, the data store 118 stores sensor data 228. Sensor data 228 is an example of sensor data 122 shown in Figure 1. Generally, sensor data 228 is data provided from one or more sensors of the sensor system 102. Thus, sensor data 228 includes observations of the surrounding environment of the vehicle 100 and / or observations of the interior environment of the vehicle 100. For example, the vehicle 100 is equipped with interior cameras that capture images of occupants inside the vehicle 100. Sensor data 228 includes the output (i.e., images) of these cameras. These camera images can serve as the basis for rendering occupant-based digital avatars.

[0029] As will be described later, the vehicle occupants instruct the avatar rendering system 126 to apply specific elements or themes from objects in the surrounding environment to the generated avatar. Such commands are issued in various forms, including gesture commands and voice commands. In the case of gesture commands, the vehicle 100's interior camera captures an image of the occupant so that the gesture command is detected. The vehicle 100 also includes an interior microphone for capturing audio signals, from which voice commands are extracted. In both cases, the outputs of these interior sensors are included in the sensor data 228.

[0030] Furthermore, the vehicle 100 includes outward-facing cameras 108 that capture images of the environment surrounding the vehicle 100 and images of objects in the environment. As will be described later, the avatar rendering system 126 processes these images to 1) identify objects in the images, including objects of particular interest to the vehicle occupants, and 2) identify the elements and / or thematic features of the objects. Thus, the sensor data 228 also includes the outputs of these outward-facing cameras.

[0031] In one embodiment, the data store 118 stores sensor data 228 together with metadata that characterizes various aspects of the sensor data 228. For example, the metadata may include location coordinates (e.g., latitude and longitude), relative map coordinates or tile identifiers, a timestamp / date stamp indicating when the individual sensor data 228 was generated, and so on.

[0032] In one embodiment, the data store 118 further includes an avatar model 230, which can be used by an avatar module 234 to generate digital avatars. As an example, the avatar rendering system 126 is a generative neural network-based system that generates presentation digital avatars based on patterns identified from a corpus of digital images (e.g., digital content available on the Internet). In the context of this specification, the generative neural network-based avatar rendering system 126 utilizes some form of supervised, unsupervised, reinforcement learning, or other types of generative machine learning to identify physical elements of an object or themes of visual features of an object based on a vehicle capture image. The avatar rendering system 126 transfers these elements or themes onto a basic digital avatar of the vehicle occupant. In any case, the avatar model 230 includes weights (including trainable and untrainable), biases, variables, activation functions, rules, algorithms, parameters, and other elements that operate to output occupant-specific and customized digital avatars based on a vehicle capture image of the surrounding environment. Further details regarding the operation of the neural network-based avatar generation system 126 are provided below in relation to Figure 10.

[0033] The avatar rendering system 126 includes an avatar module 234. Generally, the avatar rendering system 126 converts an image of an occupant into a digital avatar. This includes, in some examples, converting the occupant's features into a stylized version by incorporating thematic or physical elements identified in the surrounding environment of the vehicle 100. The avatar module 234 processes the captured image and deploys a neural network to generate a presentation digital avatar by modifying a base digital avatar based on a given input. The avatar module 234 then sends the presentation digital avatar to the display module 236, which generates the presentation digital avatar on the display device of the vehicle 100.

[0034] As will be described in more detail below, the avatar rendering system 126, more specifically the avatar module 234, is a neural network-based system that creates new digital content, such as digital avatars, based on identified patterns learned from large datasets. In this example, the avatar module 234 accesses a corpus 238 of digital content via a communication system 127. In one example, the corpus 238 is the Internet and all information accessible on the Internet. The corpus 238 may include data from retail websites, video sharing applications, images, social media platforms, and other sources. Thus, as will be described later, the avatar module 234 can access the corpus 238 to help identify the physical or thematic elements of an object to be incorporated into a digital avatar. While a specific corpus 238 (e.g., the Internet) is mentioned, the corpus 238 may include other datasets, such as a structured dataset of images tagged with metadata indicating the thematic and / or physical features of an object.

[0035] The avatar module 234 includes instructions to the processor 101 to extract the physical characteristics of the occupants of vehicle 100 from the vehicle capture images of the occupants. That is, the final digital avatar for presentation will resemble the occupants. Therefore, the avatar module 234 captures images of the occupants and analyzes the images to extract characteristic features of the occupants that will be converted into a digital form such as a basic digital avatar. For example, each occupant has distinctive facial features. Examples include the position and shape of the eyes, the position and shape of the nose, the position and shape of the nose, the position and shape of the mouth, the cheek line, the hairline, the jawline, and other distinctive features including the structure of the face. While specific features are mentioned, the avatar module 234 may identify features of other occupants. Furthermore, while facial features are mentioned in particular, in some examples the avatar module 234 may extract other features of the occupants. For example, the avatar module 234 may extract the user's hair length, hair color, hairstyle, shoulder width, etc.

[0036] In any case, the avatar module 234 includes a processor that identifies these characteristics of the occupant from images captured by the vehicle's onboard camera. For example, feature identification is performed by analyzing the pixels and their color values ​​that make up the image. In addition to identifying features in the image, the avatar module 234 can also identify the location of features. For example, the avatar module 234 can determine the relative distance between different detected features. As a result, the avatar module 234 generates a map of the occupant's face based on the detected facial features and their relative and absolute locations.

[0037] In one example, the avatar module 234 extracts features from multiple images, which may, for example, capture various aspects of the occupant as they rotate / turn their heads while the vehicle 100 is in motion. For example, the digital avatar may be a three-dimensional avatar that can be viewed from multiple angles (i.e., by animating the digital avatar). In this example, a single image does not contain enough information to generate a 3D model. Therefore, in this example, the avatar module 234 extracts features from multiple images to establish a three-dimensional representation of the occupant.

[0038] In one specific example, the avatar module 234 deploys a machine learning or neural network-based system to extract these characteristic features of the user. For example, the machine learning system detects, identifies, and maps facial features such as eyes, nose, mouth, and face shape.

[0039] The avatar module 234 also includes instructions to the processor 101 to render a basic digital avatar of the occupant based on the extracted physical characteristics of the occupant. Generally, the digital avatar is a digital representation of the occupant and has similar features to the occupant, such as the position and shape of the eyes, the position and shape of the nose, the position and shape of the mouth, the shape of the mouth, the cheek line, the hairline, the jawline, the facial structure, etc. Therefore, the avatar module 234 uses the extracted features to generate a digital caricature of the vehicle's occupant.

[0040] In some cases, the base digital avatar is a modifiable digital representation of the occupant from multiple viewpoints. That is, the base digital avatar can be moved, rotated, or manipulated and displayed from various angles. For example, the head of the digital model may be animated to look upwards and to the right. Thus, the base digital avatar is not merely a static copy of the input image, but a complete three-dimensional representation of the occupant so that the display module 236 can present the base digital avatar so that it can be viewed from various angles. As mentioned above, a single vehicle capture image may not provide sufficient detail for all angles of the occupant in order to generate a multi-viewpoint model of the occupant. Therefore, the avatar module 234 generates a three-dimensional avatar by combining multiple images taken from various viewpoints and captured while the occupant is operating the vehicle 100. In other words, the base digital avatar is a three-dimensional representation of the occupant generated from multiple two-dimensional images.

[0041] The avatar module 234 includes a command to the processor 101 to extract the features of objects in the external environment of the vehicle 100 from the vehicle-captured images of the objects. As described above, the occupants of the vehicle 100 may want to incorporate the features of objects in the environment onto their digital avatars in the vehicle. Conventionally, the occupants of the vehicle 100 have had no means to capture images of objects in a moving vehicle while driving or in motion, and to transfer the features of those objects onto their digital avatars in the moving vehicle 100. This avatar rendering system 126 achieves this by using external vehicle sensors to identify and capture images of objects, extract the features of those objects, and apply such features to the digital avatars in the vehicle for the occupants while the vehicle 100 is moving or on the road.

[0042] In one example, the features extracted are the physical structure of an object. For instance, the driver of vehicle 100 might see clothing in a shop window or clothing worn by a pedestrian. The driver might want to see what they (i.e., the driver) would look like if they wore the clothing in the shop window or the clothing worn by the pedestrian. Therefore, the avatar module 234 identifies the object, generates a digital representation of the object, and transposes the object onto the driver's base digital avatar to generate a presentation digital avatar incorporating the clothing. Further details regarding object identification and extraction of features of the object's physical structure are described below in relation to Figures 6 and 7.

[0043] In another example, the features are the visual elements of an object. For instance, while driving along a road, a passenger might see a poster for a Victorian-era period film. This film poster might have a distinctive visual feature among other visual features, such as a specific color palette, color grading, lighting effects, texture, composition, and / or motif. In this example, the avatar module 234 identifies and extracts these and other visual features from the object to generate a theme, which is defined by various visual features. The avatar module 234 then applies the theme (i.e., the distinctive visual features from the object) to a base digital avatar, thus generating a presentation digital avatar based on real-time information collected as the vehicle 100 moves along the road. Additional details regarding identifying the theme of an object and applying such a theme to the occupant's base digital avatar in order to generate the occupant's presentation digital avatar are described below in relation to Figures 8 and 9.

[0044] It should be noted that the avatar module 234 can deploy a neural network in any of these examples. That is, as will be described later in relation to Figure 10, the avatar module 234 identifies additional digital content from the corpus 238 that has similar visual features to the object, in addition to utilizing features extracted from the object itself. The avatar module 234 applies 1) visual features from the object (such as those extracted from vehicle capture images) and 2) visual features from the additional digital content to the base digital avatar in order to generate a digital avatar for presentation.

[0045] In either case, the avatar module 234 includes a command to the processor 101 to render a presentation digital avatar similar to the occupant by transferring the features of an object (the physical structure of the object or the thematic elements of the object) onto the occupant's basic digital avatar.

[0046] Again, this can be implemented with neural networks. For example, deep learning models such as generative adversarial networks (GANs) can apply thematic effects to digital models to create avatars. As mentioned above, neural networks are trained on data such as text, audio, or images to identify structures and patterns in the data. In the context of this specification, GANs are trained on datasets containing hundreds of thousands of tagged images (i.e., supervised learning on curated datasets) or untagged images (i.e., unsupervised learning on unregulated datasets such as the internet). Neural networks are tuned using gradient descent to identify the network's parameters and enhance its ability to leverage specific visual features of objects.

[0047] Although the avatar module 234 is described as controlling various sensors to provide sensor data 228, in one or more embodiments, the avatar module 234 may acquire sensor data 228 using other active or passive techniques. For example, the avatar module 234 may passively sniff sensor data 228 from a stream of electronic information provided by various sensors to further components within the vehicle 100.

[0048] It should be understood that the avatar module 234 can be combined with the avatar model 230 to form a computational model such as a neural network model. In any case, when the avatar module 234 is implemented in a neural network model or another model, in one embodiment it implements the functional aspects of the avatar model 230, with further aspects such as learned weights stored in the data store 118. Thus, the avatar model 230 is generally integrated with the avatar module 234 as a cohesive functional structure.

[0049] The avatar rendering system 126 also includes a display module 236, which includes commands to the processor 101 to animate a presentation digital avatar on the display device of the vehicle 100. For example, the display module 236 can animate the mouth of the presentation digital avatar in sync with the output sound. While specific animations are mentioned, the display module 236 may animate the presentation of the digital avatar in various ways. Furthermore, the avatar rendering system 126 may also use a neural network having a pre-trained model that transforms the presentation digital avatar to appear animated.

[0050] As a result, the avatar rendering system 126 of this specification generates a digital avatar that not only has a similarity to the occupant, but also incorporates features of the environment objects captured by the moving vehicle 100, such as the physical structure of the objects (e.g., clothing) and thematic elements (e.g., color scheme, composition, etc.).

[0051] In the example, the avatar rendering system 126, as shown in Figure 2, is implemented in a vehicle 100 as depicted in Figure 1, or in a cloud environment 340 as depicted in Figure 3. As shown in Figure 3, at least a portion of the avatar rendering system 126 is implemented in the cloud environment 340. That is, as described above, the avatar rendering system 126 is implemented in various embodiments, partially in the vehicle 100 and as a cloud-based service. For example, in one method, at least some functions related to at least one module of the avatar rendering system 126 are implemented in the vehicle 100, and further functions are implemented in the cloud environment 340. For example, each vehicle 100-1, vehicle 100-2, and vehicle 100-3 includes instances of the avatar rendering system 126-1, avatar rendering system 126-2, and avatar rendering system 126-3, respectively, each capturing images of its respective occupants and the surrounding environment. In one particular example, vehicle-based instances of avatar rendering systems 126-1, 126-2, and 126-3 also perform some image processing, such as identifying features in an image and animating the presentation of a digital avatar, in response to reception from a cloud-environment-based instance of avatar rendering system 126-4. In any example, the cloud-environment-based instance of avatar rendering system 126-4 may perform other operations described herein, such as neural network-based object detection and / or theme generation. For example, the cloud-environment-based instance of avatar rendering system 126-4 may interact with the corpus of digital content 238 to identify other images of objects having similar physical structures, or other images of objects having themes similar to the vehicle-captured object.

[0052] An additional aspect of generating occupant-specific digital avatars based on vehicle sensors will be discussed in relation to Figure 4. Figure 4 shows a flowchart of Method 400 relating to generating occupant avatars in a vehicle based on vehicle sensor capture data. Method 400 will be discussed in relation to the avatar rendering system 126 in Figures 1, 2, and 3. Although Method 400 is discussed in conjunction with the avatar rendering system 126, it should be understood that Method 400 is not limited to being implemented within the avatar rendering system 126, but is merely an example of a system that implements Method 400.

[0053] In 410, the avatar rendering system 126 extracts the physical characteristics of the occupants of the vehicle 100 from the vehicle capture images of the occupants. That is, as described above, occupant-facing cameras installed in the interior space of the vehicle 100 capture images or video streams of the occupants. The avatar module 234 includes a processor 101 that can detect objects in the image, such as the occupants' faces or other physical characteristics, and determine the location of the features. Thus, in 410, the avatar module 234 extracts features and determines the relative locations of various features of the occupants in the image. This information is later used to generate a basic digital avatar similar to the occupants.

[0054] In 420, the avatar module 234 renders a basic digital avatar of the occupant based on the extracted physical characteristics of the occupant. Specifically, the avatar module 234 constructs a representation of the occupant based on the extracted facial features (e.g., shape, tone, size, etc.). In particular, the avatar module 234 positions the pixels that define the basic digital avatar to match the detected facial features of the occupant captured by the onboard camera.

[0055] In 430, the avatar module 234 extracts the features of objects in the external environment of the vehicle 100. That is, the occupants of the vehicle 100 may want to incorporate objects observed in the real world into their personalized digital avatars. For example, the occupants may want to incorporate the physical structure of real-world objects into their avatars. In another example, the occupants may want to incorporate the visual features of real-world objects such as movie posters or pedestrians into their avatars. In either case, after identifying the target object, the avatar module 234 extracts the features of the target object, which are either distinctive features of the object that allow for the identification of the object's digital representation in the digital content corpus 238, or thematic features of the object that allow for the application of the theme to the digital avatar. Specifically, the avatar module 234 analyzes the pixels that make up the image to identify the structure and / or visual features of the object.

[0056] In 440, the avatar module 234 renders a digital avatar for the crew by transferring the features of an object (e.g., the physical structure or thematic features of an object) onto a base digital avatar. Specifically, the avatar module 234 can modify the pixel configuration of the base digital avatar to incorporate a digital representation of an object or include the thematic elements of an object.

[0057] In 450, the avatar rendering system 126, more specifically the display module 236, animates the presentation digital avatar on the display device of the vehicle 100. That is, to make the presentation digital avatar more appealing, it is animated to move. For example, when giving instructions, the display module 236 animates the mouth of the presentation digital avatar to match the words of the instruction. In another example, the display module 236 can animate the limbs of the presentation digital avatar or move the head of the presentation digital avatar to direct the occupant's attention to a specific area of ​​the infotainment display. While specific animations are mentioned, the presentation digital avatar may be animated in various ways as needed to communicate with the occupant of the vehicle 100.

[0058] Figures 5A and 5B illustrate an example of generating a basic occupant-based digital avatar 544 according to an embodiment disclosed herein. Specifically, Figure 5A depicts a captured image of an occupant 542 of vehicle 100, and Figure 5B depicts a basic digital avatar 544 that is similar to the occupant 542 and based on vehicle capture information. That is, as described above, vehicle 100 includes a camera that captures an interior image of the occupant 542 of vehicle 100.

[0059] As shown in Figure 5A, the image of crew member 542 is two-dimensional and may not provide crew characteristic data for certain parts of crew member 542, such as the back of the crew member's head and / or parts of the crew member's body that are obscured. In the example shown in Figure 5A, part of the crew member's torso is hidden by the outstretched arm of crew member 542. Therefore, when generating the basic digital avatar 544 shown in Figure 5B, the avatar module 234 captures multiple images of crew member 542 so that features of crew member 542 that are hidden or not visible in one image are complemented by other captured images of crew member 542.

[0060] As described above, the image processor of the avatar module 234 may be a neural network-based processor that analyzes images captured by the in-vehicle camera to identify and locate certain characteristic features of the occupant 542. Such features include facial features, such as the structure of the face and the characteristics of key facial features such as the eyes, nose, mouth, and ears. Exemplary features include the size of the feature, the shape of the feature, and the tone of the feature. While specific features of occupant 542 extracted from the image are mentioned in order to define occupant 542, the avatar module 234 may extract other features from the vehicle-captured image of occupant 542.

[0061] As depicted in Figure 5B, the avatar module 234 generates a basic digital avatar 544 for crew member 542. The basic digital avatar 544 is similar to crew member 542, meaning that it is rendered to show the same characteristics and features of crew member 542 as those extracted from the vehicle capture image of crew member 542. In other words, the basic digital avatar 544 has the same facial structure and key feature characteristics as crew member 542, but is stylized by changing the color scheme, line width, etc., of the features of the basic digital avatar 544 while maintaining the physical characteristics of crew member 542.

[0062] Figure 6 shows a flowchart of one embodiment of Method 600 related to generating a crew-based avatar by transferring objects onto an avatar, according to an embodiment disclosed herein. See Figure 7, which depicts a scenario in which Method 600 is performed.

[0063] As described above, the avatar rendering system 126 applies, transposes, or transfers objects detected by vehicle sensors, such as the outward-facing camera 108, onto the occupant-based basic digital avatar 544. As a result, a user-customized presentation digital avatar 750 is presented on the vehicle 100's display device 748. In the examples depicted in Figures 6 and 7, the transferred features are the physical structure of the object. Specifically, the object depicted in Figure 7 is a pair of sunglasses and a jacket worn by a pedestrian 746 detected in the environment.

[0064] In 602, as depicted and described in relation to Figure 5A, the avatar module 234 extracts the physical characteristics of the vehicle occupant 542 from the vehicle capture image, and in 604, the avatar module 234 renders a basic digital avatar 544 of the occupant 542. The remainder of Method 600 describes how the features of environment objects are transposed onto the basic digital avatar 544 in order to generate a presentation digital avatar 750. First, the avatar rendering system 126 identifies the target object on which to transpose its features (in this example, the physical structure of the object). Thus, in 606, the avatar rendering system 126 detects the actions of the occupant 542 that identify the object in the environment. That is, the avatar rendering system 126 includes an instruction to the processor 101 to detect the actions of the occupant 542 that identify the object and the location of the object. The actions can take various forms.

[0065] For example, as shown in Figure 7, occupant 542 points to an object. In this example, occupant 542 points to a pedestrian 746 crossing in front of vehicle 100. In another example, occupant 542 identifies an object by voice. For example, occupant 542 says, "Update my avatar to include the sunglasses and jacket worn by the man crossing the street in front of me." As described above, vehicle 100 may include various sensors, including an in-vehicle camera, that capture gestures and identify the target of the gesture (e.g., the location that occupant 542 is pointing to). In the example, vehicle 100 may identify the target object by analyzing the direction of occupant 542's gaze / head posture. Furthermore, vehicle 100 may also include a microphone for capturing voice commands. The processor 101 of the avatar rendering system 126 analyzes one or more of these inputs (e.g., voice commands, body gestures, head / gaze direction) to identify the target object and its location. Specifically, the avatar rendering system 126 identifies the location of the target object by partially identifying the direction of the gesture, the direction of the head / gaze, or other locations around the vehicle 100 where the object is located.

[0066] In step 608, the avatar rendering system 126 acquires an image captured by a vehicle camera having a field of view that overlaps with the object's position. That is, the avatar rendering system 126 can identify a sensor with a field of view that captures an object by analyzing the user's movements. For example, the avatar rendering system 126 determines that the occupant 542 is looking / gestured toward the front of the vehicle 100. Therefore, the avatar rendering system 126 acquires an image from the outward-facing camera 108 in front of the vehicle 100. In other words, the system can receive a vehicle-captured image of the object from the vehicle camera.

[0067] In some cases, the captured image may have low resolution, be obscured, or otherwise lack sufficient resolution to extract specific features of an object. That is, when attempting to extract features, the image processor includes an arbitrary number of thresholds, by which it determines that it cannot accurately extract the identifying features of an object from the image. Therefore, in 610, the avatar module 234 determines the level of detail of the object's features in the vehicle captured image and, based on a predetermined threshold index, determines whether the image has a sufficient level of detail to extract the object's features. Depending on whether the level of detail is greater than the threshold amount, in 614, the avatar module 234 extracts the prominent features of the object from the vehicle captured image. In this example, the avatar module 234 can access the outputs of different external cameras to identify images of objects with a detail greater than the threshold amount.

[0068] The avatar module 234 identifies previously captured images of the object from the log of vehicle capture images, depending on whether the level of detail is below a threshold. That is, the sensor data 228 described above includes a history of images collected from the outward-facing camera 108. Therefore, once the target object is identified in the current image, the avatar rendering module 234 processes other images in the log to determine whether the target object can be identified in other images. This is done based on the identification of the object at a lower resolution. For example, an image may not be clear enough to characterize the object for rendering to an avatar, but may be clear enough to facilitate the identification of the object in other images. In either case, in 612 and 614, the avatar module 234 extracts prominent features of the object from either the current vehicle capture image of the object or a previously captured vehicle capture image of the object.

[0069] As described above, the digital model of an object may be reconstructed based on a combination of multiple images of the object. That is, it is difficult to generate a complete digital model of an object based on a single vehicle capture image, or even based on multiple vehicle capture images. Therefore, the avatar rendering system 126 acquires additional data points that serve as a source guide for rendering the digital model of the object, in some examples using a neural network. Specifically, in 616, the avatar module 234 works to identify the object in other images of the corpus 238 of digital content. This is done based on the object's distinctive features; that is, each object has certain features that distinguish it from other objects. Examples of distinctive features include logos, patterns, colors, etc. For example, the sunglasses worn by pedestrian 746 may include the manufacturer's logo. Furthermore, the jacket worn by pedestrian 746 may include distinguishable features such as the shape and color of the collar. While specific distinctive features have been mentioned, the avatar module 234 may also extract other distinctive features of the object, such as shape, color, and material. The avatar module 234 extracts these prominent features and other prominent features from the vehicle capture images. Then, based on these prominent features, the avatar module 234 identifies objects in other images. In other words, the avatar module 234 can explore other images in the corpus 238 to identify prominent features in various images.

[0070] In either case, in 618, the avatar module 234 renders a digital model of the object by combining the vehicle capture image of the object with other images of the object. That is, in some examples where a neural network is deployed, the avatar module 234 can identify patterns in the image of the object and utilize these patterns to generate a digital representation of the object.

[0071] However, in some cases, the multiple images that form the basis of the digital model may not include views of the object from all angles, or the lighting in the images may differ from the lighting intended for the presentation digital avatar 750. In other words, combining multiple images may result in an incomplete digital representation. In this example, the neural network avatar module 234 infers missing features of the object from the vehicle capture image of the object and other images of the object. For example, the image of the jacket worn by pedestrian 746 may not include details about the underarm area of ​​the jacket. In this example, the generative neural network avatar module 234 supplements the details in this area of ​​the jacket to render a three-dimensional representation of the jacket. In another example, the avatar module 234 modifies the lighting of the jacket to reflect the lighting of the environment of the presentation digital avatar 750.

[0072] In 620, the avatar module 234 renders the crew member 542's presentation digital avatar 750 by transferring objects onto the crew member 542's base digital avatar 544. For example, the avatar module 234 generates the crew member's presentation digital avatar 750 by transferring sunglasses and a jacket onto the crew member 542's base digital avatar 544. Specifically, the avatar module 234 modifies the pixels that define the base digital avatar 544 so that the pixels that define the base digital avatar 544 match the pixels of the digital model of the objects.

[0073] As described in 622 and above, the display module 236 animates the presentation digital avatar 750 on the display device 748 of the vehicle 100, along with objects transposed onto the presentation digital avatar 750. Thus, the avatar rendering system 126 provides a uniquely generated, user-based, customized presentation digital avatar 750 based on vehicle sensor acquisition data, thereby enabling the real-time integration of environmental elements in a moving environment onto the avatar in the vehicle.

[0074] Figure 8 shows a flowchart of one embodiment of Method 800 related to generating a crew-based avatar by transferring an object theme onto a basic digital avatar 544, according to an embodiment disclosed herein. See Figure 9, which depicts a scenario in which Method 800 is performed.

[0075] As described above, the avatar rendering system 126 applies, transposes, or transfers objects detected by vehicle sensors, such as the outward-facing camera 108, onto the occupant-based basic digital avatar 544. As a result, a user-customized presentation digital avatar 750 is presented on the display device 748 of the vehicle 100. In the examples depicted in Figures 8 and 9, the transferred features are thematic features of an object that is a movie poster 952. Although the object of the movie poster 952 is specifically mentioned, the object may be of various types, including a person, a video stream, a billboard, or other physical object with distinctive visual features. That is, an image has certain visual features that give it a specific aesthetic sense and tone. Examples of thematic features include subject matter, color palette, color scheme (including shading, hue, tone, and contrast), composition or arrangement of elements in the image, shape in the image, lighting in the image (including light quality, direction, and intensity), lighting effects (e.g., natural light, artificial light, shadows, and highlights), texture (e.g., photographic texture, material texture, or brushwork), style (e.g., cartoon, anime, realism, surrealism, vintage, modern, etc.), viewpoint, tonal range, contrast, filters, framing, layout, margins, and typography. While specific visual features are mentioned, an image or object may contain various visual features that contribute to the theme of the image or object. In this example, avatar module 234 uses a neural network to identify these visual features of the object and other similar objects and defines the theme applied to the basic digital avatar 544.

[0076] In 802, as depicted and described in relation to Figure 5A, the avatar module 234 extracts the physical characteristics of the vehicle occupant 542 from the vehicle capture image, and in 804, the avatar module 234 renders a basic digital avatar 544 of the occupant 542. The remainder of Method 800 explains how the features of the environment object are transposed onto the basic digital avatar 544.

[0077] As described above, in 806, the avatar rendering system 126 detects the actions of the occupant 542 in identifying objects in the environment. In 808, the avatar rendering system 126 acquires an image from a vehicle camera having a field of view that overlaps with the location of the object.

[0078] As described above, the captured image may have low resolution, be obscured, or otherwise lack sufficient resolution to extract the object's identifying features. Therefore, in 810, the avatar module 234 determines, based on a predetermined threshold index, whether the image has a sufficient level of detail to extract the object's features. Depending on whether the level of detail is greater than the threshold, in 814, the avatar module 234 extracts the object's definitive visual features (e.g., those contributing to the object's theme) from the vehicle capture image. Depending on whether the level of detail is less than the threshold, in 812, the avatar module 234 identifies previously captured images of the object from the log of the vehicle capture image and extracts the object's definitive visual features from the previously captured vehicle capture image.

[0079] In one example, a theme is more fully constructed by considering additional images containing similar visual features. That is, the object itself (e.g., movie poster 952 itself) has visual features that fit a particular theme, but there may be other visual features that fit a particular theme. Therefore, in 816, the avatar module 234 identifies the definitive visual features of other images. Specifically, the avatar module 234 searches the corpus 238 to identify instances similar to the object (e.g., movie poster 952), or to identify other images in the corpus 238 that have the same or similar definitive visual features as the object. For example, the avatar module 234 may look for other images containing movie poster 952, or other images / screenshots from the movie, or other promotional materials. That is, the corpus 238 contains additional instances of the object, or images of other objects that are visually similar to the object, determined by comparing the definitive visual features of the images in the corpus 238 with the definitive visual features of the current vehicle capture image of the object. It should be noted that when analyzing images from Corpus 238, the Avatar Module 234 can identify not only images / objects with identical visual features, but also images / objects with similar visual features that have a similarity measured by a predetermined threshold. For example, images in Corpus 238 may have similar but not identical subjects, color palettes, color schemes, compositions, lighting, textures, styles, tonal ranges, contrasts, and / or layouts. These images are paired with vehicle capture images on the same theme based on an arbitrary predetermined similarity criterion, the similarity criterion being machine-learned and / or based on user feedback.

[0080] In either case, in 818, the avatar module 234 constructs themes for the presentation digital avatar 750 based on the definitive visual features of the object. Specifically, the avatar module 234 constructs themes related to the object by deploying a neural network that identifies the visual features of the object and patterns of visual features in the vast repository of digital content within the corpus 238.

[0081] Avatar module 234 constructs a theme by aggregating the definitive visual features of objects and other images. For example, avatar module 234 weights various visual features when constructing a theme. In any case, in 820, avatar module 234 renders the presentation digital avatar 750 of crew member 542 by transferring the visual features defined by the theme onto the base digital avatar 544 of crew member 542. It should be noted again that when applying visual features, avatar module 234 can apply variations of the exact visual features identified in the object. That is, avatar module 234 does not apply specific visual features of the object to the base digital avatar 544, but rather applies the visual features of a theme that is partially defined by the visual features of the object, and the visual features of the theme are broader than those defined by the visual features of the object. For example, as depicted in Figure 9, movie poster 952 is defined by the visual features of movie poster 952, which include an image of a cowboy riding a horse wearing a trench coat and waving a hat. In contrast, the presentation digital avatar 750 may have a similar theme, but may also have different types of cowboy hats, bandanas, and leather vests that fit the Western theme adopted by the movie poster 952. While specific examples are provided regarding the transposition of thematic and compositional components from the movie poster 952 to the presentation digital avatar 750, it should be noted that in other examples, other visual features may be transposed to the presentation digital avatar 750.

[0082] Figures 8 and 9 specifically illustrate the transposition of visual features to the presentation digital avatar 750, but it should be noted that other elements that fit the theme may also be transposed to the presentation digital avatar 750. For example, specific vocabulary, phrases, accents, etc., may be associated with a particular theme, and specific forms of behavior may be associated with a particular theme. In this example, by utilizing the deployed generative neural network, the avatar module 234 can construct an auditory theme for the presentation digital avatar 750 based on the definitive visual features of the object. That is, the avatar module 234 deploys a neural network trained on the dataset to identify patterns between the visual and auditory features of the digital content. Therefore, upon receiving an input with visual features that fit the theme, the generative neural network deployed by the avatar module 234 identifies patterns between the visual and auditory characteristics associated with the theme and outputs sounds related to the presentation digital avatar 750 based on the auditory theme. Similarly, upon receiving an input with visual features that match the theme, the generative neural network deployed by the avatar module 234 identifies behavioral behavior and animates the presentation digital avatar 750 to match the behavioral behavior associated with the theme by the generative neural network.

[0083] In 822, as described above, the display module 236 animates the presentation digital avatar 750 on the display device of the vehicle 100, along with additional objects transposed on the presentation digital avatar 750. Thus, the avatar rendering system 126 provides a uniquely generated, user-based, customized presentation digital avatar 750 based on vehicle sensor acquisition data, thereby enabling the real-time integration of environmental elements in a moving environment onto the avatar in the vehicle.

[0084] Figure 10 shows one or more embodiments of a neural network-based avatar rendering system 126 related to generating a presentation digital avatar 750 based on vehicle sensor data, according to embodiments disclosed herein.

[0085] In one approach, the avatar rendering system 126 implements and / or uses a machine learning algorithm. Generally, a machine learning algorithm identifies patterns and deviations based on previously unseen data. In the context of this specification, the machine learning avatar rendering system 126 utilizes several forms of machine learning, including supervised, unsupervised, reinforcement learning, or other types, to identify patterns in an image and generate avatars based on these patterns. In the context of this specification, the neural network-based avatar rendering system 126 creates novel digital avatars that mimic or reproduce the visual features of a received image. That is, once trained, the neural network generates novel digital avatars based on learned patterns in response to an input image.

[0086] In one particular example, the machine learning model is a neural network comprising: 1) any number of input nodes that receive crew images 1054, object images 1056, and digital content from corpus 238; 2) any number of hidden nodes located in layers connected to the input nodes and / or other hidden nodes, containing computational instructions for calculating outputs; and 3) any number of output nodes connected to hidden nodes that generate presentation digital avatars 750. Various types of neural networks, including feedforward neural networks (FNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), artificial neural networks (ANNs), deep neural networks (DNNs), generative adversarial networks (GANs), or diffusion models, can be implemented according to the principles described herein. Naturally, in further embodiments, the avatar module 234 may employ different machine learning algorithms or implement different techniques. Regardless of which specific technique the avatar module 234 implements, the avatar module 234 provides an output of a presentation digital avatar 750 modified by vehicle-captured images of the surrounding environment. In this way, the avatar rendering system 126 generates a digital avatar influenced by the moving environment for the occupants of the vehicle 100.

[0087] As shown in Figure 10, each layer includes nodes that process input data and generate an output. The nodes in the input layer receive raw data, including occupant images 1054, object images 1056, and images from the corpus 238, as described above. Nodes in the hidden layer perform calculations and identify patterns in the input data. Nodes in the output layer generate a presentation digital avatar 750, which is presented on the display device 748 of the vehicle 100. The avatar model 230 described above may include weights, biases, activation functions, and other parameters that guide the learning of the neural network.

[0088] During operation, the neural network performs any number of complex operations, such as 1) forward propagation, where input data passes through hidden layers and is transformed based on the weights, biases, and activation functions in the Avatar Model 230, and 2) backpropagation, where the weights and biases are adjusted based on the loss function or error.

[0089] As described above, the avatar module 234 uses trained or untrained neural networks to extract features from the crew images 1054, object images 1056, and corpus 238. These include various features such as edges, lines, patterns, and colors, as well as visual features such as the visual features that define the theme described above. Nodes in the hidden layer then adjust the images based on the learned patterns, ensuring that the structure of the crew features captured in the crew images 1054 is preserved. The avatar rendering system 126 repeatedly compares the presentation digital avatars 750 being developed with the crew images 1054 to ensure content preservation. Furthermore, the avatar rendering system 126 may compare the presentation digital avatars 750 being developed with the object images 1056 and corpus 238 to ensure theme fit.

[0090] It should be understood that machine learning algorithms are generally trained to perform predefined tasks. Therefore, unless otherwise specified, training a machine learning algorithm is understood to be distinct from the general use of a machine learning algorithm. Thus, the Avatar Rendering System 126, or any other system, generally trains machine learning algorithms according to specific training methods, including supervised learning, self-supervised learning, reinforcement learning, etc. In contrast to training / learning a machine learning algorithm, the Avatar Rendering System 126 implements the machine learning algorithm to perform inference. Therefore, the general use of a machine learning algorithm is described as inference.

[0091] Figure 1 is described in more detail below as an exemplary environment in which the systems and methods disclosed herein operate. In some cases, the vehicle 100 is configured to selectively switch between autonomous mode, one or more semi-autonomous modes, and / or manual mode. “Manual mode” means that all or most of the control and / or steering of the vehicle is operated by a user (e.g., a human driver) and is performed in accordance with inputs received via the human-machine interface (HMI) of the vehicle 100 (e.g., steering wheel, accelerator pedal, brake pedal, etc.). In one or more configurations, the vehicle 100 may be a manually controlled vehicle configured to be operated only in manual mode.

[0092] In one or more configurations, the vehicle 100 implements several levels of autonomous driving to operate autonomously or semi-autonomously. As used herein, the autonomous control of the vehicle 100 is defined along a range according to the SAE J3016 standard, which defines six levels of autonomous driving from 0 to 5. Generally, as described herein, semi-autonomous modes mean levels 0 to 2, while autonomous modes mean levels 3 to 5. For this reason, autonomous modes generally involve controlling and / or steering the vehicle 100 along a travel path via a computing system that controls the vehicle 100 with minimal or no input from a human driver. In contrast, semi-autonomous modes, sometimes referred to as advanced driver-assistance systems (ADAS), provide part of the control and / or steering of the vehicle along a travel path via a computing system, together with a vehicle operator (i.e., a driver) who provides at least part of the control and / or steering of the vehicle 100.

[0093] Continuing with the various components shown in Figure 1, the vehicle 100 includes one or more processors 101. In one or more configurations, the processor 101 may be the primary / centralized processor of the vehicle 100, or it may represent a number of distributed processing units. For example, the processor 101 may be an electronic control unit (ECU). Alternatively or in addition, the processor may include a central processing unit (CPU), a graphics processing unit (GPU), an ASIC, a microcontroller, a system-on-a-chip (SoC), and / or other electronic processing units that assist in the operation of the vehicle 100.

[0094] Vehicle 100 may include one or more data stores 118 for storing one or more types of data. The data stores 118 may consist of volatile and / or non-volatile memory. Examples of memory forming a data store 118 include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, magnetic disks, optical disks, hard drives, solid-state drives (SSDs), and / or other non-temporary electronic storage media. In one configuration, the data store 118 is a component of the processor 101. Generally, the data store 118 is operably connected to the processor 101 for use by the processor 101. As used herein, the term “operably connected” may include direct or indirect connections, including connections without direct physical contact.

[0095] In one or more configurations, one or more data stores 118 contain various data elements to support functions of the vehicle 100, such as semi-autonomous driving functions and / or autonomous driving functions. Thus, the data store 118 stores map data 119 and / or sensor data 122. In at least one method, the map data 119 includes maps of one or more geographical areas. In some cases, the map data 119 may include information about roads (e.g., lane and / or road maps), traffic control devices, road markings, structures, features, and / or landmarks in one or more geographical areas. In at least one method, the map data 119 is characterized as a high-definition (HD) map that provides information for autonomous driving and / or semi-autonomous driving functions.

[0096] In one or more configurations, map data 119 may include one or more topographic maps 120. The topographic maps 120 may include information about the ground, terrain, roads, surfaces, and / or other features of one or more geographic areas. The topographic maps 120 may include elevation data for one or more geographic areas. In one or more configurations, map data 119 may include one or more static obstacle maps 121. The static obstacle maps 121 may include information about one or more static obstacles located within one or more geographic areas. "Static obstacles" are physical objects whose location and general properties do not substantially change over a period of time. Examples of static obstacles include trees, buildings, curbs, fences, etc.

[0097] Sensor data 122 is data provided from one or more sensors of the sensor system 102. Therefore, sensor data 122 includes observations of the surrounding environment of the vehicle 100 and / or information about the vehicle 100 itself. In some cases, one or more data stores 118 mounted on the vehicle 100 store at least a portion of the map data 119 and / or sensor data 122. Alternatively or in addition, at least a portion of the map data 119 and / or sensor data 122 may be stored in one or more data stores 118 located remotely from the vehicle 100.

[0098] As described above, the vehicle 100 may include a sensor system 102. The sensor system 102 may include one or more sensors. As used herein, “sensor” means an electronic and / or mechanical device that generates an output (e.g., an electrical signal) in response to a physical phenomenon such as electromagnetic radiation (EMR), sound, etc. The sensor system 102 and / or one or more sensors may be operably connected to the processor 101, the data store 118, and / or other elements of the vehicle 100.

[0099] Various examples of different types of sensors are described herein. However, it should be understood that embodiments are not limited to the specific sensors described. In various configurations, the sensor system 102 includes one or more vehicle sensors 103 and / or one or more environmental sensors. The vehicle sensors 103 function to detect information about the vehicle 100 itself. In one or more configurations, the vehicle sensors 103 include one or more accelerometers, one or more gyroscopes, inertial measuring units (IMUs), dead reckoning systems, global navigation satellite systems (GNSS), global positioning systems (GPS), and / or other sensors for monitoring aspects of the vehicle 100.

[0100] As described, the sensor system 102 may include one or more environmental sensors 104, one or more of which detect the surrounding environment of the vehicle 100 (e.g., the external environment) and / or, in at least one configuration, the environment inside the vehicle 100. For example, one or more environmental sensors 104 may detect objects in the surrounding environment of the vehicle 100. Such obstacles may be static objects and / or dynamic objects. Various examples of sensors in the sensor system 102 are described herein. Exemplary sensors are part of one or more environmental sensors 104 and / or one or more vehicle sensors 103. However, it should be understood that embodiments are not limited to the specific sensors described. As an example, in one or more configurations, the sensor system 102 may include one or more radar sensors 105, one or more LiDAR sensors 106, one or more sonar sensors 107 (e.g., ultrasonic sensors), and / or one or more cameras 108 (e.g., monocular, stereoscopic, RGB, infrared, etc.).

[0101] Continuing the explanation of the elements in Figure 1, the vehicle 100 may include an input system 123. The input system 123 generally comprises one or more devices that enable the machine to acquire information from an external source, such as an operator. The input system 123 can receive input from the vehicle occupants (e.g., driver / operator, and / or passengers). In addition, in at least one configuration, the vehicle 100 includes an output system 124. The output system 124 comprises one or more devices that enable the provision of information / data to an external target (e.g., a person, vehicle occupants, another vehicle, another electronic device, etc.).

[0102] Furthermore, vehicle 100 includes one or more vehicle systems 109 in various configurations. Various examples of one or more vehicle systems 109 are shown in Figure 1. However, vehicle 100 may include different configurations of vehicle systems. Although specific vehicle systems are defined separately, it should be understood that each system, or any of the systems, or parts thereof, may be combined or separated in different ways via hardware and / or software within vehicle 100. As shown, vehicle 100 includes a propulsion system 110, a braking system 111, a steering system 112, a throttle system 113, a transmission system 114, a signaling system 115, and a navigation system 116.

[0103] The navigation system 116 may include one or more devices, applications, and / or combinations thereof to determine the geographical location of the vehicle 100 and / or the travel route of the vehicle 100. The navigation system 116 may include, for example, one or more mapping applications to determine the travel route of the vehicle 100 according to map data 119. The navigation system 116 may include a global positioning system, a local positioning system, or a geolocation information system, or at least provide a connection to a global positioning system, a local positioning system, or a geolocation information system.

[0104] In one or more configurations, the vehicle system 109 functions in conjunction with other components of the vehicle 100. For example, the processor 101, the avatar rendering system 126, and / or the autonomous driving module 125 may be operably connected to communicate with various vehicle systems 109 and / or their individual components. For example, the processor 101 and / or the autonomous driving module 125 may communicate to send and / or receive information from various vehicle systems 109 to control the navigation and / or steering of the vehicle 100. The processor 101, the avatar rendering system 126, and / or the autonomous driving module 125 may control some or all of these vehicle systems 109.

[0105] For example, when operating in autonomous mode, the processor 101 and / or the autonomous driving module 125 control the direction and speed of the vehicle 100. The processor 101 and / or the autonomous driving module 125 accelerate the vehicle 100 (for example, by increasing the amount of energy / fuel supplied to the motors), decelerate it (for example, by applying the brakes), and / or change direction (for example, by steering the two front wheels). As used herein, “cause” or “causing” means to cause, compel, force, direct, command, guide, and / or enable an event or action to occur in a direct or indirect manner.

[0106] As shown, in at least one configuration, the vehicle 100 includes one or more actuators 117. The actuators 117 are operable elements that move and / or control one or more of the vehicle systems 109 or mechanisms such as components thereof in response to electronic signals or other inputs from, for example, the processor 101 and / or the autonomous driving module 125. The one or more actuators 117 may include motors, pneumatic actuators, hydraulic pistons, relays, solenoids, piezoelectric actuators, and / or other forms of actuators that generate the desired control.

[0107] As described above, the vehicle 100 may include one or more modules, at least some of which are described herein. In at least one configuration, a module is implemented as non-transient computer-readable instructions that, when executed by the processor 101, perform one or more of the various functions described herein. In various configurations, one or more of the modules are components of the processor 101, or one or more of the modules run on and / or are distributed between other processing systems to which the processor 101 is operablely connected. Alternatively or in addition, one or more modules are implemented at least partially in hardware. For example, one or more modules include a combination of logic gates (e.g., metal-oxide-semiconductor field-effect transistors (MOSFETs)), ASICs, programmable logic arrays (PLAs), field-programmable gate arrays (FPGAs), and / or other electronic hardware-based implementations configured to perform the functions described herein. Furthermore, in one or more configurations, one or more of the modules may be distributed among multiple modules described herein. In one or more configurations, two or more of the modules described herein may be combined into a single module.

[0108] Furthermore, the vehicle 100 includes one or more autonomous driving modules 125. The autonomous driving modules 125 receive data from the sensor system 102 and / or other systems associated with the vehicle 100 in at least one manner. In one or more configurations, the autonomous driving modules 125 use such data to perceive the surrounding environment of the vehicle. The autonomous driving modules 125 determine the position of the vehicle 100 in the surrounding environment and map the characteristics of the surrounding environment. For example, the autonomous driving modules 125 determine the location of obstacles or other environmental features, including traffic signs, trees, shrubs, surrounding vehicles, pedestrians, etc.

[0109] The autonomous driving module 125 may be configured to determine the driving path, the current autonomous driving operation of the vehicle 100, future autonomous driving operations, and / or modifications to the current autonomous driving operation based on data acquired by the sensor system 102 and / or other sources. Generally, the autonomous driving module 125 functions to implement various levels of autonomous driving, including, for example, advanced driver-assistance system (ADAS) functions, semi-autonomous driving functions, and fully autonomous driving functions, as described above.

[0110] Detailed embodiments are disclosed herein. However, it should be understood that the disclosed embodiments are intended to be illustrative only. For this reason, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as the basis for the claims and as representative grounds for teaching those skilled in the art to apply the embodiments described herein in a variety of substantially any suitable detailed structures. Furthermore, the terms and phrases used herein are not intended to be limiting, but rather to provide an understandable description of possible implementations. Various embodiments are shown in Figures 1 to 10, but these embodiments are not limited to the illustrated structures or uses.

[0111] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in a flowchart or block diagram represents a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. Note that in some alternative embodiments, the functions described within a block may occur in a different order than that shown in the diagram. For example, two consecutively shown blocks may actually be executed almost simultaneously, or these blocks may be executed in reverse order depending on the functions they relate to.

[0112] The systems, components, and / or processes described above can be implemented in hardware or a combination of hardware and software, and can be implemented in a centralized manner in a single processing system or in a distributed manner in which various elements are spread across several interconnected processing systems. The systems, components, and / or processes can also be embedded in computer-readable storage, such as a machine-readable computer program product or other data program storage device, which concretely embodies a program of machine-executable instructions for performing the methods and processes described herein. These elements can also be embedded in an application product that includes features enabling the implementation of the methods described herein and, when loaded into a processing system, can perform these methods.

[0113] Furthermore, the configurations described herein may take the form of a computer program product embodied in one or more computer-readable media, for example, stored in which computer-readable program code is embodied. Any combination of one or more computer-readable media may be used. The term "computer-readable storage medium" means a non-temporary storage medium. A computer-readable storage medium is, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, equipment, or device, or any suitable combination thereof. A non-exclusive list of computer-readable storage media may include portable computer diskettes, hard disk drives (HDDs), solid-state drives (SSDs), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), optical storage devices, magnetic storage devices, or any combination thereof. In the context of this document, a computer-readable storage medium is, for example, a tangible medium that stores a program used by or in connection with an instruction execution system, equipment, or device.

[0114] Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, cable, RF, or any suitable combination thereof. Computer program code for performing the operations of this configuration is written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java®, Smalltalk, C++, or their equivalents, and conventional procedural programming languages ​​such as the C programming language or a similar programming language. The program code may be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or fully on a remote computer or server. In the latter scenario, the remote computer is connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or to an external computer (e.g., through the Internet using an Internet Service Provider).

[0115] As used herein, the articles "a" and "an" are defined as one or more. As used herein, the term "plural" is defined as two or more. As used herein, the term "another" is defined as at least two or more. As used herein, the terms "contain" and / or "have" are defined as having (i.e., open-language). As used herein, the phrase "at least one of" refers to and encompasses any possible combination of one or more of the relevant list items. For example, the phrase "at least one of A, B, and C" includes A only, B only, C only, or any combination thereof (e.g., AB, AC, BC, or ABC).

[0116] The embodiments described herein may be embodied in other forms without departing from these ideas or essential attributes. Accordingly, the following claims, rather than the above specification, should be used to illustrate the scope of the invention.

Claims

1. Processor and Memory for storing machine-readable instructions and A system equipped with, When the machine-readable instruction is executed by the processor, the processor will be instructed to: The physical characteristics of the vehicle occupants are extracted from the vehicle capture images of the occupants. Based on the extracted physical characteristics of the crew member, a basic digital avatar of the crew member is rendered. The characteristics of an object in the external environment of the vehicle are extracted from the vehicle-captured image of the object. By transferring the features of the object onto the basic digital avatar of the crew member, a presentation digital avatar similar to the crew member is rendered. A system for animating the digital avatar for presentation on the display device of the vehicle.

2. The aforementioned machine-readable instruction includes further machine-readable instructions. When the aforementioned further machine-readable instruction is executed by the processor, the processor will be instructed to: The actions of the occupant that identify the object and the position of the object are detected. The system according to claim 1, wherein an image captured by a vehicle camera having a field of view that overlaps with the position of the aforementioned object is acquired.

3. The aforementioned machine-readable instruction includes further machine-readable instructions. When the aforementioned further machine-readable instruction is executed by the processor, the processor will be instructed to: Determine the level of detail of the object features in the vehicle capture image. Depending on whether the level of detail is below a threshold amount, From the vehicle capture image log, previously captured images of the object are identified. The system according to claim 1, which extracts features of the object from previously captured images of the object.

4. The aforementioned machine-readable instruction includes further machine-readable instructions. When the aforementioned further machine-readable instruction is executed by the processor, the processor will be instructed to: The vehicle camera receives the vehicle capture image of the object, From the vehicle capture image, the distinctive features of the object are extracted. Based on the aforementioned distinctive features, the object can be identified in other images within the corpus of digital content, and The system according to claim 1, wherein a neural network trained to render a digital model of the object is deployed by combining the vehicle capture image of the object with the other images of the object.

5. The system according to claim 4, wherein the machine-readable instructions causing the processor to deploy the trained neural network include machine-readable instructions causing the processor to infer features of the object that have been omitted from the vehicle capture image of the object and the other images of the object.

6. The aforementioned machine-readable instruction includes further machine-readable instructions. When the aforementioned further machine-readable instructions are executed by the processor, the processor causes the processor to deploy a neural network. To identify the definitive visual features of the aforementioned object, Based on the definitive visual characteristics of the aforementioned object, the theme of the presentation digital avatar is constructed. The machine-readable instruction causing the processor to render the digital avatar for presentation that resembles the occupant is, The system according to claim 1, comprising a machine-readable command to deploy the neural network and transfer the visual features defined by the theme onto the crew member's basic digital avatar.

7. The machine-readable instruction that causes the processor to deploy the neural network causes the processor to deploy the neural network, Identify other images in a corpus of digital content that have the same or similar definitive visual features as the aforementioned object. The system according to claim 6, comprising machine-readable instructions for constructing the theme by aggregating the definitive visual features of the object and the other images.

8. The machine-readable instruction that causes the processor to deploy the neural network causes the processor to deploy the neural network, Based on the definitive visual characteristics of the object, construct an auditory theme for the presentation digital avatar. The system according to claim 6, comprising a machine-readable command to output sound related to the presentation digital avatar based on the auditory theme.

9. A non-temporary machine-readable medium containing instructions, When the aforementioned instruction is executed by the processor, the processor will: The physical characteristics of the vehicle occupants are extracted from the vehicle capture images of the occupants. Based on the extracted physical characteristics of the crew member, a basic digital avatar of the crew member is rendered. The characteristics of an object in the external environment of the vehicle are extracted from the vehicle-captured image of the object. By transferring the features of the object onto the basic digital avatar of the crew member, a presentation digital avatar similar to the crew member is rendered. A non-temporary machine-readable medium for animating the digital avatar for presentation on the display device of the vehicle.

10. The machine-readable medium includes further instructions, When the aforementioned further instruction is executed by the processor, the processor will: The actions of the occupant that identify the object and the position of the object are detected. A non-temporary machine-readable medium according to claim 9, which allows an image captured by a vehicle camera having a field of view overlapping with the position of the object to be acquired.

11. The machine-readable medium includes further instructions, When the aforementioned further instruction is executed by the processor, the processor will: Determine the level of detail of the object features in the vehicle capture image. Depending on whether the level of detail is below a threshold amount, From the vehicle capture image log, previously captured images of the object are identified. A non-temporary machine-readable medium according to claim 9, which extracts features of the object from a previously captured image of the object.

12. The aforementioned non-temporary machine-readable medium includes further instructions, When the aforementioned further instruction is executed by the processor, the processor will: The vehicle camera receives the vehicle capture image of the object, From the vehicle capture image, the distinctive features of the object are extracted. Based on the aforementioned distinctive features, the object can be identified in other images within the corpus of digital content, and A non-temporary machine-readable medium according to claim 9, wherein a neural network trained to render a digital model of the object is deployed by combining the vehicle capture image of the object with the other images of the object.

13. The machine-readable medium includes further instructions, When the aforementioned further instruction is executed by the processor, it causes the processor to deploy a neural network. To identify the definitive visual features of the aforementioned object, Based on the definitive visual characteristics of the aforementioned object, the theme of the presentation digital avatar is constructed. The instruction causing the processor to render the presentation digital avatar similar to the occupant is given to the processor, A non-temporary machine-readable medium according to claim 9, comprising an instruction to deploy the neural network and transfer the visual features defined by the theme onto the crew member's basic digital avatar.

14. The instruction that causes the processor to deploy the neural network causes the processor to deploy the neural network, Identify other images in a corpus of digital content that have the same or similar definitive visual features as the aforementioned object. A non-temporary machine-readable medium according to claim 13, comprising instructions for constructing the theme by aggregating the definitive visual features of the object and the other images.

15. Extracting the physical characteristics of the vehicle occupants from the vehicle capture images of the occupants, Rendering a basic digital avatar of the crew member based on the extracted physical characteristics of the crew member, Extracting the characteristics of objects in the external environment of the vehicle from the vehicle-captured image of the object, Rendering a presentation digital avatar similar to the occupant by transferring the features of the object onto the occupant's basic digital avatar, Animating the digital avatar for presentation on the vehicle's display device. Methods that include...

16. To detect the actions of the occupant that identify the object and the position of the object, To acquire an image captured by a vehicle camera having a field of view that overlaps with the position of the aforementioned object. The method according to claim 15, further comprising:

17. To determine the level of detail of the object features in the vehicle capture image, Depending on whether the level of detail is below a threshold amount, From the vehicle capture image log, previously captured images of the object can be identified, Extracting features of the object from previously captured images of the object. The method according to claim 15, further comprising:

18. Deploy a neural network, The vehicle camera receives the vehicle capture image of the object, From the vehicle capture image, the distinctive features of the object are extracted. Based on the aforementioned distinctive features, the object can be identified in other images within the corpus of digital content. The method according to claim 15, further comprising rendering a digital model of the object by combining the vehicle capture image of the object with the other images of the object.

19. Deploy a neural network, Identify the definitive visual features of the aforementioned object, The further includes constructing a theme for the presentation digital avatar based on the definitive visual features of the object, The method according to claim 15, wherein rendering the presentation digital avatar similar to the occupant comprises deploying the neural network to transfer the visual features defined by the theme onto the occupant's base digital avatar.

20. Deploying the aforementioned neural network means deploying the aforementioned neural network, Identify other images in a corpus of digital content that have the same or similar definitive visual features as the aforementioned object. The method according to claim 19, further comprising aggregating the definitive visual features of the object and the other images to construct the theme.

Images