Emotional interaction method and device based on vehicle-mounted scene and vehicle

Abstract

The application discloses a kind of based on vehicle scene emotional interaction method, device and vehicle, it is related to emotional interaction technical field.The method thereof includes: in response to the emotional interaction trigger event generated in the vehicle cabin, the object model associated with target real object is acquired;Object model is constructed based on the multimedia data representing target real object;Based on emotional interaction trigger event, determine the emotional interaction action and emotional interaction expression of object model corresponding to be triggered;In the vehicle cabin, the emotional interaction content corresponding to object model is presented;In emotional interaction content, object model shows emotional interaction expression and executes emotional interaction action.In this way, the emotional interaction of user and exclusive object can be realized.

Description

Technical Field

[0001] This application relates to the field of emotional interaction technology, and in particular to an emotional interaction method, device and vehicle based on in-vehicle scenarios. Background Technology

[0002] With the rapid development of vehicle technology, the integrated functions of in-vehicle smart cockpits are constantly being enriched. Pets have become emotional companions for many users, so enabling interaction between users and pets within the vehicle cockpit is an important function that vehicle cockpits need to possess.

[0003] Currently, user-pet interaction is mainly achieved by triggering the in-vehicle pet's response through vehicle cabin information. In this method, the pet is generally the same, meaning that different users interact with the same pet.

[0004] Therefore, how to achieve emotional interaction between users and their specific objects has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides an emotional interaction method, device, and vehicle based on an in-vehicle scenario, which can realize emotional interaction between users and dedicated objects.

[0006] In a first aspect, embodiments of this application provide an emotion interaction method based on an in-vehicle scenario, the method comprising:

[0007] In response to emotional interaction trigger events generated within the vehicle cabin, an object model associated with the target real object is obtained; wherein, the object model is constructed based on multimedia data representing the target real object;

[0008] Based on the emotional interaction triggering event, determine the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model;

[0009] The system presents emotional interaction content corresponding to the object model within the vehicle cabin; within this emotional interaction content, the object model displays emotional expressions and performs emotional interaction actions.

[0010] By adopting the embodiments of this application, an object model is constructed by acquiring multimedia data of real objects that the user already has emotions, and emotional interaction is realized based on the object model. In this way, an exclusive emotional space with exclusive objects can be created, thereby realizing emotional interaction between the user and exclusive objects.

[0011] In an optional implementation of the first aspect, the method further includes: obtaining the target skeleton rotation angle corresponding to the emotional interaction action based on a first correspondence; wherein the first correspondence includes the correspondence between multiple action sequences of the target real object and multiple skeleton rotation angles; and controlling the object model based on the target skeleton rotation angle so that the object model performs the emotional interaction action.

[0012] By adopting this implementation method, the target bone rotation angle corresponding to the emotional interaction action can be quickly determined by a first correspondence based on the correspondence between multiple action sequences and multiple bone rotation angles of the target real object, thereby improving the efficiency of the object model in performing emotional interaction actions.

[0013] In one optional implementation of the first aspect, presenting emotional interaction content corresponding to an object model in a vehicle includes: obtaining target voiceprint information corresponding to a target real object based on a second correspondence; the second correspondence includes correspondences between multiple real objects and multiple voiceprint information; generating emotional interaction content corresponding to the object model based on the target voiceprint information, emotional interaction actions, and emotional interaction expressions; the emotional interaction content includes emotional interaction audio and emotional interaction visuals; displaying the emotional interaction visuals in the vehicle and playing the emotional interaction audio.

[0014] By adopting this implementation method, by introducing the target voiceprint information corresponding to the real object, the emotional interaction content can include not only emotional interaction images but also emotional interaction audio, thereby making the emotional interaction more exclusive and interactive.

[0015] In one optional implementation of the first aspect, determining the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model based on the emotional interaction triggering event includes: obtaining the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model and associated with the emotional interaction triggering event based on a third correspondence relationship; the third correspondence relationship includes the correspondence relationship between multiple emotional interaction triggering events and multiple combinations, and each combination includes the emotional interaction action and emotional interaction expression to be triggered.

[0016] This implementation method can quickly determine the emotional interaction actions and expressions to be triggered for the object model, thereby improving the efficiency of emotional interaction.

[0017] In one optional implementation of the first aspect, the object model is constructed as follows: acquiring multimedia data collected from multiple angles of the target real object, and acquiring images of the target real object from each angle based on the multimedia data; segmenting the contour data of the target real object from each image to obtain a contour dataset, and determining the point cloud data corresponding to the target real object based on the contour dataset; determining the appearance morphology pixels of the target real object based on each image, and generating an appearance morphology map of the target real object based on the features of each pixel in the appearance morphology pixels; generating a normal map and a roughness map corresponding to the target real object based on the appearance morphology map; and constructing an object model associated with the target real object based on the point cloud data, the normal map, and the roughness map.

[0018] This implementation method has two advantages. First, by utilizing the contour dataset corresponding to the target real object at multiple angles based on multimedia data, the point cloud data of the target real object can be determined, and an object model can be constructed based on the point cloud data. This makes the constructed object model more consistent with the target real object. Second, by introducing the appearance shape pixels corresponding to the target real object, the normal map and roughness map corresponding to the target real object can be determined, and an object model corresponding to the target real object can be constructed based on the normal map, roughness map, and point cloud data. This improves the realism of the constructed object model.

[0019] In an optional implementation of the first aspect, the method further includes: when the emotional interaction triggering event is a touch event, converting the two-dimensional coordinates of the touch position corresponding to the touch event into a three-dimensional spatial ray; performing collision detection based on the three-dimensional spatial ray and the emotional interaction content to obtain a collision detection result; and when the collision detection result indicates that the three-dimensional spatial ray did not hit the emotional interaction content, adjusting the presentation position of the emotional interaction content based on a preset distance.

[0020] This implementation method can improve visual rationality, thereby enhancing the emotional interaction experience between the user and the specific object.

[0021] In an optional implementation of the first aspect, the method further includes: using the refresh cycle of the screen used to display the emotional interaction content as a time reference, performing time axis interpolation processing on keyframes in the emotional interaction content to generate target rotation values ​​and target positions; wherein the target rotation value is used to characterize the orientation of the object model in the emotional interaction content in three-dimensional space, and the target position is used to characterize the position of the root skeleton of the object model in three-dimensional space; based on the target position, determining the horizontal position of the root skeleton of the emotional interaction object model; and based on the target rotation value and the horizontal position, performing anti-clipping processing on the emotional interaction content to obtain the processed emotional interaction content.

[0022] By adopting this implementation method, clipping of emotional interaction content can be avoided as much as possible.

[0023] In an optional implementation of the first aspect, the method further includes: determining the ambient light intensity inside the vehicle's cabin and mapping the ambient light intensity to the brightness of the main light source; and adjusting the brightness inside the cabin based on the brightness of the main light source and the brightness of the fill light, when the brightness inside the cabin meets the brightness adjustment conditions, so as to adjust the presentation effect of the emotional interactive content in the vehicle.

[0024] By adopting this implementation method, the presentation effect of emotional interactive content can be improved by adjusting the brightness inside the cockpit.

[0025] Secondly, embodiments of this application provide an emotion interaction device based on an in-vehicle scenario, the device comprising:

[0026] The acquisition module is used to acquire an object model that is relatively related to the target in response to emotional interaction trigger events generated in the vehicle cabin; wherein, the object model is constructed based on multimedia data representing the target real object;

[0027] The determination module is used to determine the corresponding emotional interaction action and emotional interaction expression to be triggered based on the emotional interaction trigger event;

[0028] The presentation module is used to present the emotional interaction content corresponding to the object model in the vehicle cabin; in the emotional interaction content, the object model displays emotional interaction expressions and performs emotional interaction actions.

[0029] Thirdly, embodiments of this application provide a vehicle including a memory and a controller. The memory stores a computer program, and the controller executes the computer program to implement the steps of the method provided in the first aspect.

[0030] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an in-vehicle terminal device, implements the steps of the method provided in the first aspect above.

[0031] Fifthly, this application also provides a computer program product, including a computer program that, when executed by an in-vehicle terminal device, implements the steps of the method provided in the first aspect above.

[0032] Regarding the beneficial effects of any of the technical solutions in the second to fifth aspects mentioned above, refer to the beneficial effects of the corresponding technical solutions in the first aspect; repeated examples will not be listed here. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is an optional flowchart illustrating an emotional interaction method based on an in-vehicle scenario provided in an embodiment of this application.

[0035] Figure 2 This is another optional flowchart illustrating an emotional interaction method based on an in-vehicle scenario provided in an embodiment of this application;

[0036] Figure 3 This is a schematic diagram of an optional structure of an emotional interaction device based on an in-vehicle scenario provided in an embodiment of this application;

[0037] Figure 4 This is an optional structural schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] With the rapid development of vehicle technology, the integrated functions of in-vehicle smart cockpits are constantly being enriched. Pets have become emotional companions for many users, so enabling interaction between users and pets within the vehicle cockpit is an important function that vehicle cockpits must possess. Currently, user-pet interaction is mainly achieved by triggering responses from the in-vehicle pet based on vehicle cockpit information. In this method, the pet is generally the same, meaning that different users interact with the same pet.

[0040] To address the aforementioned issues, this application provides an emotional interaction method, device, and vehicle based on an in-vehicle scenario. The method includes: responding to an emotional interaction trigger event generated within the vehicle cabin; acquiring an object model associated with the target real object, constructed based on multimedia data representing the target real object; determining the emotional interaction action and emotional expression to be triggered corresponding to the object model based on the emotional interaction trigger event; presenting the emotional interaction content corresponding to the object model in the vehicle cabin; and in the emotional interaction content, the object model displays emotional expressions and performs emotional interaction actions. By employing this method, an object model constructed from multimedia data of a real object representing a user's existing emotions is created, and emotional interaction is achieved based on this object model. This allows for the creation of a personalized emotional space with a unique object, thereby enabling emotional interaction between the user and their personalized object.

[0041] The following describes the emotion interaction method based on vehicle scenarios provided in the embodiments of this application.

[0042] Please see Figure 1 , Figure 1 This is an optional flowchart illustrating an emotion-based interaction method for in-vehicle scenarios provided in this application embodiment. The method can be applied to vehicles or systems including vehicles and electronic devices, and is implemented through the interaction between the vehicle and electronic devices. For ease of explanation, this application embodiment uses the application of the method to a vehicle, executed by a controller within the vehicle, as an example. Figure 1 As shown, this emotion-based interaction method for in-vehicle scenarios may include, but is not limited to, the following steps:

[0043] S101. In response to emotional interaction triggering events generated in the vehicle cabin, obtain an object model associated with the target real object; the object model is constructed based on multimedia data representing the target real object.

[0044] Emotional interaction triggering events refer to specific, identifiable events that occur in a particular time and space and are used for emotional interaction. These events can be triggered by the user or by the controller based on vehicle driving status information, etc., without limitation. User-triggered emotional interaction events can include, but are not limited to, touch events and voice events. Touch events include, for example, a user tapping the screen (such as the central control screen), while voice events include, for example, a user calling out the name of the object to be interacted with, or outputting a command to instruct the object to perform a certain action.

[0045] In one alternative implementation, the controller, in response to an emotional interaction triggering event generated within the vehicle cabin, obtains an object model associated with the target real object. This can be achieved by: in response to an emotional interaction triggering event generated within the vehicle cabin, determining whether the emotional interaction triggering event meets the emotional interaction triggering conditions; if so, obtaining the object model associated with the target real object.

[0046] Emotional interaction triggering conditions refer to behaviors or signals that can initiate, alter, or deepen emotional responses and connections during human-computer interaction. These triggering conditions may include, but are not limited to, user touch input on the screen being any of a variety of preset touch operations, or the vehicle controller determining, based on vehicle driving status information, that the user's current driving time has reached a preset time threshold.

[0047] For example, suppose the emotional interaction triggering condition includes the controller in the vehicle determining that the user has been driving the vehicle for 2 hours based on the vehicle's driving status information, and suppose the controller determines that the user has been driving the vehicle for exactly 2 hours based on the vehicle's driving status information. In this case, the controller can determine that the emotional interaction triggering event meets the emotional interaction triggering condition, and at this time, the controller can obtain the object model.

[0048] For example, suppose the emotional interaction triggering conditions include a click operation on the central control screen, and suppose the user has clicked the central control screen. In this case, the controller can determine that the emotional interaction triggering event meets the emotional interaction triggering conditions, and the controller can obtain the object model.

[0049] The target real object can be a person, a pet, etc., and there is no restriction here.

[0050] Multimedia data may include, but is not limited to, video data, multiple image data in consecutive or non-consecutive frames, etc. Multimedia data may be acquired periodically by the controller in the vehicle (which can be referred to as historical multimedia data) or acquired in real time (which can be referred to as real-time multimedia data), etc., without limitation here.

[0051] S102. Based on the emotional interaction trigger event, determine the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model.

[0052] Among them, emotional interaction actions refer to the sequence of physical behaviors performed by the object model through body movement, posture changes, etc., which have clear social meaning and emotional communication purpose.

[0053] Among them, emotional interaction expressions refer to the patterned visual symbols displayed by the object model through its face that conform to the user's expectations of biological emotional expression.

[0054] S103. Present the emotional interaction content corresponding to the object model in the vehicle cabin; in the emotional interaction content, the object model displays emotional interaction expressions and performs emotional interaction actions.

[0055] The emotional interaction content corresponding to the object model can be displayed on the vehicle's screen (such as the central control screen or the passenger screen), or it can be projected onto the vehicle's windshield through the head-up display (HUD) function, etc. There are no restrictions here.

[0056] In this embodiment, in response to an emotional interaction trigger event generated within the vehicle cabin, an object model associated with the target real object is acquired, constructed based on multimedia data representing the target real object. Based on the emotional interaction trigger event, the emotional interaction actions and expressions to be triggered corresponding to the object model are determined. The emotional interaction content corresponding to the object model is presented in the vehicle cabin. Within the emotional interaction content, the object model displays emotional expressions and performs emotional interaction actions. By employing this method, an object model is constructed from multimedia data of the user's existing emotional real object, and emotional interaction is realized based on this object model. This allows for the creation of a personalized emotional space with a unique object, thereby enabling emotional interaction between the user and their personalized object.

[0057] In one alternative implementation, Figure 1 In the emotional interaction method based on the vehicle scene shown, the controller can also obtain the target skeleton rotation angle corresponding to the emotional interaction action based on the first correspondence relationship; wherein, the first correspondence relationship includes the correspondence between multiple action sequences of the target real object and multiple skeleton rotation angles; the object model is controlled based on the target skeleton rotation angle so that the object model performs the emotional interaction action.

[0058] The bone rotation angle refers to the Euler angle or quaternion value of the virtual bone of the driving object model to rotate around its local coordinate axes (usually X, Y, Z axes). The X axis usually points to the side of the bone (e.g., from left to right), the Y axis usually points to the top of the bone (e.g., from top to bottom), and the Z axis usually points to the front of the bone (e.g., from inside to outside).

[0059] In some embodiments, the first correspondence may be a table stored in the controller (denoted as the first correspondence table), or a table stored in an electronic device that the controller can read (denoted as the first correspondence table), etc., without limitation here. The first correspondence table includes the correspondence between multiple action sequences of the real object and multiple skeletal rotation angles.

[0060] By adopting this implementation method, the target bone rotation angle corresponding to the emotional interaction action can be quickly determined by a first correspondence based on the correspondence between multiple action sequences and multiple bone rotation angles of the target real object, thereby improving the efficiency of the object model in performing emotional interaction actions.

[0061] In one alternative implementation, Figure 1 Step S103 in the in-vehicle scenario-based emotional interaction method, namely the way the controller presents the emotional interaction content corresponding to the object model in the vehicle, can be: obtaining the target voiceprint information corresponding to the target real object based on the second correspondence; the second correspondence includes the correspondence between multiple real objects and multiple voiceprint information; generating the emotional interaction content corresponding to the object model based on the target voiceprint information, emotional interaction actions, and emotional interaction expressions; the emotional interaction content includes emotional interaction audio and emotional interaction visuals; displaying the emotional interaction visuals in the vehicle and playing the emotional interaction audio.

[0062] In some embodiments, the second correspondence may be a table stored in the controller (denoted as the second correspondence table), or a table stored in an electronic device that the controller can read (denoted as the second correspondence table), etc., without limitation here. The second correspondence table includes correspondences between various real objects and various voiceprint information.

[0063] Optionally, the second correspondence can be constructed by the controller based on the voiceprint information corresponding to different real objects collected in advance.

[0064] In some embodiments, the controller generates emotional interaction content corresponding to the object model based on the target voiceprint information, emotional interaction actions, and emotional interaction expressions. This can be achieved by: generating emotional interaction audio based on the target voice information; generating emotional interaction visuals based on the emotional interaction actions and emotional interaction expressions; and performing audio-visual synthesis processing on the emotional interaction audio and emotional interaction visuals to obtain the emotional interaction content corresponding to the object model.

[0065] By adopting this implementation method, by introducing the target voiceprint information corresponding to the real object, the emotional interaction content can include not only emotional interaction images but also emotional interaction audio, thereby making the emotional interaction more exclusive and interactive.

[0066] In one alternative implementation, Figure 1Step S102 in the in-vehicle scenario-based emotional interaction method shown is the way the controller determines the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model based on the emotional interaction trigger event. This can be: based on a third correspondence, obtaining the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model associated with the emotional interaction trigger event; the third correspondence includes the correspondence between multiple emotional interaction trigger events and multiple combinations, and each combination includes the emotional interaction action and emotional interaction expression to be triggered.

[0067] In some embodiments, the third correspondence may be a table stored in the controller (denoted as the third correspondence table), or a table stored in an electronic device that the controller can read (denoted as the third correspondence table), etc., without limitation here. The third correspondence table includes the correspondence between multiple emotional interaction trigger events and multiple combinations, where each combination includes an emotional interaction action to be triggered and an emotional interaction expression to be triggered. For example, the third correspondence table may be shown in Table 1 below.

[0068] Table 1

[0069]

[0070] This implementation method can quickly determine the emotional interaction actions and expressions to be triggered for the object model, thereby improving the efficiency of emotional interaction.

[0071] In one alternative implementation, Figure 1 In the emotional interaction method based on the in-vehicle scenario shown, the object model can be constructed by the controller in the following way: acquiring multimedia data collected from multiple angles of the target real object, and acquiring images of the target real object from multiple angles based on the multimedia data; segmenting the contour data of the target real object from each image to obtain a contour dataset, and determining the point cloud data corresponding to the target real object based on the contour dataset; determining the appearance shape pixels of the target real object based on each image, and generating the appearance shape map of the target real object based on the features of each pixel in the appearance shape pixels; generating the normal map and roughness map corresponding to the target real object based on the appearance shape map; and constructing an object model associated with the target real object based on the point cloud data, normal map, and roughness map.

[0072] Multimedia data may include, but is not limited to, video data, multiple image data in consecutive or non-consecutive frames, etc.

[0073] In some embodiments, the controller acquires images of the target real object from multiple angles based on multimedia data. This can be achieved by segmenting and extracting images of the target real object from multiple angles, excluding other information, from the various data included in the multimedia data. Here, "other information" refers to information that does not belong to the target real object. For example, assuming the multimedia data includes Image 1, which captures the target real object and includes the real object, a tree, a flower, and a watermark, then the "other information" would be the tree, the flower, and the watermark.

[0074] In some embodiments, the controller segments the contour data of the target real object from each image to obtain a contour dataset. This can be achieved by: performing foreground segmentation on each image to obtain the contour data of the target real object in each image; and constructing a contour dataset based on the contour data of the target real object in each image. Optionally, the contour data can be the front, side, and back views corresponding to the real object.

[0075] In some embodiments, the controller determines the point cloud data corresponding to the target real object based on the contour dataset. This can be achieved by converting the two-dimensional coordinates corresponding to each contour data in the contour dataset into three-dimensional coordinates through triangulation to obtain the point cloud data corresponding to the target real object.

[0076] In the case where the target real object is a real pet, the appearance pixels may include, but are not limited to, pixels such as fur color, pattern, and beak shape; in the case where the real object is a person, the appearance pixels may include pixels such as hair color and skin color.

[0077] Among them, the appearance morphology diagram is usually an RGB color image (RGB is an abbreviation for Red, Green and Blue), and the appearance morphology diagram may include, but is not limited to, information such as shadows, highlights, and textures.

[0078] The normal map is a special RGB image in which the color value (R, G, B) of each pixel is used to encode the normal direction of the surface of the model (object model) corresponding to that pixel; the roughness map is a single-channel grayscale image in which the grayscale value of each pixel defines the micro-roughness of the corresponding surface area.

[0079] In some embodiments, the controller generates a normal map and a roughness map corresponding to the target real object based on the appearance shape map. This can be achieved by: performing delighting or deshading processing on the appearance shape map, and extracting features from the processed appearance shape map to obtain geometric cue features and material cue features corresponding to the appearance shape map; assigning a three-dimensional normal vector to each pixel based on the geometric cue features to obtain the normal map corresponding to the target real object; and generating a roughness map corresponding to the target real object based on the material cue features.

[0080] Among them, geometric cue features may include, but are not limited to, gradient changes (color abrupt changes may be edges or bumps), shadows (suggesting depressions), highlights (suggesting convexity and smoothness), perspective and texture gradients (such as the deformation of brick wall textures suggesting surface orientation), etc.; material cue features may include, but are not limited to, the sharpness and blurriness of highlights (directly related to roughness), reflection blurriness, color saturation changes, etc.

[0081] In some embodiments, the controller constructs an object model associated with the target real object based on point cloud data, normal maps, and roughness maps. This can be achieved by: dividing the point cloud data corresponding to the target real object into blocks based on the body parts of the target real object to obtain multiple block point cloud data; performing hole-filling and smoothing processing on each block point cloud data in the multiple block point cloud data to obtain a closed mesh; determining the initial object model corresponding to the target real object based on the closed mesh; and constructing an object model associated with the target real object based on the initial object model, normal maps, and roughness maps.

[0082] This implementation method has two advantages. First, by utilizing the contour dataset corresponding to the target real object at multiple angles, based on multimedia data of the target real object, the point cloud data of the target real object is determined, and an object model is constructed based on the point cloud data. This makes the constructed object model more consistent with the target real object. Second, by introducing the appearance shape pixels corresponding to the target real object, the normal map and roughness map corresponding to the target real object can be determined, and an object model corresponding to the target real object is constructed based on the normal map, roughness map, and point cloud data. This improves the realism of the constructed object model.

[0083] In one alternative implementation, Figure 1 In the emotional interaction method based on the vehicle scene shown, the controller can also convert the two-dimensional coordinates of the touch position corresponding to the touch event into a three-dimensional spatial ray when the emotional interaction trigger event is a touch event; perform collision detection based on the three-dimensional spatial ray and the emotional interaction content to obtain the collision detection result; if the collision detection result indicates that the three-dimensional spatial ray does not hit the emotional interaction content, adjust the presentation position of the emotional interaction content based on a preset distance.

[0084] In some embodiments, the preset distance can be in the range of 4cm-6cm.

[0085] For example, assuming the real object is a real pet, the object model is a pet model; assuming the event type of the emotional interaction trigger event is a touch event, the controller can convert the two-dimensional coordinates of the touch position into a three-dimensional spatial ray, and perform collision detection between the three-dimensional spatial ray and the tip of the pet model's nose included in the emotional interaction content; if the three-dimensional spatial ray does not hit the tip of the pet model's nose, the controller will automatically shift the presentation position of the emotional interaction content by a preset distance so that the tip of the pet model's nose is close to the touch position.

[0086] This implementation method can improve visual rationality, thereby enhancing the emotional interaction experience between the user and the specific object.

[0087] In one alternative implementation, Figure 1 In the emotional interaction method based on the vehicle scene shown, the controller can also use the refresh cycle of the screen used to display the emotional interaction content as a time reference to perform time axis interpolation processing on the keyframes in the emotional interaction content to generate target rotation values ​​and target positions. Among them, the target rotation value is used to characterize the orientation of the object model in the emotional interaction content in three-dimensional space, and the target position is used to characterize the position of the root skeleton of the object model in three-dimensional space. Based on the target position, the horizontal position of the root skeleton of the emotional interaction object model is determined. Based on the target rotation value and the horizontal position, anti-clipping processing is performed on the emotional interaction content to obtain the processed emotional interaction content.

[0088] In some embodiments, the refresh cycle of the screen used to display emotional interactive content is, for example, 14ms-18ms.

[0089] In some embodiments, the controller uses the refresh cycle of the screen used to display the emotional interactive content as a time reference to perform time axis interpolation processing on key frames in the emotional interactive content to generate smooth rotation values ​​and displacements. This can be achieved by using the refresh cycle of the screen used to display the emotional interactive content as a time reference to perform linear-Bezier hybrid interpolation processing on key frames in the emotional interactive content to obtain smooth rotation values ​​and displacements.

[0090] Optionally, the controller performs linear-Bezier hybrid interpolation on keyframes in the emotional interaction content to obtain the target rotation value. This can be achieved by: selecting two reference keyframes from the keyframe sequence of the emotional interaction content based on the current time; performing linear interpolation (such as quaternion linear interpolation) on the rotation values ​​of the two reference keyframes to obtain a first rotation value; the first rotation value is used to describe the initial rotation transition effect between the two reference keyframes; performing Bezier curve interpolation on the rotation values ​​of the two reference keyframes to obtain a second rotation value; the second rotation value is used to smooth the initial rotation transition effect; and performing a weighted summation of the first and second rotation values ​​to obtain the target rotation value.

[0091] Optionally, the controller performs linear-Bezier hybrid interpolation on the keyframes in the emotional interaction content to obtain the target displacement. This can be achieved by: performing linear interpolation (such as vector linear interpolation) on the positions of two consecutive reference keyframes to obtain the linear displacement between the two reference frames; constructing a Bezier curve and determining the curved displacement between the two reference frames based on the Bezier curve; and performing a weighted summation of the linear displacement and the curved displacement to obtain the target displacement between the two reference frames.

[0092] In some embodiments, the controller performs anti-mold processing on the emotional interaction content based on the target rotation value and horizontal displacement to obtain the processed emotional interaction content. This can be achieved by: determining whether the horizontal position is within a prohibited rendering area; if so, determining a safe horizontal position based on the horizontal position and the boundary position of the prohibited rendering area, and correcting the horizontal position based on the safe horizontal position to obtain the corrected horizontal position; and updating the pose of the object model in the emotional interaction content based on the corrected horizontal position, the target vertical position, and the target rotation value to obtain the processed emotional interaction content; wherein the target vertical position is determined based on the target position.

[0093] By adopting this implementation method, clipping of emotional interaction content can be avoided as much as possible.

[0094] In one alternative implementation, Figure 1 In the emotional interaction method based on the vehicle scene shown, the controller can also determine the ambient light intensity in the vehicle cabin and map the ambient light intensity to the brightness of the main light source; when the brightness in the cabin meets the brightness adjustment conditions, the brightness in the cabin is adjusted based on the brightness of the main light source and the brightness of the fill light to adjust the presentation effect of the emotional interaction content in the vehicle.

[0095] Ambient light intensity refers to the actual light level in the physical environment as measured by the in-vehicle light sensor, and is an objective physical quantity.

[0096] The brightness of the main light source refers to the intensity of light simulating the main natural light source (such as the sun or skylight) in a 3D rendered scene. It is a virtual parameter that has been artistically processed.

[0097] Fill brightness refers to the intensity of an auxiliary light source added specifically to enhance the visibility of a 3D model under low-light conditions; it is a completely artificially designed functional light source parameter.

[0098] In some embodiments, the controller determines the ambient light intensity inside the vehicle's cabin by: performing noise reduction and outlier removal processing on the light data collected by the light sensor inside the vehicle's cabin to obtain processed light data; and performing normalization processing on the processed light data to obtain the ambient light intensity.

[0099] In some embodiments, the brightness adjustment condition may be that the brightness inside the cabin is less than a preset brightness threshold. The preset brightness threshold may be set based on empirical values ​​or determined based on multiple experiments, etc., and is not limited here.

[0100] By adopting this implementation method, the presentation effect of emotional interactive content can be improved by adjusting the brightness inside the cockpit.

[0101] The following is combined with Figure 2 This application describes a method for emotional interaction based on an in-vehicle scenario, using a mobile terminal (hereinafter referred to as the mobile terminal) for controlling a vehicle, a vehicle, and electronic devices. Taking a real pet as an example, the method illustrates this approach. The electronic device can be a terminal or a server. Terminals mentioned here include, but are not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart TVs, smart air conditioners, smart in-vehicle devices, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server mentioned here can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services; no limitation is made here.

[0102] Please see Figure 2 , Figure 2 This is another optional flowchart illustrating an emotional interaction method based on an in-vehicle scenario provided in an embodiment of this application. For example... Figure 2 As shown, this emotion-based interaction method for in-vehicle scenarios may include, but is not limited to, the following steps:

[0103] S201. The mobile terminal acquires image data and voiceprint data of multiple real pets.

[0104] Among them, real pets include cats, dogs, birds, etc.

[0105] S202, The mobile terminal sends multiple images and voiceprints of real pets to the electronic device, and the electronic device receives the images and voiceprints of multiple real pets accordingly.

[0106] S203. The electronic device segments and extracts the image data for each real pet to obtain qualified images from different angles; wherein, the qualified image is an image of a real pet that does not include other information, and other information refers to information that does not belong to a real pet.

[0107] For example, assuming that the image data of a real pet includes a face, name, watermark, or a background with obvious people, the electronic device will crop out the face, name, watermark, or background with obvious people from the image data of the real pet, and only retain the target corresponding to the real pet.

[0108] Optionally, step S203 can be referred to as the image screening process.

[0109] S204. The electronic device performs foreground segmentation on qualified images from different angles to obtain the contour dataset corresponding to the real pet, and records the pixel appearance of the real pet.

[0110] Among them, the appearance pixels may include, but are not limited to, the pixels of real pets such as fur color, pattern, and beak shape.

[0111] Optionally, step S204 can be referred to as the contour extraction process.

[0112] S205. The electronic device performs three-dimensional mapping on each contour data in the contour dataset to obtain the point cloud data corresponding to the real pet.

[0113] S206. Electronic devices construct pet models corresponding to real pets based on the point cloud data of real pets.

[0114] In some embodiments, the electronic device constructs a pet model corresponding to a real pet based on the corresponding point cloud data of a real pet. This can be achieved by: dividing the point cloud data into blocks based on the body parts of the real pet to obtain multiple block point cloud data; performing hole-filling and smoothing processing on each block point cloud data to obtain a closed mesh; and setting a hierarchical skeleton inside the closed mesh to obtain a pet model corresponding to the real pet.

[0115] In the hierarchical skeleton, the parent node serves as the root skeleton, and the child nodes are arranged according to the joint positions of a real pet. The weights of the edges associated with each child node are determined based on the degree of influence that edge has on its neighboring nodes. This ensures that when the pet model's skeleton bends, the skin naturally follows the changes, thereby improving the rendering effect.

[0116] Optionally, step S206 can be referred to as the 3D point cloud reconstruction and skeleton binding process.

[0117] S207. The electronic device determines the correspondence between the action sequence of each real pet and the skeletal rotation angle based on the collected action sequence of each real pet, and obtains the first correspondence table.

[0118] For example, electronic devices can record a set of general action sequences for real pets such as cats, dogs, and birds, such as standing, sitting, lying down, sniffing, licking paws, wagging tails, spreading wings, curling up, sleeping, and yawning; and export the correspondence between the action sequences and the skeletal rotation angles frame by frame.

[0119] Optionally, step S207 can be referred to as the action template library modeling process.

[0120] S208, Electronic devices construct the target facial expressions of each realistic pet.

[0121] In some embodiments, the electronic device constructs the target facial expression form of each real pet by: constructing multiple initial facial expression forms for each real pet; generating intermediate facial expression forms corresponding to each initial facial expression form using interpolation; and generating the target facial expression form based on each initial facial expression form and the intermediate facial expression forms corresponding to each initial facial expression form.

[0122] For example, an electronic device can construct 8-12 extreme facial expression forms (initial expression forms) for a real pet's eyes, mouth, ears, and beak opening and closing, such as open eyes, closed eyes, open mouth, and closed mouth. Then, an interpolation method is used to generate intermediate expression forms, such as squinting eyes and half-open mouth. Finally, based on the initial and intermediate expression forms, the target expression form is generated. In this way, the micro-expressions of a real pet, such as blinking and opening its mouth, can be realized, which is beneficial for making the emotional interaction expressions of the pet model more natural during subsequent emotional interactions.

[0123] Optionally, step S208 can be referred to as the expression blending morphology construction process.

[0124] S209. The electronic device generates an appearance map corresponding to each real pet based on the features of each pixel in the real pet's appearance shape pixels, and generates a normal map and roughness map corresponding to the real pet based on the appearance map. This helps the fur of the pet model to have natural highlights and shadows under lighting.

[0125] Optionally, step S209 can be referred to as the material and texture packaging process.

[0126] S210. For each real pet, the electronic device divides the voiceprint data of the real pet into multiple voiceprint segments according to a preset duration, performs noise reduction processing on each voiceprint segment to obtain voiceprint information, and stores the correspondence between each real pet and each voiceprint information to obtain a second correspondence table.

[0127] The preset duration is, for example, 0.5s-2s.

[0128] Optionally, step S210 can be referred to as the voiceprint matching process.

[0129] S211. The electronic device constructs real pet data based on the pet model, target expression shape, first correspondence table, second correspondence table, normal map and roughness map corresponding to each real pet, and compresses and encrypts the real pet data to obtain compressed and encrypted real pet data.

[0130] S212. The electronic device sends compressed and encrypted real pet data to the vehicle, and the vehicle receives the compressed and encrypted real pet data accordingly.

[0131] S213. The vehicle decrypts and decompresses the compressed and encrypted real pet data to obtain real pet data. The real pet data includes the pet model, target expression shape, first correspondence table, second correspondence table, normal map and roughness map corresponding to each real pet.

[0132] S214. In response to emotional interaction trigger events generated in the vehicle cabin, obtain the target pet model from the pet models corresponding to each real pet.

[0133] In some embodiments, the vehicle obtains the target pet model from the pet models corresponding to each real pet, which may be: based on the identification information of the real pets included in the emotional interaction triggering event, obtain the target pet model that matches the identification information from the pet models corresponding to each real pet.

[0134] S215. Based on the emotional interaction trigger event, the vehicle determines the emotional interaction action and emotional interaction expression to be triggered corresponding to the target pet model, and obtains the target skeleton rotation angle corresponding to the emotional interaction action to be triggered based on the first correspondence table included in the real pet data, so as to control the target pet model to execute the emotional interaction action based on the target skeleton rotation angle.

[0135] In some embodiments, the vehicle determines the emotional interaction action to be triggered corresponding to the target pet model based on the emotional interaction trigger event. This can be done by: determining the emotional interaction action to be triggered corresponding to the target pet model from the pre-stored correspondence between multiple emotional interaction trigger events and multiple emotional interaction actions based on the emotional interaction trigger event.

[0136] In some embodiments, the vehicle determines the emotional interaction expression corresponding to the target pet model based on the sensory interaction trigger event. This can be achieved by: determining the emotional interaction expression corresponding to the target pet model that matches the emotional interaction trigger event from the target expression forms included in the real pet data.

[0137] S216. Based on the second correspondence table included in the real pet data, obtain the target voiceprint information of the real pet represented by the target pet model.

[0138] S217. Generate emotional interactive audio based on target voiceprint information, and generate emotional interactive visuals based on emotional interactive actions and emotional interactive expressions.

[0139] S218. Display emotional interaction visuals and play emotional interaction audio in the vehicle.

[0140] In some embodiments, the display of emotional interaction images in the vehicle can be achieved by copying the emotional interaction images to the in-vehicle central control screen or the passenger entertainment screen, or by projecting the emotional interaction images onto the windshield through the HUD function; there is no limitation here.

[0141] In some embodiments, the vehicle plays emotional interaction audio by: when the frame in the emotional interaction scene includes the emotional interaction audio, calling the vehicle's external speaker function to play the emotional interaction audio corresponding to the pet model.

[0142] Optionally, the playback volume of the emotional interactive audio can be automatically adjusted by the vehicle based on the current vehicle speed. Specifically, the vehicle can determine the current noise level based on the current speed; determine a volume gain value based on the current noise level, the baseline noise level, and the target signal-to-noise ratio; and adjust the playback volume of the emotional interactive audio based on the volume gain value. For example, the volume gain value could range from +5dB to +7dB.

[0143] In some embodiments, the display of emotional interaction screens in the vehicle can be achieved by using the off-screen buffer of the graphics processing unit (GPU) in the vehicle's infotainment system. For each frame of the emotional interaction screen, the depth value is cleared when the frame begins to be rendered to prevent the data of the previous frame from interfering with the current frame. Then, the background layer, ground shadow layer, pet layer (i.e., rendering of the 3D pet model), and foreground layer are drawn. The contents of the off-screen buffer are copied to the display buffer to display the emotional interaction screen.

[0144] In some embodiments, when the emotional interaction trigger event is a touch event, the vehicle can also convert the two-dimensional coordinates of the touch position corresponding to the touch event into a three-dimensional spatial ray; perform collision detection based on the three-dimensional spatial ray and the emotional interaction content to obtain a collision detection result; and if the collision detection result indicates that the three-dimensional spatial ray did not hit the emotional interaction content, adjust the presentation position of the emotional interaction content based on a preset distance.

[0145] In some embodiments, the vehicle can also use the refresh cycle of the screen displaying the emotional interaction content as a time reference to perform time axis interpolation processing on keyframes in the emotional interaction content to generate target rotation values ​​and target positions. The target rotation value is used to characterize the orientation of the object model in the emotional interaction content in three-dimensional space, and the target position is used to characterize the position of the root skeleton of the object model in three-dimensional space. Based on the target position, the horizontal position of the root skeleton of the emotional interaction object model is determined. Based on the target rotation value and the horizontal position, anti-mold-breaking processing is performed on the emotional interaction content to obtain the processed emotional interaction content.

[0146] In some embodiments, the vehicle may also trigger the display of a corresponding emotional interaction expression within a preset time period after the pet model performs an emotional interaction action. This minimizes the abrupt changes in the emotional interaction action caused by the introduction of facial expressions. For example, the vehicle may display a half-closed eye expression 0.2s-0.4s after the pet model performs a sniffing action; or a slowly closing eye expression 0.2s-0.4s after the pet model performs a sleeping action, etc.

[0147] In some embodiments, the vehicle can also determine the ambient light intensity within the vehicle's cabin and map the ambient light intensity to the brightness of the main light source. When the brightness within the cabin meets the brightness adjustment conditions, the brightness within the cabin is adjusted based on the brightness of the main light source and the brightness of the fill light to regulate the presentation effect of emotional interactive content within the vehicle. For example, when the brightness within the cabin is at the level corresponding to night mode, the vehicle can adjust the brightness within the cabin based on the brightness of the main light source and the brightness of the fill light to ensure that the fur color of the pet model remains visible without being overexposed.

[0148] In some embodiments, the vehicle may also release the temporary interpolation cache within a first preset duration after the emotional interaction action in the emotional interaction screen has been displayed; compress the rendering resolution of the pet model to the target resolution; and unload the pet model if no new emotional interaction trigger event is detected within a second preset duration. This reduces memory usage and minimizes the risk of low navigation performance due to the pet model being displayed on the screen for an extended period. Optionally, the first preset duration may range from 0.5s to 1.5s; the target resolution may be, for example, 256*256; and the second preset duration may range from, for example, 3min to 6min.

[0149] In this embodiment, the introduction of a real pet image can enhance the user's willingness to interact emotionally with the pet model. This can not only effectively alleviate the negative emotions caused by traffic conditions or other factors, but also alleviate the car owner's longing for their pet and enhance the emotional atmosphere of the cabin.

[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0151] Based on the same inventive concept, this application also provides a vehicle-based emotional interaction device for implementing the above-mentioned vehicle-based emotional interaction method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more vehicle-based emotional interaction device embodiments provided below can be found in the limitations of the vehicle-based emotional interaction method described above, and will not be repeated here.

[0152] Please see Figure 3 , Figure 3 This is a schematic diagram of an optional structure for an emotion interaction device based on an in-vehicle scenario, provided in an embodiment of this application. For example... Figure 3 As shown, the emotional interaction device based on the in-vehicle scenario may include, but is not limited to:

[0153] The acquisition module 301 is used to acquire an object model associated with the target real object in response to an emotional interaction trigger event generated in the vehicle cabin; wherein, the object model is constructed based on multimedia data representing the target real object;

[0154] The determination module 302 is used to determine the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model based on the emotional interaction trigger event;

[0155] The presentation module 303 is used to present the emotional interaction content corresponding to the object model in the vehicle cabin; in the emotional interaction content, the object model displays emotional interaction expressions and performs emotional interaction actions.

[0156] In some embodiments, the device may further include a control module; the acquisition module 301 is further configured to acquire the target skeleton rotation angle corresponding to the emotional interaction action based on a first correspondence; wherein the first correspondence includes the correspondence between multiple action sequences of the target real object and multiple skeleton rotation angles; the control module is configured to control the object model based on the target skeleton rotation angle so that the object model performs the emotional interaction action.

[0157] In some embodiments, when the presentation module 303 presents the emotional interaction content corresponding to the object model in the vehicle, the acquisition module 301 is used to acquire the target voiceprint information corresponding to the target real object based on the second correspondence relationship; the second correspondence relationship includes the correspondence relationship between multiple real objects and multiple voiceprint information; the determination module 302 is used to generate the emotional interaction content corresponding to the object model based on the target voiceprint information, emotional interaction actions and emotional interaction expressions; the emotional interaction content includes emotional interaction audio and emotional interaction images; the presentation module 303 is used to display the emotional interaction images in the vehicle and play the emotional interaction audio.

[0158] In some embodiments, when determining the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model based on the emotional interaction triggering event, the determining module 302 is specifically used to: determine the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model associated with the emotional interaction triggering event based on a third correspondence relationship; the third correspondence relationship includes the correspondence relationship between multiple emotional interaction triggering events and multiple combinations, and each combination includes the emotional interaction action and emotional interaction expression to be triggered.

[0159] In some embodiments, the device may further include a construction module; an acquisition module 301 is further configured to acquire multimedia data collected from multiple angles of a target real object, and based on the multimedia data, acquire images of the target real object from each angle; a determination module 302 is configured to segment the contour data of the real object from each image to obtain a contour dataset, and based on the contour dataset, determine the point cloud data corresponding to the real object; determine the appearance morphology pixels of the real object based on each image; generate an appearance morphology map of the real object based on the features of each pixel in the appearance morphology pixels; generate a normal map and a roughness map corresponding to the real object based on the appearance morphology map; and a construction module is configured to construct a pet model corresponding to the target real object based on the point cloud data, the normal map, and the roughness map.

[0160] In some embodiments, the device further includes a processing module, which is configured to convert the two-dimensional coordinates of the touch position corresponding to the touch event into a three-dimensional spatial ray when the emotional interaction trigger event is a touch event; perform collision detection based on the three-dimensional spatial ray and the emotional interaction content to obtain a collision detection result; and adjust the presentation position of the emotional interaction content based on a preset distance when the collision detection result indicates that the three-dimensional spatial ray did not hit the emotional interaction content.

[0161] In some embodiments, the processing module is further configured to perform time-axis interpolation processing on key frames in the emotional interaction content using the refresh cycle of the screen used to display the emotional interaction content as a time reference, to generate transition frames; to perform smoothing processing on the emotional interaction content based on the transition frames, to obtain processed emotional interaction content; and to present the processed emotional interaction content in the vehicle.

[0162] In some embodiments, the processing module is further configured to determine the ambient light intensity inside the vehicle's cabin and map the ambient light intensity to the brightness of the main light source; when the brightness inside the cabin meets the brightness adjustment conditions, the brightness inside the cabin is adjusted based on the brightness of the main light source and the brightness of the fill light to adjust the presentation effect of emotional interactive content in the vehicle.

[0163] It is understood that the specific implementation of each module in the vehicle-based emotional interaction device provided in this application embodiment and the beneficial effects that can be achieved can be referred to the description of the aforementioned vehicle-based emotional interaction method embodiment, and will not be repeated here.

[0164] Each module in the aforementioned vehicle-based emotional interaction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the vehicle terminal device in hardware form or independent of it, or stored in the memory of the vehicle-based emotional interaction device in software form, so that the processor can call and execute the corresponding operations of each module.

[0165] In one exemplary embodiment, a vehicle is provided whose internal structure diagram can be as follows: Figure 4As shown, the vehicle includes a processor, memory, input / output interfaces, a communication interface, a display unit, and input devices. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input devices are also connected to the system bus via the input / output interfaces. The vehicle's processor provides computing and control capabilities. The vehicle's memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The vehicle's input / output interfaces are used for exchanging information between the processor and external devices. The vehicle's communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an emotional interaction method based on the in-vehicle scenario. The vehicle's display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the vehicle can be a touch layer covering the display screen, or it can be a button, trackball, or touchpad installed in the vehicle.

[0166] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the vehicle to which the present application is applied. A specific vehicle may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0167] In one exemplary embodiment, this application provides a vehicle including a memory and a controller, wherein the memory stores a computer program; when the controller executes the computer program, it implements the steps in the above-described emotional interaction methods based on in-vehicle scenarios.

[0168] In one exemplary embodiment, this application provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps in the aforementioned emotional interaction methods based on in-vehicle scenarios.

[0169] In one exemplary embodiment, this application provides a computer program product, including a computer program. When executed by a processor, the computer program implements the steps in the above-described emotional interaction methods based on in-vehicle scenarios.

[0170] In one exemplary embodiment, this application also provides an emotional interaction system based on an in-vehicle scenario, which includes a mobile terminal for controlling the vehicle, the vehicle, and electronic devices.

[0171] Among them, the mobile terminal used to control the vehicle is used to acquire multimedia data of the target real object and send the multimedia data of the target real object to the electronic device;

[0172] An electronic device is used to construct an object model corresponding to a target real object based on multimedia data of the target real object; construct multiple facial expression states corresponding to the target real object; construct the correspondence between the action sequence of the target real object and the skeletal rotation angle based on the action sequence of the target real object; construct object data corresponding to the target real object based on the object model, the skeletal rotation angle table, and multiple facial expression forms; and send the object data corresponding to the target real object to a vehicle.

[0173] The vehicle is used to respond to emotional interaction trigger events generated within the vehicle cabin. It obtains the object model associated with the target object from the object data corresponding to the target real object received; based on the emotional interaction trigger event, it determines the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model; it presents the emotional interaction content corresponding to the object model in the vehicle cabin; in the emotional interaction content, the object model displays emotional interaction expressions and performs emotional interaction actions.

[0174] It should be noted that the data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0175] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0176] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0177] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An emotional interaction method based on in-vehicle scenarios, characterized in that, The method includes: In response to an emotional interaction trigger event generated within the vehicle cabin, an object model associated with a target real object is obtained; wherein, the object model is constructed based on multimedia data representing the target real object; Based on the emotional interaction triggering event, determine the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model; The emotional interaction content corresponding to the object model is presented in the vehicle cabin; in the emotional interaction content, the object model displays the emotional interaction expression and performs the emotional interaction action.

2. The method according to claim 1, characterized in that, The method further includes: Based on the first correspondence, the target skeleton rotation angle corresponding to the emotional interaction action is obtained; wherein, the first correspondence includes the correspondence between multiple action sequences of the target real object and multiple skeleton rotation angles; The object model is controlled based on the rotation angle of the target skeleton so that the object model can perform the emotional interaction action.

3. The method according to claim 1, characterized in that, Presenting emotional interaction content corresponding to the object model in the vehicle includes: Based on the second correspondence, the target voiceprint information corresponding to the target real object is obtained; the second correspondence includes the correspondence between multiple real objects and multiple voiceprint information. Based on the target voiceprint information, the emotional interaction actions, and the emotional interaction expressions, emotional interaction content corresponding to the object model is generated; the emotional interaction content includes emotional interaction audio and emotional interaction visuals. The emotional interaction screen is displayed in the vehicle, and the emotional interaction audio is played.

4. The method according to claim 1, characterized in that, The step of determining the emotional interaction action and emotional expression to be triggered corresponding to the object model based on the emotional interaction trigger event includes: Based on the third correspondence, obtain the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model associated with the emotional interaction triggering event; the third correspondence includes the correspondence between multiple emotional interaction triggering events and multiple combinations, and each combination includes the emotional interaction action and emotional interaction expression to be triggered.

5. The method according to claim 1, characterized in that, The object model is constructed in the following way: Acquire multimedia data collected from multiple angles of the target real object, and based on the multimedia data, acquire images of the target real object from each of the multiple angles; From each of the images, the contour data of the target real object is segmented to obtain a contour dataset, and based on the contour dataset, the point cloud data corresponding to the target real object is determined; Based on each of the images, the appearance shape pixels of the target real object are determined, and based on the features of each pixel in the appearance shape pixels, the appearance shape map of the target real object is generated. Based on the appearance shape map, generate the normal map and roughness map corresponding to the target real object; Based on the point cloud data, the normal map, and the roughness map, an object model associated with the target real object is constructed.

6. The method according to claim 1, characterized in that, The method further includes: When the emotional interaction trigger event is a touch event, the two-dimensional coordinates of the touch position corresponding to the touch event are converted into a three-dimensional spatial ray; Collision detection is performed based on the three-dimensional spatial rays and the emotional interaction content to obtain the collision detection result; If the collision detection result indicates that the three-dimensional spatial ray did not hit the emotional interaction content, the presentation position of the emotional interaction content is adjusted based on a preset distance.

7. The method according to claim 1, characterized in that, The method further includes: Using the refresh cycle of the screen used to display the emotional interaction content as a time reference, time axis interpolation processing is performed on the keyframes in the emotional interaction content to generate target rotation values ​​and target positions; wherein, the target rotation value is used to characterize the orientation of the object model in the emotional interaction content in three-dimensional space, and the target position is used to characterize the position of the root skeleton of the object model in three-dimensional space; Based on the target location, determine the horizontal position of the root skeleton of the emotional interaction object model; Based on the target rotation value and horizontal position, anti-clipping processing is performed on the emotional interaction content to obtain the processed emotional interaction content.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Determine the ambient light intensity inside the vehicle's cabin and map the ambient light intensity to the brightness of the main light source; When the brightness inside the cabin meets the brightness adjustment conditions, the brightness inside the cabin is adjusted based on the brightness of the main light source and the brightness of the fill light to adjust the presentation effect of the emotional interactive content in the vehicle.

9. An emotional interaction device based on an in-vehicle scenario, characterized in that, The device includes: The acquisition module is used to acquire an object model associated with a target real object in response to an emotional interaction trigger event generated in the vehicle cabin; wherein, the object model is constructed based on multimedia data representing the target real object; The determination module is used to determine the emotional interaction action and emotional interaction expression to be triggered corresponding to the object model based on the emotional interaction trigger event. A presentation module is used to present the emotional interaction content corresponding to the object model in the vehicle cabin; in the emotional interaction content, the object model displays the emotional interaction expression and performs the emotional interaction action.

10. A vehicle, characterized in that, It includes a memory and a controller; the memory stores a computer program, and the controller executes the computer program to implement the steps of the method as claimed in any one of claims 1 to 9.

Images