Interaction methods and devices, storage media and electronic devices

Abstract

This invention provides an interaction method, apparatus, storage medium, and electronic device. The method includes: acquiring an image of the interior of a vehicle to obtain a target image; performing facial recognition on the target image to obtain a target person and their corresponding information; determining at least one feature dimension; performing feature detection on the target image using a preset detection model corresponding to each feature dimension to obtain dimensional features corresponding to each feature dimension; determining at least one interaction method; determining a target praise interaction scheme corresponding to each interaction method based on the person information and the feature dimensions; determining various interaction operations based on the person information, the feature dimensions, and the target praise interaction schemes; and executing the interaction operations to achieve the interaction. Applying the method of this invention, personalized interactions with praise implications can be achieved based on interaction operations associated with the user's current state, which helps improve the interaction effect and create a positive driving atmosphere.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle technology, and in particular to an interaction method and device, a storage medium and an electronic device. Background Technology

[0002] With the development of artificial intelligence technology, the functions of in-vehicle systems are becoming increasingly intelligent. Interacting with people inside the vehicle is one of the main functions of in-vehicle systems.

[0003] In the interactive functions of in-vehicle systems, there are usually some proactive interactions to enhance the user experience. Currently, in-vehicle systems generally play fixed voice messages at fixed times to achieve proactive interaction; for example, when the in-vehicle system starts up, it plays a fixed greeting voice message.

[0004] In real-world driving scenarios, long distances and traffic congestion often lead to a poor atmosphere inside the vehicle, potentially triggering road rage in drivers. Given the current interaction methods of in-vehicle systems, relying solely on single, fixed voice commands for proactive interaction is merely a programmed broadcast to those in the driving environment. This results in limited interaction, minimal improvement to the user experience, and is detrimental to creating a pleasant driving atmosphere. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an interaction method to solve the problems of the existing in-vehicle system's active interaction being singular and fixed, resulting in poor interaction effects and failing to create a good driving atmosphere.

[0006] This invention also provides an interactive device to ensure the practical implementation and application of the above method.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] An interaction method, applied to an in-vehicle system, the method comprising:

[0009] The interior of the current vehicle is captured to obtain the target image;

[0010] Perform facial recognition on the target image to obtain the target person corresponding to the target image and the person information corresponding to the target person;

[0011] Determine the current set of feature dimensions, wherein the current set of feature dimensions contains at least one feature dimension;

[0012] By using a preset detection model corresponding to each feature dimension, feature detection is performed on the target image to obtain the dimensional features corresponding to each feature dimension;

[0013] Determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method;

[0014] Based on the character information and the dimensional features corresponding to each feature dimension, among the multiple preset praise interaction schemes corresponding to each interaction method, a target praise interaction scheme corresponding to each interaction method is determined;

[0015] Based on the character information, the dimensional features corresponding to each of the aforementioned feature dimensions, and the respective target praise interaction schemes, determine the interaction operation corresponding to each of the aforementioned interaction methods;

[0016] Perform each of the aforementioned interactive operations to interact with the target character.

[0017] Optionally, in the above method, each of the feature dimensions includes a skin feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the skin feature dimension includes:

[0018] The target image is input into a preset detection model corresponding to the skin feature dimension to perform skin feature recognition on the face image of the target person contained in the target image;

[0019] Obtain the skin feature recognition result of the target person output by the preset detection model corresponding to the skin feature dimension;

[0020] Among a number of preset skin states, the target skin state corresponding to the skin feature recognition result is determined, and the target skin state is used as the dimensional feature corresponding to the skin feature dimension.

[0021] Optionally, in the above method, each of the feature dimensions includes a makeup feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the makeup feature dimension includes:

[0022] The target image is input into a preset detection model corresponding to the makeup feature dimension to perform makeup feature recognition on the face image of the target person contained in the target image;

[0023] Obtain the makeup feature recognition result of the target person output by the preset detection model corresponding to the makeup feature dimension;

[0024] Among a number of preset makeup styles, the target makeup style corresponding to the makeup feature recognition result is determined, and the target makeup style is used as the dimension feature corresponding to the makeup feature dimension.

[0025] Optionally, in the above method, each of the feature dimensions includes a clothing feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the clothing feature dimension includes:

[0026] The target image is input into a preset detection model corresponding to the clothing feature dimension to perform clothing feature recognition on the clothing image of the target person contained in the target image;

[0027] Obtain the clothing feature recognition result of the target person corresponding to the preset detection model corresponding to the clothing feature dimension, and use the clothing feature recognition result as the dimension feature corresponding to the clothing feature dimension.

[0028] Optionally, in the above method, each of the feature dimensions includes an emotion feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the emotion feature dimension includes:

[0029] The target image is input into a preset detection model corresponding to the emotion feature dimension to perform emotion recognition on the facial image of the target person contained in the target image;

[0030] Obtain the emotion recognition result of the target person output by the preset detection model corresponding to the emotion feature dimension, and use the emotion recognition result as the dimension feature corresponding to the emotion feature dimension.

[0031] Optionally, in the above method, each of the feature dimensions includes an item feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the item feature dimension includes:

[0032] The target image is input into a preset detection model corresponding to the feature dimension of the item to perform item recognition on the target image;

[0033] Obtain the object category output by the preset detection model corresponding to the item feature dimension, and use the object category as the dimension feature corresponding to the item feature dimension.

[0034] Optionally, the interaction methods described above include: voice interaction, image interaction, light interaction, and tactile interaction.

[0035] An interactive device, applied to an in-vehicle system, the device comprising:

[0036] The image acquisition unit is used to acquire images of the interior of the current vehicle and obtain the target image;

[0037] A face recognition unit is used to perform face recognition on the target image to obtain the target person corresponding to the target image and the person information corresponding to the target person;

[0038] The first determining unit is used to determine the current feature dimension set, wherein the current feature dimension set contains at least one feature dimension;

[0039] The feature detection unit is used to perform feature detection on the target image by using a preset detection model corresponding to each feature dimension, so as to obtain the dimensional features corresponding to each feature dimension.

[0040] The second determining unit is used to determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method.

[0041] The third determining unit is used to determine the target praise interaction scheme corresponding to each interaction method from among multiple preset praise interaction schemes corresponding to each interaction method, based on the person information and the dimension features corresponding to each feature dimension;

[0042] The fourth determining unit is used to determine the interactive operation corresponding to each interactive method based on the person information, the dimensional features corresponding to each of the feature dimensions, and each of the target praise interaction schemes;

[0043] An interactive unit is used to perform various interactive operations to interact with the target person.

[0044] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides executes the interaction method described above.

[0045] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the interaction method described above.

[0046] Based on the interaction method provided by the above embodiments of the present invention, multiple feature dimensions to be referenced during interaction can be preset. When interaction with people inside the vehicle is required, the system identifies the person's information and current dimensional features through collected images. Then, based on a preset praise interaction scheme, interactive operations with a strong correlation to the person's current state and a praise-like connotation can be determined for interaction. This enables personalized interaction that is correlated with the real-time state of the interaction object, resulting in better interaction effects and improved user experience. Secondly, the praise-like interactive operations help bring pleasant emotions to the interaction object and create a good driving atmosphere. In addition, diverse interactive operations can increase the fun of the interaction and further improve the interaction effect. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0048] Figure 1 A flowchart of an interactive method provided in an embodiment of the present invention;

[0049] Figure 2 Another flowchart of an interaction method provided in an embodiment of the present invention;

[0050] Figure 3 An example diagram of an interaction process provided in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the structure of an interactive device provided in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] This invention provides an interaction method applied to an in-vehicle system, wherein the executing entity can be the system's processor. The method flowchart is shown below. Figure 1 As shown, it includes:

[0056] S101: Acquire images of the interior of the current vehicle to obtain the target image;

[0057] In the method provided by this invention, when active interaction with the driver or passengers inside the vehicle is required, a preset camera inside the vehicle can be triggered to capture images of the vehicle's interior, i.e., to take pictures, and the captured images can be used as target images. The camera can be installed inside the vehicle, above the windshield, or in other locations where it can capture images of the front of the people inside the vehicle.

[0058] In the method provided by this invention, the in-vehicle system can actively interact with the driver or passengers in the vehicle at predetermined time points after the system is started. For example, the system startup completion time can be set as an interaction time point, and during system operation, a preset time interval can be set as an interaction time point, with image acquisition performed at each interaction time point.

[0059] S102: Perform face recognition on the target image to obtain the target person corresponding to the target image and the person information corresponding to the target person;

[0060] In the method provided by this invention, a face detection model can be pre-built. Face recognition is then performed on the target image using this model, primarily identifying facial features to determine the individuals contained within the image. For example, it identifies whether a face in the target image belongs to a pre-registered user. If so, the face image is associated with the corresponding user, and the associated user is considered the target individual. If the face image does not belong to a pre-registered user, an unregistered person is created and associated with that face image, and this unregistered person is considered the target individual. During processing, the "person" can be represented by a user identifier, corresponding to the person inside the vehicle. In the face recognition process, information such as the person's age and gender can also be further identified. Based on the face recognition results and pre-recorded user information in the system, the person information corresponding to the target individual can be determined. For example, person information can include person attributes, name, age, and gender. Person attributes refer to whether the person is a registered user. The name can be a pre-registered name or nickname. For unregistered individuals, a pre-configured name can be provided.

[0061] In practical applications, there may be multiple people in the target image. Each identified person can be treated as a target person and processed separately.

[0062] S103: Determine the current set of feature dimensions, wherein the current set of feature dimensions contains at least one feature dimension;

[0063] In the method provided by this invention, the system can pre-set multiple feature dimensions. At different interaction points or based on the target person's information, interaction can be performed based on different feature dimensions. The current feature dimensions to be referenced can be determined based on the current interaction point or person information, thus obtaining the current feature dimension set. For example, the preset feature dimensions may include emotional feature dimensions, skin feature dimensions, makeup feature dimensions, etc. At the interaction point when the system starts, the current feature dimension set may include skin feature dimensions and makeup feature dimensions; at the interaction point during driving, the current feature dimension set may include emotional feature dimensions.

[0064] It should be noted that in specific applications, the same feature dimension can be used for each interaction without affecting the functionality of the method provided in the embodiments of the present invention.

[0065] S104: Using the preset detection model corresponding to each feature dimension, perform feature detection on the target image respectively to obtain the dimensional features corresponding to each feature dimension;

[0066] In the method provided by this invention, the system pre-sets a detection model corresponding to each feature dimension. Each preset detection model can identify and detect features of the corresponding dimension in the image. The target image can be input into the preset detection model corresponding to each feature dimension, so that feature detection is performed on the target image by each preset detection model. After each preset detection model performs feature detection on the target image, it can output the corresponding detection result, and the dimensional features of the corresponding feature dimension can be determined based on the detection result.

[0067] S105: Determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method;

[0068] In the method provided by the embodiments of the present invention, the system has a variety of interaction methods pre-set. Based on the current vehicle configuration, the set of interaction methods corresponding to the current vehicle can be determined, which includes various interaction methods supported by the current vehicle, such as voice interaction, image interaction on the screen, etc.

[0069] In practical applications, due to the widespread availability of hardware support and the intuitiveness of interactive information, the set of interaction methods usually includes voice interaction or image interaction.

[0070] S106: Based on the person information and the dimension features corresponding to each of the feature dimensions, determine the target praise interaction scheme corresponding to each of the multiple preset praise interaction schemes corresponding to each of the interaction methods;

[0071] In the method provided by this invention, the system pre-sets multiple preset praise interaction schemes corresponding to each type of interaction. These preset praise interaction schemes are interaction schemes with a praising meaning. For example, in voice interaction, multiple voice templates are formulated as interaction schemes based on praise phrases. Each preset praise interaction scheme has its corresponding application attributes, which may include: the type of person the interaction scheme is applicable to (unregistered / registered), the person's gender, the person's age range, applicable feature dimensions, and dimensional features, etc.

[0072] In the method provided by this invention, the person information corresponding to the target person and the dimensional features of each detected feature dimension can be used as person attribute information. For each type of interaction, the person attribute information is matched with the application attributes corresponding to each preset praise interaction scheme for that interaction method. The preset praise interaction scheme that matches the application attributes with the person attribute information is used as the target praise interaction scheme corresponding to that interaction method.

[0073] S107: Based on the character information, the dimensional features corresponding to each of the feature dimensions, and each of the target praise interaction schemes, determine the interaction operation corresponding to each of the interaction methods;

[0074] In the method provided by this invention, for each target praise interaction scheme, if the target praise interaction scheme requires the application of person information and dimensional features, then according to the requirements, the required information data is extracted from the person information and various dimensional features, and the information data is applied to the target praise interaction scheme to generate the corresponding interactive operation. For example, in the voice interaction method, the target praise interaction scheme is a preset voice broadcast template, such as "Dear (person's name), you look so beautiful today (makeup name)." The name corresponding to the target person is extracted from the person information. The makeup name is associated with the makeup feature dimension, so the dimensional feature corresponding to the makeup feature dimension is obtained. The name corresponding to the target person - A and the dimensional feature corresponding to the makeup feature dimension - makeup nails are applied to the voice broadcast template to generate the voice broadcast information "Dear A, your makeup nails look so beautiful today." The interactive operation of the voice interaction is to play the voice according to the voice broadcast information.

[0075] S108: Perform each of the aforementioned interactive operations to interact with the target person.

[0076] In the method provided by this invention, each interactive operation can be triggered to execute the corresponding object, thereby enabling interaction with the target person. For example, for a voice-based interactive operation, the player is activated to play the corresponding voice. Similarly, for an image-based interactive operation, the corresponding animated image is played on the front-end interface of the in-vehicle system.

[0077] It should be noted that in practical applications, the target praise interaction scheme corresponding to a certain interaction method can be a no-action scheme. That is, given the current character information and various dimensional characteristics, there is no need to interact based on that interaction method. The interaction operation corresponding to that method can be regarded as an interaction operation without action effect. If all interaction operations corresponding to all interaction methods are interaction operations without action effect, the execution result will not produce any actual interaction action. It can be considered that there is no need to interact with the target character at this time, and the current interaction process with the target character can be ended.

[0078] Based on the method provided in this invention, multiple feature dimensions to be referenced during interaction can be pre-set. When interaction with people inside the vehicle is required, the system identifies the person's information and current features based on the acquired images. Then, based on a pre-set praise-based interaction scheme, it determines interactive operations that are strongly correlated with the person's current state and have a praising connotation, which are then used for interaction. This enables personalized interaction that is correlated with the real-time state of the interaction object, resulting in better interaction effects and improved user experience. Secondly, praising interactive operations help bring pleasant emotions to the interaction object and create a good driving atmosphere. In addition, diverse interactive operations can increase the fun of the interaction and further improve the interaction effect.

[0079] exist Figure 1 Based on the method shown, in the method provided by the embodiments of the present invention, the interaction methods mentioned in step S105 include the following: voice interaction method, image interaction method, light interaction method and tactile interaction method.

[0080] The method provided in this invention supports voice interaction, image interaction, light interaction, and tactile interaction. Voice interaction can combine voice-based actions, facial expressions, and voice to provide audio feedback. Image interaction can display images or videos through in-vehicle screens, AR-HUDs, pixel headlights, and other image-displaying systems to provide image interaction. Light interaction can display static or dynamic lighting effects through ambient lighting, headlights, and external lighting systems to provide lighting interaction. Tactile interaction can control the movement of corresponding objects according to preset vibration frequencies and motion patterns through vibration systems of seats, steering wheels, and in-vehicle screens to provide tactile interaction.

[0081] It should be noted that the specific interaction methods mentioned in the embodiments of the present invention are only a specific embodiment provided to better illustrate the method of the present invention. In the specific implementation process, one or any number of the above-mentioned interaction methods can be applied, and other interaction methods can be combined without affecting the implementation function of the method provided in the embodiments of the present invention.

[0082] To better illustrate the method provided by this invention, the feature detection process of the target image will be described with an example below.

[0083] This invention provides yet another interaction method, in Figure 1Based on the method shown, in the method provided by the embodiments of the present invention, the feature dimension set mentioned in step S103 includes a skin feature dimension. Therefore, in the process of performing feature detection on the target image by means of a preset detection model corresponding to each feature dimension mentioned in step S104, the process of performing feature detection on the target image by means of a preset detection model corresponding to the skin feature dimension is included.

[0084] refer to Figure 2 The flowchart shown illustrates the process of feature detection of the target image using a preset detection model corresponding to the skin feature dimension, as provided in this embodiment of the invention. The process includes:

[0085] S201: Input the target image into the preset detection model corresponding to the skin feature dimension to perform skin feature recognition on the face image of the target person contained in the target image;

[0086] In the method provided by this invention, the feature detection of the target image includes the detection of the skin condition of the person. A preset detection model corresponding to the skin feature dimension can be constructed based on image algorithms. This preset detection model can be called a skin detection model. The skin detection model can identify skin features in a specified image. The identified feature elements can include skin color, blemishes, acne scars, blackheads, etc.

[0087] The target image is input into a skin detection model, which then identifies skin features of the target person's face within the image to obtain the characteristics of the target person's skin condition. After identifying and detecting the target image, the skin detection model outputs the identified skin features.

[0088] S202: Obtain the skin feature recognition result of the target person output by the preset detection model corresponding to the skin feature dimension;

[0089] In the method provided by this invention, the skin feature recognition result corresponding to the target person can be obtained from the output layer of the skin detection model. This result may include feature data representing the skin condition, such as skin color features, freckle features, acne scar features, and blackhead features, obtained by performing skin feature recognition on the face image of the target person.

[0090] S203: Among a plurality of preset skin states, determine the target skin state corresponding to the skin feature recognition result, and use the target skin state as the dimensional feature corresponding to the skin feature dimension.

[0091] In the method provided by this invention, multiple skin states can be preset, each representing the degree of skin condition. Each preset skin state has its corresponding skin attributes, such as skin color, blemishes, acne scars, etc. The skin feature recognition results of the target person can be matched with the skin attributes corresponding to each skin state, and the skin state corresponding to the matching skin attribute is taken as the target skin state, which is then used as the dimensional feature corresponding to the skin feature dimension.

[0092] To better illustrate the method provided in the embodiments of the present invention, the interactive operation is illustrated using the skin feature dimension as an example. The interactive operations corresponding to each determined interactive method can be referred to the examples in Table 1 below:

[0093] Table 1

[0094]

[0095]

[0096] This invention provides another interaction method, in Figure 1 Based on the method shown, the method provided in this embodiment of the invention includes a makeup feature dimension among the feature dimension sets. The process of performing feature detection on the target image using a preset detection model corresponding to the makeup feature dimension includes:

[0097] The target image is input into a preset detection model corresponding to the makeup feature dimension to perform makeup feature recognition on the face image of the target person contained in the target image;

[0098] In the method provided by this invention, the feature detection of the target image includes the detection of the person's makeup style. A preset detection model corresponding to the makeup feature dimension can be constructed based on image algorithms. This preset detection model can be called a makeup detection model. The makeup detection model can identify the makeup features of a specified face image. The identified feature elements can include the structure and color of parts such as the eyes, mouth, forehead, cheeks, and nose, skin tone, and the light and shadow on the bridge of the nose and cheeks, etc.

[0099] The target image is input into the makeup detection model, which then identifies the makeup features of the target person's face within the image to obtain their makeup characteristics. After detecting the makeup features in the target image, the makeup detection model outputs the identified makeup features.

[0100] Obtain the makeup feature recognition result of the target person output by the preset detection model corresponding to the makeup feature dimension;

[0101] In the method provided by the embodiments of the present invention, the output of the makeup detection model can be obtained as the makeup feature recognition result corresponding to the target person. This result may include feature data of various feature elements related to makeup obtained by performing makeup feature recognition on the face image of the target person.

[0102] Among a number of preset makeup styles, the target makeup style corresponding to the makeup feature recognition result is determined, and the target makeup style is used as the dimension feature corresponding to the makeup feature dimension.

[0103] In the method provided by this invention, multiple makeup styles can be preset, such as no-makeup makeup, nude makeup, smoky makeup, Western makeup, etc., and a makeup style representing no makeup can also be set. Each preset makeup style has its corresponding makeup attributes, such as eye color attributes, lip color attributes, skin tone attributes, nose bridge lighting attributes, etc. The makeup feature recognition results are matched with the makeup attributes corresponding to each makeup style, and the makeup style corresponding to the matching makeup attributes is taken as the target makeup style, which is used as the dimension feature corresponding to the makeup feature dimension.

[0104] Taking the makeup feature dimension as an example, the interactive operations are illustrated below. Examples of each interactive operation can be found in Table 2:

[0105] Table 2

[0106]

[0107] This invention provides yet another interaction method, in Figure 1 Based on the method shown, the method provided in this embodiment of the invention includes a clothing feature dimension among the feature dimension sets. The process of performing feature detection on the target image using a preset detection model corresponding to the clothing feature dimension includes:

[0108] The target image is input into a preset detection model corresponding to the clothing feature dimension to perform clothing feature recognition on the clothing image of the target person contained in the target image;

[0109] In the method provided by this invention, the feature detection of the target image includes the detection of clothing matching. A preset detection model corresponding to the feature dimensions of clothing can be constructed based on image algorithms. This preset detection model can be called a clothing detection model. The clothing detection model can identify clothing features in a specified image. The identified feature elements can include the identification of clothing and accessories such as glasses, hats, scarves, necklaces, earrings, and clothes, as well as attributes such as color, brand, and shape of each piece of clothing and accessory.

[0110] The target image is input into the clothing detection model, which then identifies the clothing features of the target person within the image to obtain the characteristics of the clothing and accessories worn by the target person. After identifying and detecting the target image, the clothing detection model can output the names of the identified clothing and accessories, as well as attribute data such as color and brand of each item.

[0111] Obtain the clothing feature recognition result of the target person corresponding to the preset detection model corresponding to the clothing feature dimension, and use the clothing feature recognition result as the dimension feature corresponding to the clothing feature dimension.

[0112] In the method provided by this invention, the output of the clothing detection model can be obtained as the clothing feature recognition result of the target person, that is, the relevant feature data of the clothing and accessories worn by the target person obtained by recognizing the clothing image of the target person. This clothing feature recognition result is used as the dimensional feature corresponding to the clothing feature dimension.

[0113] Taking the clothing feature dimension as an example, the various interactive operations can be seen in the examples in Table 3 below:

[0114] Table 3

[0115]

[0116] This invention provides yet another interaction method, in Figure 1 Based on the method shown, the method provided in this embodiment of the invention includes an emotion feature dimension among the feature dimensions in the feature dimension set. The process of performing feature detection on the target image using a preset detection model corresponding to the emotion feature dimension includes:

[0117] The target image is input into a preset detection model corresponding to the emotion feature dimension to perform emotion recognition on the facial image of the target person contained in the target image;

[0118] In the method provided by this invention, the feature detection of the target image includes the detection of human emotions. A preset detection model corresponding to the emotional feature dimension can be constructed based on image algorithms. This preset detection model can be called an emotion detection model. The emotion detection model can perform emotion recognition on a specified facial image. By detecting the state of the face, it can identify emotional expressions, such as smiles or anger, to identify the current emotional characteristics of the detected object.

[0119] The target image is input into the emotion detection model, which then identifies the emotional features of the facial image of the target person contained within the image. After processing the target image, the emotion detection model outputs the identified emotional features.

[0120] Obtain the emotion recognition result of the target person output by the preset detection model corresponding to the emotion feature dimension, and use the emotion recognition result as the dimension feature corresponding to the emotion feature dimension.

[0121] In the method provided by this embodiment of the invention, the emotion recognition result corresponding to the target person can be obtained from the output layer of the emotion detection model, and the emotion recognition result can be used as the dimension feature corresponding to the emotion feature dimension.

[0122] Taking the emotion feature dimension as an example, in this scenario, the various interactive operations can be referenced in Table 4 below:

[0123] Table 4

[0124]

[0125] This invention provides yet another interaction method, in Figure 1 Based on the method shown, the method provided in this embodiment of the invention includes an item feature dimension among the feature dimensions in the feature dimension set. The process of performing feature detection on the target image using a preset detection model corresponding to the item feature dimension includes:

[0126] The target image is input into a preset detection model corresponding to the feature dimension of the item to perform item recognition on the target image;

[0127] In the method provided by this invention, the feature detection of the target image includes the detection of items inside the vehicle. A preset detection model corresponding to the feature dimensions of the items can be constructed based on image algorithms. This preset detection model can be called an object detection model. The object detection model can identify the features of items in a specified image. By detecting the shape, color, texture, and other features of objects in the image, it identifies the category of the objects contained in the image. The identified items may include bags, mobile phones, computers, water cups, takeaway food (such as coffee), etc.

[0128] The target image is input into the object detection model, which identifies the items contained in the target image. After processing the target image, the object detection model can output the categories of the identified items.

[0129] Obtain the object category output by the preset detection model corresponding to the item feature dimension, and use the object category as the dimension feature corresponding to the item feature dimension.

[0130] In the method provided by the embodiments of the present invention, the output item category can be obtained from the output layer of the object detection model, and the output item category can be used as the dimension feature corresponding to the item feature dimension.

[0131] Taking the reference item feature dimension as an example, examples of various interactive operations can be found in Table 5:

[0132] Table 5

[0133]

[0134] It should be noted that the specific feature detection processes mentioned in the preceding embodiments are only for better illustrating the method provided by the present invention. They are specific embodiments provided in various feature dimension detection scenarios. In the specific implementation process, the recognition result output by the preset detection model can be used as the dimension feature, or the recognition result output by the preset detection model can be further processed to obtain the dimension feature, depending on the capability of the specific preset detection model.

[0135] Furthermore, in conjunction with practical application scenarios, this embodiment of the invention provides another interaction method. In the method provided by this embodiment of the invention, the in-vehicle system refers to a praise system, which is used to proactively praise and interact with the driver and passengers to bring a pleasant mood to the people in the vehicle and improve the driving atmosphere.

[0136] refer to Figure 3 The example diagram shown illustrates the interaction process provided in this embodiment of the invention, which includes:

[0137] S301: Activate the praise system;

[0138] In the method provided by the embodiments of the present invention, the praise system can be activated when the vehicle system is started.

[0139] S302: Enable image detection;

[0140] In the method provided by this embodiment of the invention, after the praise system is turned on, the camera can be activated to acquire images, and then feature detection can be performed on the acquired images.

[0141] S303: Detects the face, age, gender, mood, makeup, skin, accessories, etc. of occupants in the vehicle;

[0142] In the method provided by the embodiments of the present invention, the collected images can be detected by various pre-built detection models to identify the facial features, age, gender, emotions, makeup, skin, accessories and other characteristics of the people in the vehicle.

[0143] S304: Matching praise scheme;

[0144] In the method provided by this embodiment of the invention, a scheme library is pre-configured, and a praise scheme is matched in the scheme library based on the detected feature information.

[0145] S305: Praises are given to occupants through voice, images, light, and vibration.

[0146] In the method provided by the embodiments of the present invention, the matched praise scheme includes interactive operations such as voice, image, light and vibration (tactile sensation), and by executing each interactive operation, the person in the vehicle is given an interaction with a praise meaning.

[0147] In the interactive process provided by the embodiments of the present invention, the image detection that can be applied mainly includes: face recognition, skin condition detection, makeup detection, clothing matching detection, emotion detection, object detection, child detection, and age and gender detection.

[0148] The method provided in this invention allows for interaction based on facial recognition results. It uses image algorithms to detect the faces of people inside the vehicle, identify their identities, and then provides targeted praise. The primary detection model involved is a facial detection model.

[0149] Face detection model: Identifies key facial features and distinguishes different faces, ages, and genders.

[0150] Praise methods: Different praise methods can be set for different people.

[0151] If in Figure 1 In the scenario shown, interaction via facial recognition can use age and gender as feature dimensions. Figure 1 The face recognition and feature detection methods mentioned in the diagram can be implemented using the same face detection model.

[0152] Examples of praise schemes when only facial recognition is used for interaction are shown in Table 6:

[0153] Table 6

[0154]

[0155] Similar to interaction based on facial recognition to detect the identity of people, the method provided in this embodiment of the invention can interact based on the detection results of age and gender. It also primarily involves a facial detection model, but the focus of the interaction is on age and gender. Image algorithms are used to detect the age and gender of people inside the vehicle, and then specific compliments are given to those individuals. Examples of compliment schemes are shown in Table 7:

[0156] Table 7

[0157]

[0158] Furthermore, the method provided in this embodiment of the invention can detect children based on facial recognition for interactive purposes. Image algorithms can be used to detect whether there are children, their gender, and age among the people inside the vehicle, and praise can be given to the children. The detection model involved is a facial detection model, and examples of praise schemes are shown in Table 8:

[0159] Table 8

[0160]

[0161] The method provided in this embodiment of the invention allows for interaction based on the results of face recognition and skin condition detection. Image algorithms detect the skin condition of occupants inside the vehicle, including indicators such as skin color, blemishes, acne scars, and blackheads, and then compliment the occupant's skin condition. Examples of the compliment scheme can be found in Table 1 of the preceding embodiments, which involves face detection and skin detection models.

[0162] Skin detection model: Identifies skin tone, blemishes, acne scars, blackheads, etc.

[0163] In the method provided by this invention, interaction can be achieved through the results of face detection and makeup detection. Image algorithms detect the makeup of people inside the vehicle, and when makeup is detected or a specific makeup style is matched, praise is given for that person's makeup. Examples of the praise scheme can be found in Table 2 of the preceding embodiments, involving face detection and makeup detection models.

[0164] Makeup detection model: It identifies the style of makeup by recognizing the eyes, mouth, forehead, cheeks, nose, skin tone, lipstick color, and the light and shadow on the bridge of the nose and cheeks.

[0165] The method provided in this embodiment of the invention allows for interaction based on the results of face recognition and clothing matching detection. Image algorithms detect the clothing matching of occupants in the vehicle, including glasses, hats, scarves, necklaces, earrings, and clothes, and then compliment their clothing choices. Examples of the compliment scheme can be found in Table 3 of the preceding embodiments, which involves face detection and clothing detection models.

[0166] Clothing detection model: Identifies the color, brand, and shape of glasses, hats, scarves, necklaces, earrings, and clothing.

[0167] The method provided in this embodiment of the invention allows for interaction based on the results of face recognition and emotion detection. Image algorithms detect the emotions of occupants in the vehicle; if a smile is detected but anger is not, praise is given based on that emotion. Examples of the praise scheme can be found in Table 4 of the preceding embodiments, which involves face detection and emotion detection models.

[0168] Emotion detection model: Detects facial states and identifies emotional expressions such as smiles and anger.

[0169] The method provided in this embodiment of the invention allows for interaction based on the results of face recognition and object detection. Image algorithms detect items inside the vehicle, including bags, mobile phones, computers, water cups, coffee, etc., and then praise the person based on these items. Examples of the praise scheme can be found in Table 5 of the preceding embodiments, which involves face detection and object detection models.

[0170] Object detection model: Detects the shape, color, and texture of objects and identifies the category of the objects.

[0171] Based on the method provided in this invention, an image recognition algorithm can be used to identify various state features of a user, such as face, skin, emotions, makeup, and clothing, and praise can be given through images, sounds, light, and vibrations to bring the user a pleasant mood.

[0172] It should be noted that the specific interactive operations (praise schemes) and the various feature dimensions (detection content) mentioned in the preceding embodiments are all specific embodiments provided to better illustrate the method provided by the present invention. In specific applications, the feature dimensions referenced during interaction, the specific interaction methods, and the interaction schemes can all be set according to actual needs. One of the features mentioned in the preceding embodiments can be used to achieve interaction, or any combination of features can be used, or other features can be adopted. The same applies to the interaction methods, without affecting the functionality of the method provided by the embodiments of the present invention.

[0173] and Figure 1 Corresponding to the interaction method shown, this embodiment of the invention also provides an interaction device, which is applied to an in-vehicle system for use with... Figure 1 The specific implementation of the method shown is illustrated in the following diagram. Figure 4 As shown, it includes:

[0174] The image acquisition unit 401 is used to acquire images of the interior of the current vehicle to obtain a target image;

[0175] The face recognition unit 402 is used to perform face recognition on the target image to obtain the target person corresponding to the target image and the person information corresponding to the target person;

[0176] The first determining unit 403 is used to determine the current feature dimension set, wherein the current feature dimension set contains at least one feature dimension.

[0177] The feature detection unit 404 is used to perform feature detection on the target image by using a preset detection model corresponding to each feature dimension, so as to obtain the dimensional features corresponding to each feature dimension.

[0178] The second determining unit 405 is used to determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method.

[0179] The third determining unit 406 is used to determine the target praise interaction scheme corresponding to each interaction method from among multiple preset praise interaction schemes corresponding to each interaction method, based on the person information and the dimension features corresponding to each feature dimension.

[0180] The fourth determining unit 407 is used to determine the interactive operation corresponding to each interactive method based on the person information, the dimensional features corresponding to each of the feature dimensions, and each of the target praise interaction schemes;

[0181] Interaction unit 408 is used to perform various interaction operations to interact with the target person.

[0182] The device provided in this invention allows for the pre-setting of multiple feature dimensions to be referenced during interaction. When interaction with occupants inside a vehicle is required, the device identifies the occupants' information and current dimensional features based on captured images. Then, based on a pre-set praise-based interaction scheme, it determines praise-laden interactive operations strongly correlated with the occupants' current state for use in the interaction. This enables personalized interaction linked to the real-time state of the interacting object, resulting in better interaction effects and improved user experience. Furthermore, praise-laden interactive operations help create a pleasant driving atmosphere and enhance the enjoyment of the interaction. Additionally, diverse interactive operations increase the fun of the interaction and further improve its effectiveness.

[0183] Based on the apparatus provided in the above embodiments, in the apparatus provided in this embodiment of the invention, each of the feature dimensions includes a skin feature dimension, and the feature detection unit 404 includes:

[0184] The first recognition subunit is used to input the target image into a preset detection model corresponding to the skin feature dimension, so as to perform skin feature recognition on the face image of the target person contained in the target image;

[0185] The first acquisition subunit is used to acquire the skin feature recognition result of the target person output by the preset detection model corresponding to the skin feature dimension;

[0186] The first determining subunit is used to determine the target skin state corresponding to the skin feature recognition result among a preset plurality of skin states, and to use the target skin state as the dimensional feature corresponding to the skin feature dimension.

[0187] Based on the apparatus provided in the above embodiments, in the apparatus provided in this embodiment of the invention, each of the feature dimensions includes a makeup feature dimension, and the feature detection unit 404 includes:

[0188] The second recognition subunit is used to input the target image into a preset detection model corresponding to the makeup feature dimension, so as to perform makeup feature recognition on the face image of the target person contained in the target image;

[0189] The second acquisition subunit is used to acquire the makeup feature recognition result of the target person output by the preset detection model corresponding to the makeup feature dimension;

[0190] The second determining subunit is used to determine the target makeup style corresponding to the makeup feature recognition result among a plurality of preset makeup styles, and to use the target makeup style as the dimension feature corresponding to the makeup feature dimension.

[0191] Based on the apparatus provided in the above embodiments, in the apparatus provided in this embodiment of the invention, each of the feature dimensions includes a clothing feature dimension, and the feature detection unit 404 includes:

[0192] The third identification subunit is used to input the target image into a preset detection model corresponding to the clothing feature dimension, so as to identify the clothing features of the target person in the target image.

[0193] The third acquisition subunit is used to acquire the clothing feature recognition result of the target person output by the preset detection model corresponding to the clothing feature dimension, and to use the clothing feature recognition result as the dimension feature corresponding to the clothing feature dimension.

[0194] Based on the apparatus provided in the above embodiments, in the apparatus provided in this embodiment of the invention, each of the feature dimensions includes an emotion feature dimension, and the feature detection unit 404 includes:

[0195] The fourth recognition subunit is used to input the target image into a preset detection model corresponding to the emotion feature dimension, so as to perform emotion recognition on the face image of the target person contained in the target image;

[0196] The fourth acquisition subunit is used to acquire the emotion recognition result of the target person output by the preset detection model corresponding to the emotion feature dimension, and to use the emotion recognition result as the dimension feature corresponding to the emotion feature dimension.

[0197] Based on the apparatus provided in the above embodiments, in the apparatus provided in this embodiment of the invention, each of the feature dimensions includes an item feature dimension, and the feature detection unit 404 includes:

[0198] The fifth identification subunit is used to input the target image into a preset detection model corresponding to the feature dimension of the item, so as to identify the item in the target image;

[0199] The fifth acquisition subunit is used to acquire the object category output by the preset detection model corresponding to the item feature dimension, and use the object category as the dimension feature corresponding to the item feature dimension.

[0200] Based on the device provided in the above embodiments, the interaction methods provided in the device of the present invention include: voice interaction method, image interaction method, light interaction method and tactile interaction method.

[0201] This invention also provides a storage medium that includes stored instructions, wherein the execution of the instructions controls the device where the storage medium is located to perform the interaction method described above.

[0202] This invention also provides an electronic device, the structural schematic of which is shown below. Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein one or more instructions 502 are stored in the memory 501 and configured to be executed by one or more processors 503 to perform the following operations:

[0203] The interior of the current vehicle is captured to obtain the target image;

[0204] Perform facial recognition on the target image to obtain the target person corresponding to the target image and the person information corresponding to the target person;

[0205] Determine the current set of feature dimensions, wherein the current set of feature dimensions contains at least one feature dimension;

[0206] By using a preset detection model corresponding to each feature dimension, feature detection is performed on the target image to obtain the dimensional features corresponding to each feature dimension;

[0207] Determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method;

[0208] Based on the character information and the dimensional features corresponding to each feature dimension, among the multiple preset praise interaction schemes corresponding to each interaction method, a target praise interaction scheme corresponding to each interaction method is determined;

[0209] Based on the character information, the dimensional features corresponding to each of the aforementioned feature dimensions, and the respective target praise interaction schemes, determine the interaction operation corresponding to each of the aforementioned interaction methods;

[0210] Perform each of the aforementioned interactive operations to interact with the target character.

[0211] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0212] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0213] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interaction method, characterized in that, The method is applied to an in-vehicle system, and the method includes: The system acquires images of the vehicle's interior to obtain the target image. The target image is subjected to facial recognition to obtain the target person corresponding to the target image and the person information corresponding to the target person; the person information includes person attributes, person name, person age, and person gender; the person attributes are used to indicate whether the target person is a registered user; The current set of feature dimensions is determined based on the person information and / or the interaction time point. The current set of feature dimensions includes at least one feature dimension on which the interaction is based. The current set of feature dimensions is different for different interaction time points or different target persons, so as to enable interaction based on different feature dimensions at different time points or for different person information. For each feature dimension included in the current feature dimension set, feature detection is performed on the target image using a preset detection model corresponding to each feature dimension to obtain the dimensional features corresponding to each feature dimension. Determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method; Based on the person information and the dimensional features corresponding to each feature dimension, among the multiple preset praise interaction schemes corresponding to each interaction method, a target praise interaction scheme corresponding to each interaction method is determined; wherein, each preset praise interaction scheme has a corresponding application attribute, the application attribute including the type of person to which the praise interaction scheme is applicable, the person's gender, the person's age range, the applicable feature dimension, and the dimensional features. Based on the person information, the dimension features corresponding to each of the feature dimensions, and each of the target praise interaction schemes, the interaction operation corresponding to each of the interaction methods is determined; wherein, if the target praise interaction scheme requires the application of person information and dimension features, the information data of the person information and each dimension feature is extracted, and the extracted information data is applied to the target praise interaction scheme to generate the interaction operation; Perform each of the aforementioned interactive operations to interact with the target character.

2. The method according to claim 1, characterized in that, Each of the aforementioned feature dimensions includes a skin feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the skin feature dimension includes: The target image is input into a preset detection model corresponding to the skin feature dimension to perform skin feature recognition on the face image of the target person contained in the target image; Obtain the skin feature recognition result of the target person output by the preset detection model corresponding to the skin feature dimension; Among a number of preset skin states, the target skin state corresponding to the skin feature recognition result is determined, and the target skin state is used as the dimensional feature corresponding to the skin feature dimension.

3. The method according to claim 1, characterized in that, Each of the aforementioned feature dimensions includes a makeup feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the makeup feature dimension includes: The target image is input into a preset detection model corresponding to the makeup feature dimension to perform makeup feature recognition on the face image of the target person contained in the target image; Obtain the makeup feature recognition result of the target person output by the preset detection model corresponding to the makeup feature dimension; Among a number of preset makeup styles, the target makeup style corresponding to the makeup feature recognition result is determined, and the target makeup style is used as the dimension feature corresponding to the makeup feature dimension.

4. The method according to claim 1, characterized in that, Each of the aforementioned feature dimensions includes a clothing feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the clothing feature dimension includes: The target image is input into a preset detection model corresponding to the clothing feature dimension to perform clothing feature recognition on the clothing image of the target person contained in the target image; Obtain the clothing feature recognition result of the target person corresponding to the preset detection model corresponding to the clothing feature dimension, and use the clothing feature recognition result as the dimension feature corresponding to the clothing feature dimension.

5. The method according to claim 1, characterized in that, Each of the aforementioned feature dimensions includes an emotion feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the emotion feature dimension includes: The target image is input into a preset detection model corresponding to the emotion feature dimension to perform emotion recognition on the facial image of the target person contained in the target image; Obtain the emotion recognition result of the target person output by the preset detection model corresponding to the emotion feature dimension, and use the emotion recognition result as the dimension feature corresponding to the emotion feature dimension.

6. The method according to claim 1, characterized in that, Each of the aforementioned feature dimensions includes an item feature dimension. The process of performing feature detection on the target image using a preset detection model corresponding to the item feature dimension includes: The target image is input into a preset detection model corresponding to the feature dimension of the item to perform item recognition on the target image; Obtain the object category output by the preset detection model corresponding to the item feature dimension, and use the object category as the dimension feature corresponding to the item feature dimension.

7. The method according to claim 1, characterized in that, The various interaction methods include: voice interaction, image interaction, light interaction, and tactile interaction.

8. An interactive device, characterized in that, The device is used in a vehicle-mounted system, and the device includes: The image acquisition unit is used to acquire images of the interior of the current vehicle and obtain the target image; A face recognition unit is used to perform face recognition on the target image to obtain the target person corresponding to the target image and the person information corresponding to the target person; the person information includes person attributes, person title, person age, and person gender; the person attributes are used to indicate whether the target person is a registered user; The first determining unit is configured to determine the current feature dimension set based on the character information and / or the interaction time point, wherein the current feature dimension set includes at least one feature dimension on which the interaction is based; wherein the current feature dimension set is different for different interaction time points or different target characters, so as to enable interaction based on different feature dimensions at different time points or for different character information. The feature detection unit is used to perform feature detection on the target image for each feature dimension included in the current feature dimension set, using a preset detection model corresponding to each feature dimension, to obtain the dimensional features corresponding to each feature dimension. The second determining unit is used to determine the set of interaction methods corresponding to the current vehicle, wherein the set of interaction methods includes at least one interaction method. The third determining unit is used to determine the target praise interaction scheme corresponding to each interaction method from among multiple preset praise interaction schemes corresponding to each interaction method, based on the person information and the dimension features corresponding to each feature dimension; wherein each preset praise interaction scheme has corresponding application attributes, the application attributes include the type of person to which the praise interaction scheme is applicable, the person's gender, the person's age range, the applicable feature dimension, and the dimension features. The fourth determining unit is used to determine the interactive operation corresponding to each interactive method based on the person information, the dimension features corresponding to each of the feature dimensions, and each of the target praise interaction schemes; wherein, if the target praise interaction scheme requires the application of person information and dimension features, the information data of the person information and each dimension feature is extracted, and the extracted information data is applied to the target praise interaction scheme to generate the interactive operation; An interactive unit is used to perform various interactive operations to interact with the target person.

9. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform the interaction method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors using the interaction method as described in any one of claims 1 to 7.

Images