Virtual object interaction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a virtual object interaction starting method and device, electronic equipment and storage medium. The virtual object interaction starting method comprises: first, obtaining a target weight corresponding to a user initiating an interaction request, the target weight being determined based on a plurality of user information corresponding to the user; then, if the target weight belongs to a preset weight range, starting the interaction for the user. Through the above method, the target weight corresponding to the user initiating the interaction request is obtained through a plurality of user information, and then the obtained target weight is compared with the preset weight range. If the obtained target weight belongs to the preset weight range, the virtual object starts the interaction, so that the interaction starting mode is more diversified, and the user experience is improved.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and specifically relates to a virtual object initiation interaction method, device, electronic device, and readable storage medium. Background Technology

[0002] Existing methods for initiating virtual object interaction typically leave the virtual object in a standby state when no user initiates interaction. It only starts interacting with the user after certain activation conditions are met. Furthermore, virtual objects are usually activated through voice or touch, which is a relatively simple method and can trigger accidental activation under certain circumstances. Summary of the Invention

[0003] In view of the above problems, this application proposes a virtual object initiation interaction method, device, electronic device, and storage medium to improve the above problems.

[0004] In a first aspect, embodiments of this application provide a method for initiating interaction with a virtual object, which is applied to a virtual object. The method includes: obtaining a target weight corresponding to a user who initiates an interaction request, wherein the target weight is determined based on multiple user information corresponding to the user; and if the target weight belongs to a preset weight range, initiating interaction with the user.

[0005] Secondly, embodiments of this application provide a virtual object initiation interaction device, which operates on a virtual object. The device includes a target weight acquisition unit and an initiation interaction unit. The target weight acquisition unit is used to acquire the target weight corresponding to the user who initiates the interaction request. The target weight is determined based on multiple user information corresponding to the user. The initiation interaction unit is used to initiate interaction with the user if the target weight belongs to a preset weight range.

[0006] Thirdly, embodiments of this application provide a virtual object-initiated interactive electronic device, including one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the above-described method.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code, wherein the above-described method is executed when the program code is run.

[0008] This application provides a method, apparatus, electronic device, and storage medium for initiating virtual object interaction. The method includes: first, obtaining a target weight corresponding to the user initiating the interaction request, the target weight being determined based on multiple user information; then, if the target weight falls within a preset weight range, initiating interaction with the user. By using this method, the target weight corresponding to the user initiating the interaction request is obtained through multiple user information, and then compared with a preset weight range. If the obtained target weight falls within the preset weight range, the virtual object initiates interaction, thereby diversifying the interaction initiation methods and improving the user experience. Attached Figure Description

[0009] Figure 1 A flowchart of a virtual object interaction method according to an embodiment of this application is shown;

[0010] Figure 2 A flowchart of a virtual object interaction method according to another embodiment of this application is shown;

[0011] Figure 3 A flowchart of a virtual object interaction method according to another embodiment of this application is shown;

[0012] Figure 4 A flowchart of a virtual object interaction method according to another embodiment of this application is shown;

[0013] Figure 5 A flowchart of a virtual object interaction method according to another embodiment of this application is shown;

[0014] Figure 6 A structural block diagram of a virtual object interaction system according to another embodiment of this application is shown:

[0015] Figure 7 This diagram illustrates a structural block diagram of an electronic device used to execute the virtual object interaction method of the embodiments of this application in real time.

[0016] Figure 8 This application shows a storage unit for storing or carrying program code that implements the virtual object interaction method according to the embodiments of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server comprising a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0019] Virtual objects on the market are generally in standby mode when there is no user interaction. They will only start interacting when the virtual object senses that the user has activated it through voice wake-up, touch, or infrared sensing.

[0020] The inventors, in their research on related virtual object initiation interaction methods, discovered that these methods generally involve identifying a first interactive object from potential interactive objects belonging to a specific category through object feature recognition; outputting interaction initiation information to the first interactive object through a preset virtual avatar; acquiring interactive input information from the first interactive object; processing the interactive input information to obtain interactive response information for feedback to the first interactive object; the interactive response information includes voice response information and a virtual image, synchronized with the voice response information and primarily composed of the virtual avatar; and outputting a voice response and a virtual effect, synchronized with the voice response information and primarily composed of the virtual avatar. Because these methods do not expand upon the virtual object initiation methods, they are simplistic and lack diversity.

[0021] Therefore, the inventors have proposed a virtual object initiation interaction method, apparatus, electronic device, and storage medium in this application. First, a target weight corresponding to the user initiating the interaction request is obtained, the target weight being determined based on various user information. Then, if the target weight falls within a preset weight range, interaction is initiated for the user. Through this method, the target weight corresponding to the user initiating the interaction request is obtained using various user information. The obtained target weight is then compared with a preset weight range. If the obtained target weight falls within the preset weight range, the virtual object initiates interaction, thereby diversifying the interaction initiation methods and improving the user experience.

[0022] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0023] Please see Figure 1 This application provides a method for initiating interaction with a virtual object, applied to a virtual object, the method comprising:

[0024] Step S 110: Obtain the target weight corresponding to the user who initiated the interaction request. The target weight is determined based on various user information corresponding to the user.

[0025] In this embodiment, the virtual object obtains the target weight corresponding to the user before interacting with the user. The obtained target weight is related to the user information. The target weight is a weight value calculated by the virtual object based on the user information. The target weight is obtained from a reference weight and an influence coefficient. The reference weight represents the weight values ​​corresponding to multiple user information items, and the influence coefficient represents the degree of influence of the current application scenario, determined by the application scenario in which the virtual object is currently located, on the multiple user information items.

[0026] Step S 120: If the target weight belongs to the preset weight range, initiate interaction with the user.

[0027] In this embodiment, the obtained target weight corresponding to the user is compared with the boundary value of a preset weight range to determine whether the user's target weight belongs to the preset weight range. If the target weight belongs to the preset weight range, the virtual object initiates interaction. Specifically, initiating interaction means that when the target weight belongs to the preset weight range, the virtual object grants interaction permission and begins interaction.

[0028] As one approach, if the user's target weight falls within a preset weight range, the virtual object can select a launch method based on the target weight's position within that range. Launch methods can include two types: proactive launch, which is used when the user has no clear interaction purpose, and the virtual object can proactively greet and guide the user; and passive launch, which is used when the user has a clear interaction purpose, and the virtual object analyzes the user's intent to select an appropriate interaction method to interact with the user.

[0029] As one approach, the preset weight range can be a range pre-set by the system, or it can be set according to the functional attributes of the virtual object. The attributes of the virtual object can include entertainment, service, and work, and the function of the virtual object can be waiter, salesperson, or technician, without specific limitations.

[0030] Alternatively, if the user's target weight is not within the preset weight range, the virtual object remains in standby mode and does not initiate interaction.

[0031] This application provides a method for initiating virtual object interaction. First, a target weight corresponding to the user initiating the interaction request is obtained. The target weight is determined based on multiple user information parameters. Then, if the target weight falls within a preset weight range, interaction is initiated with the user. This method obtains the target weight corresponding to the user initiating the interaction request using multiple user information parameters, and then compares the obtained target weight with a preset weight range. If the obtained target weight falls within the preset weight range, the virtual object initiates interaction, thereby diversifying the interaction initiation methods and improving the user experience.

[0032] Please see Figure 2 This application provides a method for initiating interaction with a virtual object, applied to a virtual object, the method comprising:

[0033] Step S210: Obtain multiple user information corresponding to the user.

[0034] In this embodiment, the system acquires multiple user information items through multiple external sensors. These sensors may include a distance sensor, a sound sensor, a pressure sensor, and an infrared sensor. The user information includes both subjective and objective information. Subjective information is further divided into explicit and implicit information. Explicit information may include orientation, identity, gender, age, language, and keywords. Implicit information may include emotions, facial expressions, actions, and semantics. Objective information may include distance and contact, without specific limitations. Contact refers to the user's interaction with a virtual object; keywords can be pre-set names or greetings for the virtual object.

[0035] For example, a virtual object can use an external distance sensor to collect the distance between the user and the virtual object, and then acquire the collected distance information.

[0036] Step S220: Input multiple user information corresponding to the user into a pre-trained weight prediction model to obtain the target weight corresponding to the user output by the weight prediction model.

[0037] In this embodiment of the application, after the virtual object obtains multiple user information of the user through multiple external sensors, it inputs the obtained multiple user information into a pre-trained weight prediction model. The weight prediction model calculates the corresponding weights of the obtained multiple user information and combines the weights corresponding to the obtained multiple information into a weight vector. The pre-trained weight prediction model outputs the target weight corresponding to the user based on the obtained weight vector.

[0038] For example, a virtual object acquires user information such as distance, contact, orientation, identity, gender, age, language, keywords, emotion, expression, action, and semantic information through external sensors. This acquired information is then input into a pre-trained weight prediction model. The model generates corresponding vectors for distance, contact, orientation, identity, gender, age, language, keywords, emotion, expression, action, and semantic information. Combining these vectors yields a weight vector, which is then used by the weight prediction model to calculate the user's target weight.

[0039] Step S230: If the target weight belongs to the preset weight range, initiate interaction with the user.

[0040] Step S230 can be referred to in detail in the above embodiments, and therefore will not be repeated in this embodiment.

[0041] This application provides a virtual object-initiated interaction method. First, multiple user information entries corresponding to the user are obtained. Then, this user information is input into a pre-trained weight prediction model to obtain the target weight corresponding to the user, output by the weight prediction model. If the target weight falls within a preset weight range, interaction is initiated with the user. This method obtains the target weight corresponding to the user initiating the interaction request using multiple user information entries, and then compares the obtained target weight with a preset weight range. If the obtained target weight falls within the preset weight range, the virtual object initiates interaction, thereby diversifying the interaction initiation methods and improving the user experience.

[0042] Please see Figure 3 This application provides a method for initiating interaction with a virtual object, applied to a virtual object, the method comprising:

[0043] Step S310: Obtain the target weights corresponding to the multiple users who initiated the interaction request.

[0044] In this embodiment of the application, if there are multiple users initiating the interaction request, the virtual object obtains the interaction requests initiated by the multiple users, responds to the interaction requests of the multiple users, activates multiple external sensors, obtains multiple user information corresponding to each user through the multiple external sensors, and obtains the target weight corresponding to each user based on the multiple user information.

[0045] Step S320: Obtain the target user from the plurality of users, wherein the target user is the user whose corresponding target weight belongs to the preset weight range.

[0046] In this embodiment, after obtaining the target weights corresponding to multiple users, a target user is selected from among the multiple users based on a comparison between the target weights of the multiple target users and a preset weight range. The target user is used to represent a user whose target weight belongs to the preset weight range.

[0047] Step S330: Initiate interaction with the target user.

[0048] In this embodiment of the application, after obtaining the target user, the virtual object initiates interaction with the target user.

[0049] Alternatively, if the user is not a target user, the virtual object will not initiate interaction.

[0050] This application provides a method for initiating virtual object interaction. First, it obtains the target weights corresponding to multiple users who initiated the interaction request. Then, it selects a target user from among the multiple users, where the target user's target weight falls within a preset weight range. Finally, it initiates interaction with the target user. This method obtains the target weights corresponding to the users initiating the interaction request using various user information. Then, it compares the obtained target weights with a preset weight range. If the obtained target weights fall within the preset weight range, the virtual object initiates interaction, thus diversifying the interaction initiation methods and improving the user experience.

[0051] Please see Figure 4 This application provides a method for initiating interaction with a virtual object, applied to a virtual object, the method comprising:

[0052] Step S401: If the multiple users initiate an interaction request at the same time, obtain multiple user information of the multiple users.

[0053] In this embodiment, the virtual object acquires multiple user information corresponding to each user through multiple external sensors. The user information includes two parts: subjective information and objective information. Subjective information is further divided into explicit and implicit information. Explicit information may include orientation, identity, gender, age, language, and keywords, while implicit information may include emotions, facial expressions, actions, and semantics. Objective information may include distance and contact. For example, when each user initiates an interaction request, the virtual object activates an external voice sensor to acquire the language and keyword information of multiple users.

[0054] Step S402: Based on the multiple user information, obtain the reference weight and influence coefficient corresponding to each of the multiple user information.

[0055] In this embodiment, a corresponding reference weight and influence coefficient are obtained based on multiple user information pieces acquired from the virtual object. The reference weight represents the weight value obtained based on the user information when multiple user information pieces are acquired, and the influence coefficient represents the degree of influence of the current application scenario, determined by the application scenario in which the virtual object is currently located, on the multiple user information pieces.

[0056] The reference weights corresponding to multiple user information items are also related to multiple confidence levels corresponding to the outputs of multiple machine learning engines in the virtual avatar. For example, these multiple machine learning engines may include a face recognition engine, a speech recognition engine, a voiceprint recognition engine, an expression recognition engine, a pose recognition engine, a natural language processing engine, an age recognition engine, a gender recognition engine, an emotion recognition engine, and a language recognition engine. The confidence levels corresponding to the outputs of these multiple machine learning engines are, for example, the confidence level C of the face recognition engine. FR Confidence C of speech recognition engine ASR Confidence C of voiceprint recognition engine VPR Confidence C of facial expression recognition engine EXR Confidence C of the pose recognition engine PR Confidence C of Natural Language Processing Engine NLR Age recognition engine confidence level C AR Confidence C of gender recognition engine GR Confidence C of the emotion recognition engine EMR Language recognition engine confidence level C LR .

[0057] For example, the calculation method for the reference weights corresponding to multiple user information is as follows: for the distance weight x... d When the distance d between the user and the virtual object is greater than the preset distance threshold T D At that time, the distance weight x d Set to -1 if it is less than or equal to the distance threshold T D Then the distance weight is set to Where D is the maximum distance among multiple users and the virtual object; for the contact weight, when a user makes contact with the virtual object, the contact weight x is increased. t Set to -1; if there is no contact, then the contact weight x will be increased. t Set to 0; for orientation weight, when the user is facing the virtual object, set the orientation angle 'a' to 0; if the user has their back to the virtual object, set the orientation angle to 180. The orientation weight calculation formula is: Among them, C FR For the facial recognition engine's confidence score; regarding identity weight, after obtaining the user's identity, the virtual object compares the obtained identity with the pre-stored identity information in the database. If the user's information is pre-stored in the database, it is determined that the user has interaction permissions, and the identity weight is x. i Set to 1; if there is no interaction permission, then the identity weight will be x. i Set to -1, while the face recognition engine confidence C FR With confidence level C of voiceprint recognition engine VPR Greater than the preset identity threshold T IAt this time, the virtual object adds identity activation features, such as addressing the user by name; regarding keyword weight, when the user speaks a preset keyword to the virtual object, the keyword weight is x. d =1*C ASR If no preset keywords are specified, the keyword weight is x. d =-1*C ASR C ASR For the confidence score of the speech recognition engine, preset keywords can be the names of virtual objects and greetings; for facial expression weighting, users are categorized into attentive, semi-attentive, and unattended states based on their facial expressions. If a user is attentive, the facial expression weight is set to x. e =1*C EXR If the user is in a semi-attentive state, then the emoji weight is set to x. e =0.5*C EXR If the user is not following, the emoji weight is set to x. e =-1*C EXR C EXR For the confidence score of the facial expression recognition engine, the user's state can be determined based on the duration of their gaze on a virtual object. A gaze focused on the virtual object for more than 3 seconds is considered an attentive state, less than 3 seconds is a semi-attentive state, and no gaze on the virtual object is considered an attentive state. For action weights, users are categorized into direct, indirect, and other states based on their actions. If the user is in a direct state, the action weight is set to x. p =1*C PR If the user is in an indirect state, then the action weight is set to x. p =0.5*C PR If the user is in another state, the action weight is set to x. p =-1*C PR C PR For the confidence score of the pose recognition engine, the user can be categorized into three states: waving at a virtual object (direct state), pointing at a virtual object (indirect state), and the user not pointing at a virtual object (other states). For semantic weights, the user is categorized into inquiry, small talk, and other states based on their semantic meaning. If the user is in an inquiry state, the semantic weight is set to x. n =1*C NLP If the user is in a conversational mood, the semantic weight is set to x. n =0.5*C NLP If the user is in another state, the semantic weight is set to x. n =-1*C NLP C NLP Confidence level for natural language processing engines.

[0058] Step S403: Based on the reference weights and influence coefficients corresponding to the multiple user information, determine the target weights corresponding to each user to obtain the target weights corresponding to the multiple users.

[0059] In this embodiment, after obtaining the reference weights and influence coefficients corresponding to multiple user information, the target weight of the user is calculated from the reference weights and influence coefficients. When there are multiple users, each user corresponds to a multiple target weight.

[0060] For example, the distance weight of the user obtained by the virtual object is x. d The contact weight is x t Orientation weight is x a The identity weight is x i Keyword weight is x k The weight of facial expression is x e The action weight is x p The semantic weight is x n The distance influence coefficient obtained at the same time is a D The contact influence coefficient is a T The orientation influence coefficient is a A The influence coefficient of identity is a I The keyword influence coefficient is a K The influence coefficient of facial expression is a E The action influence coefficient is a P The semantic influence coefficient is a N By combining the reference weights and influence coefficients corresponding to the multiple user information pieces obtained, the target reference weight of the user can be obtained. The formula for calculating the target reference weight obtained from the user information can be X. i =a D *x d +a T *x t +a A *x a +a I *x i +a K *x k +a E *x e +a P *x p +a N *x n The target weight X can be obtained by normalizing the obtained target reference weights. The larger the user's target weight, the more obvious the user's interaction purpose. The target weight X ranges from -1 to 1.

[0061] Step S404: Based on the target weight, prioritize the multiple users to obtain the priority ranking result.

[0062] In this embodiment, after obtaining the target weight, users are prioritized according to their target weight values, from highest to lowest. Users with larger target weight values ​​have higher priority than those with smaller target weight values. The prioritized users are then sorted to obtain a priority ranking result. The priority ranking result represents the numerical order of the users' target weights.

[0063] Step S405: Based on the priority sorting result, determine whether to initiate interaction for the multiple users.

[0064] In this embodiment, after the system obtains the priority ranking results of multiple users, it determines whether the virtual object should initiate interaction with the multiple users based on the priority ranking results. Specifically, the system checks whether the user with the highest priority among the multiple users belongs to a preset weight range. If it does not, it means that none of the users belong to the preset weight range, and the virtual object does not initiate interaction with any user. If it does, it checks whether the user with the next lower priority relative to the highest priority belongs to the preset weight range, and so on.

[0065] Step S406: Obtain the target user from the plurality of users, wherein the target user is the user whose corresponding target weight belongs to the preset weight range.

[0066] Step S406 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0067] Step S407: If the target weight belongs to the first weight range, actively initiate interaction with the user.

[0068] As one approach, the preset weight range is [-1, 1]. Within the preset weight range, there are a pre-set first threshold a and a pre-set second threshold b, and the second threshold b is greater than the first threshold a. The preset weight range is divided into three weight ranges by the first threshold a and the second threshold b. The three weight ranges are the first weight range, the second weight range, and the third weight range, where the first weight range is (a, b), the second weight range is [b, 1], and the third weight range is [-1, a].

[0069] In this embodiment, after obtaining the target weights corresponding to multiple users, the position of the obtained target weights within a preset weight range is detected. If the value of the target weight is between (a, b), it is determined that the target weight is within the first weight range, and the virtual object selects an active activation method for the user. Active activation occurs when the user has no obvious interaction purpose, and the virtual object guides the user to interact by actively greeting the user, etc.

[0070] Step S408: If the target weight belongs to the second weight range, passively initiate interaction with the user.

[0071] In this embodiment, after obtaining target weights for multiple users, the system detects the position of the obtained target weights within a preset weight range. If the target weight is within [b, 1], the system determines that the target weight is within the second weight range, and the virtual object selects a passive activation method for the user. Passive activation indicates that the user has a clear interaction purpose. The virtual object analyzes the user's intent using multiple user information pieces and selects an appropriate interaction method to interact with the user based on that intent.

[0072] Step S409: If the target weight belongs to the third weight range, do not initiate interaction with the user.

[0073] In this embodiment of the application, after obtaining the target weights corresponding to multiple users, the system detects the position of the obtained target weights within the preset weight range. If the target weight is in [-1, b], the system determines that the target weight is in the third weight range, and the virtual object chooses to continue to standby.

[0074] Step S410: Based on the priority sorting result, initiate interaction with the target users in sequence.

[0075] In this embodiment of the application, after obtaining the priority ranking results of multiple users, the interaction is initiated with the target user according to the priority ranking results.

[0076] This application provides a virtual object initiation interaction method. First, if multiple users simultaneously initiate interaction requests, multiple user information of the multiple users is obtained. Then, based on the multiple user information, a reference weight and influence coefficient corresponding to each of the multiple user information are obtained. Next, based on the reference weight and influence coefficient corresponding to each of the multiple user information, a target weight corresponding to each user is determined to obtain the target weights corresponding to each of the multiple users. Then, based on the target weights, the multiple users are prioritized to obtain a priority ranking result. Next, based on the priority ranking result, it is determined whether to initiate interaction with the multiple users. A target user is obtained from the multiple users. The target user is the user whose target weight belongs to a preset weight range. If the target weight belongs to a first weight range, interaction is actively initiated with the user. If the target weight belongs to a second weight range, interaction is passively initiated with the user. If the target weight belongs to a third weight range, interaction is not initiated with the user. Finally, based on the priority ranking result, interaction is initiated with the target users sequentially. The above method obtains the target weight corresponding to the user who initiates the interaction request by using various user information. Then, the obtained target weight is compared with a preset weight range. If the obtained target weight belongs to the preset weight range, the virtual object starts the interaction, thereby making the interaction initiation method more diversified and improving the user experience.

[0077] Please see Figure 5 This application provides a method for initiating interaction with a virtual object, applied to a virtual object, the method comprising:

[0078] Step S501: If the multiple users initiate interaction requests simultaneously, obtain multiple user information of the multiple users.

[0079] Step S501 can be specifically explained in the detailed explanation of the above embodiments, and therefore will not be repeated in this embodiment.

[0080] Step S502: Obtain the application scenario of the virtual object.

[0081] In this application embodiment, the application scenarios of the virtual object may include scenarios with sufficient lighting and no background noise, scenarios with insufficient lighting or overexposure, and scenarios with excessive background noise. The virtual object obtains the current application scenario through multiple external sensors and determines the current application scenario based on the data obtained by the external sensors.

[0082] Step S503: Based on the application scenario, determine the influence coefficients corresponding to the multiple user information for each user.

[0083] In this embodiment, the influence coefficients of multiple pieces of information are determined according to different application scenarios of the virtual object. Specifically, when there is sufficient lighting and no background noise, the system sets the influence coefficient to 0. When lighting is insufficient, the influence coefficients of visual information decrease according to the degree of insufficient lighting, and information such as identity, gender, age, and emotion are switched from image-based recognition methods to speech-based recognition methods. The influence factors of visual information, including orientation, facial expressions, and actions, decrease. When background noise is too strong, the influence coefficients of auditory information decrease sequentially according to the intensity of the background noise, and information such as identity, gender, age, and emotion are switched to speech-based recognition methods. Auditory information may include language, keywords, and speech.

[0084] In one approach, in a single-user scenario, the system directly adjusts the influence coefficient based on the current application scenario of the virtual object; in a multi-user scenario, the system uses image segmentation, voice separation, and other technologies to separate the user information of each user, and then determines the influence coefficient for the current application scenario based on the separated user information.

[0085] Step S504: Obtain the confidence coefficients corresponding to each of the multiple user information.

[0086] In this embodiment, multiple processors exist within the virtual object. These processors may include a distance detection engine, a stress detection engine, a face recognition engine, a speech recognition engine, a voiceprint recognition engine, an expression recognition engine, a pose recognition engine, a natural language processing engine, an age recognition engine, a gender recognition engine, an emotion recognition engine, and a language recognition engine. The face recognition engine, speech recognition engine, voiceprint recognition engine, expression recognition engine, pose recognition engine, natural language processing engine, age recognition engine, gender recognition engine, emotion recognition engine, and language recognition engine are related to machine learning methods. Engines involving machine learning methods output a corresponding confidence level; for example, the face recognition engine corresponds to a face recognition engine confidence level C. FR The speech recognition engine corresponds to the speech recognition engine confidence level C. ASR The voiceprint recognition engine corresponds to the voiceprint recognition engine confidence level C. VPR The facial expression recognition engine corresponds to the confidence level C of the facial expression recognition engine. EXR The pose recognition engine corresponds to the pose recognition engine confidence level C. PR Natural Language Processing Engine (NLP) corresponds to the confidence level C of the NLP engine. NLR The age recognition engine corresponds to the confidence level C of the age recognition engine. AR The gender recognition engine corresponds to the confidence level C of the gender recognition engine. GRThe emotion recognition engine corresponds to the confidence level C of the emotion recognition engine. EMR And the language recognition engine corresponds to the language recognition engine confidence level C. LR When the system calculates the weights corresponding to multiple user information entries, the weights for some user information entries need to be calculated using corresponding confidence coefficients. For example, the formula for calculating the orientation weight is... 'a' represents the angle from which the user is facing the virtual object.

[0087] Step S505: Based on the confidence coefficient, determine the reference weights and activation features corresponding to multiple user information for each user.

[0088] In this embodiment, the weight calculation for some user information involves confidence coefficients. Reference weights and activation features for multiple user information items are determined based on multiple confidence coefficients obtained from the outputs of multiple processors. For example, for orientation weights, when the user is facing a virtual object, the orientation angle α is set to 0; if the user is facing away from the virtual object, the orientation angle is set to 180°. The orientation weight calculation formula is as follows: Among them, C FR Let C be the confidence level of the face recognition engine. For identity priming features, when the face recognition engine confidence level C... FR And the confidence level C of the voiceprint recognition engine VPR All are greater than the preset threshold T I When the identity-based activation feature is enabled, the virtual object will address the user by name when interacting with the user; for the gender-based activation feature, when the gender recognition engine's confidence level C... GR Greater than the preset gender recognition threshold T G When the gender-based priming feature is enabled, virtual objects will be addressed as "Mr." or "Ms." when interacting with the user; for the age-based priming feature, when the age recognition engine's confidence level C... AR Greater than the preset age recognition threshold T A When the age-based priming feature is enabled, the virtual object's speech rate slows down as the user interacts with the user, especially if the user is older. For the language-based priming feature, when the language recognition engine's confidence level C... LR Greater than the preset language recognition threshold T L When the language activation feature is enabled, the virtual object will set the corresponding response language according to the user's language when interacting with the user.

[0089] Step S506: Based on the reference weights and influence coefficients corresponding to the multiple user information, determine the target weights corresponding to each user to obtain the target weights corresponding to the multiple users.

[0090] Step S506 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0091] Step S507: Based on the target weight, prioritize the multiple users to obtain the priority ranking result.

[0092] Step S507 can be found in the detailed explanation in the above embodiments, and therefore will not be repeated in this embodiment.

[0093] Step S508: Based on the priority sorting result, determine whether to initiate interaction for the multiple users.

[0094] Step S508 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0095] Step S509: Obtain the target user from the plurality of users, wherein the target user is the user whose corresponding target weight belongs to the preset weight range.

[0096] Step S509 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0097] Step S510: If the target weight belongs to the first weight range, actively initiate interaction with the user.

[0098] Step S510 can be specifically explained in the above embodiments, and therefore will not be repeated in this embodiment.

[0099] Step S511: If the target weight belongs to the second weight range, passively initiate interaction with the user.

[0100] Step S511 can be found in the detailed explanation in the above embodiments, and therefore will not be repeated in this embodiment.

[0101] Step S512: If the target weight belongs to the third weight range, do not initiate interaction with the user.

[0102] Step S512 can be found in the detailed explanation in the above embodiments, and therefore will not be repeated in this embodiment.

[0103] Step S513: Based on the priority sorting results, initiate interaction with the target users in sequence.

[0104] Step S513 can be found in the detailed explanation in the above embodiments, and therefore will not be repeated in this embodiment.

[0105] Step S514: Obtain the current application scenario of the virtual object.

[0106] In this embodiment of the application, the virtual object obtains the current application scenario based on multiple external sensors. The application scenario may include a scene with sufficient lighting and no background noise, a scene with insufficient lighting or overexposure, and a scene with excessive background noise.

[0107] Step S515: Based on the current application scenario of the virtual object, enable the interaction state corresponding to the application scenario.

[0108] In this embodiment, based on the acquired current application scenario, the virtual object activates an interactive state corresponding to the current application scenario. The virtual object's interactive state includes three types: first, an audiovisual state, where it responds with voice while also making corresponding actions and facial expressions; second, a visual state, where the virtual object only responds with sign language and displays subtitles; and third, an auditory state, where the virtual object only responds with voice. For example, if the acquired current application scenario is one with sufficient lighting and no background noise, the virtual object adopts an audiovisual state; in application scenarios with insufficient or overexposed lighting, the brightness of the virtual object's image is adjusted according to the lighting intensity of the current application scenario; in application scenarios with background noise, the virtual object adjusts the voice volume during interaction according to the intensity of the background noise; if the system detects that the user is not interacting with voice but only with sign language, the system determines that the current user is deaf or mute, and the virtual object adopts a visual state, only responding with sign language or subtitles; if the system detects that the user is interacting with their eyes closed, the system determines that the current user is blind, and the virtual object adopts an auditory state, only responding with voice.

[0109] Step S516: Interact with the user based on the interaction state.

[0110] In this embodiment of the application, the virtual object activates an interactive state corresponding to the current application scenario and interacts with the user based on the interactive state.

[0111] As one approach, the virtual object uses single-threaded computation in single-user scenarios and multi-threaded computation in multi-user scenarios. It separates and independently processes the user information of multiple users using techniques such as image segmentation and speech separation, and performs parallel computation on the user information corresponding to multiple users, outputting interaction schemes for each user. When the virtual object interacts with multiple users, if the interaction schemes do not conflict, the virtual object can interact with multiple users simultaneously. If the interaction schemes conflict, the virtual object prioritizes interacting with the user with the higher target weight. Whether or not there is a conflict depends on the pre-designed logic of the system. For example, if one user's interaction scheme is a photo carousel and another user's interaction scheme is to play music, the virtual object can interact with both users simultaneously. If both users' interaction schemes are to chat, then there is a conflict, and the virtual object interacts with the users sequentially according to their priority.

[0112] As another approach, if a new user is inserted while the virtual object is interacting with a user, and the new user initiates an interaction request, the system obtains the corresponding target weight based on the new user's multiple user information. If the new user's target weight is greater than the target weight of the currently interacting user, the virtual object interrupts its interaction with the current user, stores the current interaction state, and responds to the interaction request initiated by the new user, interacting with the new user. If the new user's target weight is less than the target weight of the currently interacting user, the new user is ranked in the priority sorting results according to the new user's target weight.

[0113] This application provides a virtual object initiation interaction method. First, if multiple users simultaneously initiate interaction requests, the method obtains multiple user information for each user. Then, it obtains the application scenario of the virtual object. Based on the application scenario, it determines the influence coefficient corresponding to each user's multiple user information pieces. Next, it obtains the confidence coefficient corresponding to each user's multiple user information pieces. Based on the confidence coefficient, it determines the reference weight and initiation feature corresponding to each user's multiple user information pieces. Based on the reference weight and influence coefficient, it determines the target weight for each user, thus obtaining the target weights for each of the multiple users. Finally, based on the target weights, it prioritizes the multiple users to obtain a priority ranking. Based on the priority ranking results, it is determined whether to initiate interaction with the multiple users. A target user is selected from the multiple users; the target user is the user whose target weight belongs to a preset weight range. If the target weight belongs to a first weight range, interaction is initiated proactively with that user. If the target weight belongs to a second weight range, interaction is initiated passively with that user. If the target weight belongs to a third weight range, interaction is not initiated with that user. Based on the priority ranking results, interaction is initiated with the target users sequentially. Then, the current application scenario of the virtual object is obtained, and an interaction state corresponding to the application scenario is activated based on the current application scenario. Interaction is then performed with the user based on this interaction state. Through this method, the target weight corresponding to the user initiating the interaction request is obtained through various user information. The obtained target weight is then compared with a preset weight range. If the obtained target weight belongs to the preset weight range, the virtual object initiates interaction, thus diversifying the interaction initiation methods and improving the user experience.

[0114] Please see Figure 6 This application provides a virtual object launch interaction device 600, which runs on a virtual object. The device 600 includes:

[0115] The target weight acquisition unit 610 is used to acquire the target weight corresponding to the user who initiated the interaction request. The target weight is determined based on multiple user information corresponding to the user.

[0116] In one way, the target weight acquisition unit 610 is also used to acquire the target weights corresponding to each of the multiple users who initiated the interaction request.

[0117] Optionally, the target weight acquisition unit 610 is further configured to, if the multiple users simultaneously initiate an interaction request, acquire multiple user information of the multiple users; acquire reference weights and influence coefficients corresponding to each of the multiple user information based on the multiple user information; and determine the target weight corresponding to each user based on the reference weights and influence coefficients corresponding to each of the multiple user information, so as to obtain the target weights corresponding to each of the multiple users.

[0118] Optionally, the target weight acquisition unit 610 is also used to acquire the application scenario of the virtual object; and based on the application scenario, determine the influence coefficients corresponding to the multiple user information for each user.

[0119] Optionally, the target weight acquisition unit 610 is further configured to acquire the confidence coefficients corresponding to each of the multiple user information; and based on the confidence coefficients, determine the reference weights and activation features corresponding to the multiple user information for each user.

[0120] Optionally, the target weight acquisition unit 610 is further configured to acquire multiple user information corresponding to the user; input the multiple user information corresponding to the user into a pre-trained weight prediction model, and acquire the target weight corresponding to the user output by the weight prediction model.

[0121] The interaction unit 620 is used to initiate interaction with the user if the target weight belongs to a preset weight range.

[0122] In one manner, the interaction unit 620 is also used to obtain a target user from the plurality of users, wherein the target user is a user whose corresponding target weight belongs to the preset weight range; and to initiate interaction with the target user.

[0123] Optionally, the interaction initiation unit 620 is further configured to prioritize the plurality of users based on the target weight to obtain a priority ranking result; determine whether to initiate interaction with the plurality of users based on the priority ranking result; and initiate interaction with the target user sequentially based on the priority ranking result.

[0124] Optionally, the interaction initiation unit 620 is further configured to: actively initiate interaction with the user if the target weight belongs to a first weight range; passively initiate interaction with the user if the target weight belongs to a second weight range; and not initiate interaction with the user if the target weight belongs to a third weight range.

[0125] Optionally, the interaction unit 620 is further configured to obtain the current application scenario of the virtual object; activate the interaction state corresponding to the current application scenario of the virtual object; and interact with the user based on the interaction state.

[0126] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0127] The following will combine Figure 7 This application describes an electronic device.

[0128] Please see Figure 7 Based on the aforementioned virtual object interaction method and apparatus, this application also provides another electronic device 700 capable of executing the aforementioned virtual object interaction method. The electronic device 700 includes one or more (only one shown in the figure) processors 702, a memory 704, and a network module 707 coupled together. The memory 704 stores programs capable of executing the contents of the aforementioned embodiments, and the processors 702 can execute the programs stored in the memory 704.

[0129] The processor 702 may include one or more processing cores. The processor 702 connects to various parts within the electronic device 700 using various interfaces and lines, and executes various functions of the server 700 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 704, and by calling data stored in the memory 704. Optionally, the processor 702 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 702 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 702 and may be implemented separately using a communication chip.

[0130] The memory 704 may include random access memory (RAM) or read-only memory (ROM). The memory 704 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 704 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 700 during use (such as phonebook data, audio and video data, chat log data, etc.).

[0131] The network module 706 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby communicating with communication networks or other devices, such as audio playback devices. The network module 706 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM cards), memory, etc. The network module 706 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices through wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). For example, the network module 706 can interact with base stations.

[0132] Please refer to Figure 8 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 800 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0133] The computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.

[0134] This application provides a method, apparatus, electronic device, and storage medium for initiating virtual object interaction. The method includes: first, obtaining a target weight corresponding to the user initiating the interaction request, the target weight being determined based on multiple user information; then, if the target weight falls within a preset weight range, initiating interaction with the user. By using this method, the target weight corresponding to the user initiating the interaction request is obtained through multiple user information, and then compared with a preset weight range. If the obtained target weight falls within the preset weight range, the virtual object initiates interaction, thereby diversifying the interaction initiation methods and improving the user experience.

[0135] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other modifications under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these modifications are protected by the present invention.

Claims

1. A method for initiating interaction with a virtual object, characterized in that, Applied to virtual objects, the method includes: If multiple users initiate interaction requests simultaneously, obtain multiple user information of the multiple users and the confidence coefficient corresponding to each of the multiple user information; Based on the confidence coefficient, reference weights and activation features corresponding to multiple user information for each user are determined; if the confidence coefficient of the user information is greater than the corresponding preset threshold, the activation feature corresponding to the user information is activated; the activation feature includes at least one of identity activation feature, age activation feature, gender activation feature, and language activation feature; when the identity activation feature is activated, the virtual object addresses the user by name when interacting with the user; when the gender activation feature is activated, the virtual object adds the title "Mr" or "Ms" when interacting with the user; when the age activation feature is activated, if the virtual object detects that the user is older when interacting with the user, the virtual object slows down its speech; when the language activation feature is activated, the virtual object sets the corresponding response language according to the user's language when interacting with the user. Based on the multiple user information, obtain the reference weight and influence coefficient corresponding to each of the multiple user information; the influence coefficient is used to characterize the degree of influence of the current application scenario determined by the application scenario in which the virtual object is currently located on the multiple user information. Based on the reference weights and influence coefficients corresponding to the multiple user information, the target weights corresponding to each user are determined to obtain the target weights corresponding to the multiple users. If the user's target weight falls within a preset weight range, initiate interaction with the user. Wherein, the step of initiating interaction with the user if the target weight belongs to a preset weight range includes: If the target weight falls within the first weight range, an interaction is initiated with the user. If the target weight falls within the second weight range, the interaction is passively initiated with the user. If the target weight falls within the third weight range, no interaction will be initiated with the user.

2. The method according to claim 1, characterized in that, The step of initiating interaction with the user if the target weight falls within a preset weight range further includes: A target user is obtained from the plurality of users, wherein the target user is a user whose corresponding target weight belongs to the preset weight range; Initiate interaction with the target user.

3. The method according to claim 2, characterized in that, The step of determining the target weight for each user based on the reference weights and influence coefficients corresponding to the multiple user information, to obtain the target weights for each of the multiple users, further includes: Based on the target weight, the multiple users are prioritized to obtain the priority ranking result; Based on the priority ranking result, determine whether to initiate interaction for the multiple users; The step of initiating interaction with the target user includes: Based on the priority ranking results, interactions are initiated with the target users in sequence.

4. The method according to claim 1, characterized in that, Before obtaining the reference weights and influence coefficients corresponding to each of the multiple user information pieces based on the multiple user information pieces, the method further includes: Obtain the application scenarios of the virtual object; Based on the application scenario, the influence coefficients corresponding to multiple user information for each user are determined.

5. The method according to claim 1, characterized in that, The process of determining the target weight also includes: Obtain multiple user information corresponding to the user; The user's information is input into a pre-trained weight prediction model to obtain the target weight of the user output by the weight prediction model.

6. The method according to claim 1, characterized in that, If the target weight falls within a preset weight range, the process of initiating interaction with the user further includes: Obtain the current application scenario of the virtual object; Based on the current application scenario of the virtual object, an interactive state corresponding to the application scenario is activated; wherein, the interactive state of the virtual object includes at least one of an audiovisual state, a visual state, and an auditory state; the audiovisual state refers to making a voice response while also making corresponding actions and facial expressions; the visual state refers to the virtual object only making sign language responses and displaying subtitles; the auditory state refers to the virtual object only making voice responses; Interact with the user based on the interaction state.

7. A virtual object activation interaction device, characterized in that, Running on virtual objects, the device includes: The target weight acquisition unit is used to acquire multiple user information and confidence coefficients corresponding to each of the multiple user information if multiple users simultaneously initiate interaction requests; based on the confidence coefficients, determine the reference weights and activation features corresponding to the multiple user information for each user; if the confidence coefficient of the user information is greater than a corresponding preset threshold, activate the activation feature corresponding to the user information; the activation feature includes at least one of identity activation feature, age activation feature, gender activation feature, and language activation feature; when the identity activation feature is enabled, the virtual object addresses the user by name when interacting with the user; when the gender activation feature is enabled, the virtual object adds a first... When addressing a user as "male" or "female" and enabling age-based activation, if the virtual object detects that the user is older, its speech rate slows down. When enabling language-based activation, the virtual object sets the corresponding response language based on the user's language. Based on the multiple user information entries, reference weights and influence coefficients are obtained for each entry. The influence coefficients characterize the degree of influence of the current application scenario (as determined by the virtual object's current application scenario) on the multiple user information entries. Based on the reference weights and influence coefficients corresponding to each user entry, a target weight is determined for each user to obtain the target weights for each of the multiple users. The interaction activation unit is used to activate interaction with the user if the user's target weight falls within a preset weight range. The interaction initiation unit is further configured to: actively initiate interaction with the user if the target weight belongs to a first weight range; passively initiate interaction with the user if the target weight belongs to a second weight range; and not initiate interaction with the user if the target weight belongs to a third weight range.

8. An electronic device, characterized in that, It includes one or more processors and memory, wherein one or more programs are stored in the memory and configured to be executed by one or more processors according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, which includes instructions for performing the method as claimed in any one of claims 1-6.

Images