Topic shifting method and device, electronic equipment and storage medium

Abstract

The application provides a topic transfer method and device, electronic equipment and a storage medium, wherein the method comprises: determining the topic interest degree of a user for current topic content based on at least one of the response duration, response richness and response correlation degree of the user response content in the interaction data; determining the target user emotion from each user emotion based on the time sequence relationship between each user emotion and the difference between adjacent user emotions in each user emotion; and in the case that the topic interest degree is lower than the preset interest degree threshold, determining the transfer topic content based on the target user emotion and outputting the transfer topic content, which overcomes the defects in the prior art that the interest evaluation process for the user is relatively rough, the topic transfer is very abrupt, and the user experience is poor, realizes the introduction of other topic content on the basis of the current topic content, guarantees the continuity and coherence of the conversation process and the correlation and appropriateness of the topic content, and optimizes the user experience.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a topic-shifting method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of science and technology and the continuous progress of society, people are paying more and more attention to robots, especially in the field of human-computer interaction. It is particularly important for robots to be more human-like. For example, in the field of virtual humans or companion robots, the strength of the human-like characteristics of the robot can directly affect the user experience.

[0003] Currently, robots can interact with users through text and voice during human-computer conversations. They can also adjust the topic content in a timely manner if they find that the user's interest in the current topic is not high. However, the current judgment of the user's interest is often rather crude, and there are often large differences between the content before and after the adjustment process. This not only leads to low accuracy in the topic adjustment process, but also makes the topic change very abrupt, resulting in a poor user experience. Summary of the Invention

[0004] This invention provides a topic shifting method, apparatus, electronic device, and storage medium to address the shortcomings of existing technologies, such as coarse evaluation of user interest status and abrupt topic shifts that result in significant differences in content before and after the shift, leading to a poor user experience. This invention improves the interest evaluation process and optimizes the user experience.

[0005] This invention provides a topic shifting method, comprising:

[0006] Determine the interaction data during the conversation, as well as multiple user emotions, wherein the interaction data includes at least one of the following: response duration, response richness, and response relevance.

[0007] Based on the interaction data, determine the user's level of interest in the current topic content;

[0008] Based on the temporal relationship between various user emotions and the degree of difference between adjacent user emotions, the target user emotion is determined from the various user emotions.

[0009] If the interest level of the topic is lower than a preset interest level threshold, the topic content to be shifted is determined based on the target user's sentiment, and the shifted topic content is output.

[0010] According to a topic shifting method provided by the present invention, determining the user's topic interest level for the current topic content based on the interaction data includes:

[0011] The user's topic interest in the current topic content is determined based on at least one of the first interest level, the second interest level, and the third interest level.

[0012] The first interest level is determined based on the response duration of the user's response content and a first threshold; the second interest level is determined based on the response richness of the user's response content and a second threshold; and the third interest level is determined based on the response relevance of the user's response content and a third threshold.

[0013] According to a topic shifting method provided by the present invention, the richness of the response is determined based on the character length of the user's response content and / or the content collection time.

[0014] The response relevance is determined based on the logical and / or content relevance between the user's response content and the current topic content.

[0015] According to a topic shifting method provided by the present invention, when the topic interest level is lower than a preset interest level threshold, determining the topic content to be shifted based on the target user's sentiment and outputting the shifted topic content includes:

[0016] If the topic interest level is lower than a preset interest level threshold, and the interval between the conversation time corresponding to the target user's emotion and the current time exceeds a preset duration threshold, the topic content to be transferred is determined based on the target user's emotion, and the transferred topic content is output.

[0017] According to a topic shifting method provided by the present invention, the step of determining the topic content to be shifted based on the target user's sentiment includes:

[0018] Based on the temporal relationship between various user emotions, the adjacent user emotions of the target user emotion are determined from the various user emotions;

[0019] Based on the target user's sentiment and the sentiments of neighboring users, user sentiment analysis is performed, and the topic content to be shifted is determined based on the sentiment analysis results.

[0020] According to a topic shifting method provided by the present invention, after outputting the shifted topic content, the method further includes:

[0021] Determine the user's interest level in the shifted topic content, and the multiple shifted user emotions in the conversation process corresponding to the shifted topic content;

[0022] Based on the difference between the emotions of each transferred user and the emotions of the target user, the emotions of the target transferred user are determined from the emotions of each transferred user.

[0023] If the interest level of the transferred topic is lower than the preset interest level threshold, the secondary transferred topic content is determined based on the target user's emotions, and the secondary transferred topic content is output.

[0024] According to a topic shifting method provided by the present invention, the multiple user emotions are determined based on the following steps:

[0025] Acquire multiple user images during the session;

[0026] Identify the facial regions and / or limb movement regions in each user's image;

[0027] Perform facial expression recognition on each face region to obtain multiple facial expressions and emotions, and / or perform motion recognition on each body movement region to obtain multiple motion emotions;

[0028] Multiple user emotions are determined based on multiple facial expressions and / or multiple action emotions.

[0029] The present invention also provides a topic-shifting device, comprising:

[0030] A data determination unit is used for interaction data during the conversation process, as well as multiple user emotions. The interaction data includes at least one of the following: response duration, response richness, and response relevance.

[0031] The topic shifting unit is used to determine the user's level of interest in the current topic content based on the interaction data.

[0032] Based on the temporal relationship between various user emotions and the degree of difference between adjacent user emotions, the target user emotion is determined from the various user emotions.

[0033] If the interest level of the topic is lower than the preset interest level threshold, the topic content to be shifted is determined based on the target user's emotions, and the shifted topic content is output.

[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the topic shifting method as described above.

[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the topic shifting method as described above.

[0036] The topic-shifting method, apparatus, electronic device, and storage medium provided by this invention evaluate the user's interest in the current topic content from multiple perspectives through interactive data at different levels during the conversation. When the topic interest level is lower than a preset interest threshold, the method uses the target user's emotion with the most drastic emotional shift during the conversation as a basis for user emotion analysis. Based on the emotion analysis results, the method determines and outputs the topic content to be shifted. This overcomes the shortcomings of traditional solutions, such as a crude user interest evaluation process and abrupt topic shifts that result in significant differences between the content before and after the shift, leading to a poor user experience. This invention enables the introduction of other topic content based on the current topic content, ensuring the continuity and coherence of the conversation, as well as the relevance and appropriateness of the topic content, thus optimizing the user experience. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the topic shifting method provided by the present invention;

[0039] Figure 2 This is a schematic diagram of the process for determining the content of a shifted topic provided by the present invention;

[0040] Figure 3 This is a schematic diagram of the secondary topic shifting process provided by the present invention;

[0041] Figure 4 This is a schematic diagram of the process for determining user emotions provided by the present invention;

[0042] Figure 5 This is a schematic diagram of the topic-shifting device provided by the present invention;

[0043] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] This invention provides a topic shifting method that aims to assess a user's interest in the current topic from multiple perspectives using interaction data during a conversation. When it is determined that the user's interest in the current topic is weak, it uses the user's most drastic emotional shifts during the conversation as a basis for user sentiment analysis. Based on the sentiment analysis results, it determines the topic to shift to. This method allows for the introduction of other topics on top of the current one, ensuring the continuity and coherence of the conversation, as well as the relevance and appropriateness of the topic content, thus optimizing the user experience. Figure 1 This is a flowchart illustrating the topic shifting method provided by the present invention, as follows: Figure 1 As shown, the executing entity of this method can be a device capable of conversing with the user, such as a robot or virtual human, or it can be a server that directly controls the device. This embodiment of the invention does not specifically limit this. The method includes:

[0046] Step 110: Determine the interaction data during the conversation, as well as multiple user emotions. The interaction data includes at least one of the following: response duration, response richness, and response relevance.

[0047] Specifically, during the conversation between the user and the device, user images are continuously captured. These user images can be captured by an image acquisition device, which can be a camera, webcam, scanner, etc. It can be installed on the device or independent of the device. This embodiment of the invention does not make specific limitations in this regard.

[0048] After obtaining multiple user images, user emotion analysis can be performed to determine multiple user emotions during the conversation. That is, the user emotion can be determined based on the facial expressions and / or body language in the user images.

[0049] In addition, user emotions can also be determined based on the conversational voice between the user and the device during the conversation. This voice can be the user's response to the device's output content, or the user's response voice corresponding to the device's output content. That is, the user's tone of voice, intonation, semantic information, etc. can be analyzed based on the user's response voice to determine the user's emotions.

[0050] In addition to determining the emotions of multiple users, it is also necessary to acquire interaction data between users and devices during the conversation. This interaction data refers to the user's response to the device's output content (user response content), which can be any one or more of the following: response duration, response richness, and response relevance. Here, the device output content can be content related to the current topic, such as background information on the current topic, questions raised based on the current topic, and extended content on the current topic.

[0051] Among them, response duration is the interval between the output time of the device's output content and the collection time of the user's response content; the collection time here refers to the initial moment when the user's response content is collected, not the moment when the collection is completed; response richness is the richness of the user's response content to the device's output content, which can be measured from the length of the user's response content and / or the length of the collection time of the user's response content; response relevance is the relevance between the user's response content and the device's output content. Since the device's output content is related to the current topic content, response relevance can also be the relevance between the user's response content and the current topic content, which can be considered from the logical level, content level (semantic level), etc. of the user's response content and the current topic content.

[0052] Step 120: Based on the interaction data, determine the user's level of interest in the current topic content;

[0053] Specifically, after obtaining the interaction data during the conversation in step 110, step 120 can be executed to determine the user's interest in the current topic content using the interaction data. Specifically, since the interaction data contains data from multiple levels, it can be approached from different levels, and the user's interest in the current topic content at that level can be evaluated using the data at the corresponding level, thereby obtaining the user's interest in the topic at different levels.

[0054] Furthermore, data analysis can be conducted by combining user interest in the current topic content at different levels with data from different levels in the interaction data to assess the user's interest status from an overall perspective. This allows us to obtain the user's interest in the current topic content, thus ensuring the accuracy of the interest assessment process and avoiding significant deviations in subsequent topic shifts due to an overly rough assessment process. This achieves a refined and comprehensive interest assessment process.

[0055] In this embodiment of the invention, the user's interest status can be assessed by the response duration of the user's response content in the interaction data. That is, the user's interest level in the current topic content can be estimated based on the response duration; the shorter the response duration, the higher the user's interest level, and the longer the response duration, the lower the user's interest level. Alternatively, the user's interest level can be assessed by the richness of the response; that is, the richer the response, the higher the user's interest level, and the lower the response richness, the lower the user's interest level. Or, the user's interest level can be assessed by the strength of the response relevance; that is, the stronger the response relevance, the higher the user's interest level, and the weaker the response relevance, the lower the user's interest level.

[0056] You can also combine any two of the following factors—response duration, response richness, and response relevance—or combine all three simultaneously to determine the user's interest in the current topic. This allows you to assess the user's overall interest level and obtain the overall level of user interest in the current topic through comprehensive analysis.

[0057] Step 130: Based on the temporal relationship between various user emotions and the degree of difference between adjacent user emotions in each user emotion, determine the target user emotion from various user emotions.

[0058] Specifically, after step 120, which identifies multiple user emotions during the session, step 130 is executed. Based on the temporal relationship between the various user emotions and the degree of difference between adjacent user emotions, the target user emotion is determined from the various user emotions. The specific process may include the following steps:

[0059] First, it is necessary to determine the temporal relationship between the various user emotions. The temporal relationship here is actually the order in which the various user emotions appear during the conversation, or it can be understood as the order in which the user images corresponding to each user emotion are collected.

[0060] Subsequently, based on the temporal relationship between each user's emotions, a difference analysis can be performed on each user's emotions to determine the degree of difference between adjacent user emotions. Specifically, the user's emotions can be sorted according to the temporal relationship between each user's emotions to obtain a user emotion sequence. Then, a difference analysis can be performed on each pair of adjacent user emotions in the user emotion sequence to determine the degree of difference between each pair of adjacent user emotions.

[0061] Subsequently, the difference between adjacent user emotions in the user emotion sequence can be used as a reference to filter out the target user emotion from all user emotions. The target user emotion here is the user emotion with the most drastic emotional change among multiple user emotions. It can be reflected by the difference between adjacent user emotions. That is, the greater the difference between adjacent user emotions, the greater the user's emotional change, that is, the more drastic the emotional change; conversely, the smaller the difference between adjacent user emotions, the smaller the user's emotional change, that is, the more gradual the emotional change.

[0062] In this embodiment of the invention, the largest difference between two adjacent user emotions can be selected, and the corresponding user emotion can be determined as the target user emotion.

[0063] Step 140: If the topic interest level is lower than the preset interest level threshold, determine the topic content to be transferred based on the target user's emotions and output the transferred topic content.

[0064] Specifically, after obtaining the user's interest level in the current topic and the target user's emotions through steps 120 and 130, step 140 can be executed. Using the topic interest level and a preset interest threshold, it can be determined whether the topic transfer condition is met. Specifically, if the topic interest level is lower than the preset interest threshold, that is, if the user's interest level in the current topic content is low, in other words, if the user is not interested in the current topic content, then the topic transfer condition is met, and topic transfer can be carried out. Therefore, it is necessary to determine the topic content to be transferred and output it.

[0065] Here, the preset interest threshold is a pre-set value used to determine whether a user is interested in the current topic content. It is usually the minimum value that can be tolerated to indicate that the user is interested in the current topic content. It can be set according to actual needs, for example, it can be 20%, 30%, 50%, etc.

[0066] Considering that the content difference before and after the topic shift is too large in traditional solutions, the abrupt topic shift may cause users to not adapt in time, resulting in a poor user experience. In this embodiment of the invention, the target user's emotions are used as the basis to determine the content of the topic shift through analysis. That is, user emotion analysis is performed based on the target user's emotions, and the content of the topic shift is determined through the emotion analysis results.

[0067] In this embodiment of the invention, the target user's emotions are used as a link to connect the topic content before and after the transfer, which can ensure the continuity of the conversation and the relevance between the topic content. This solves the problem of poor user experience caused by the large difference between the content before and after the transfer in traditional solutions, and improves the user experience.

[0068] The topic shifting method provided by this invention evaluates the user's interest in the current topic content from multiple perspectives through interaction data at different levels during the conversation. When the topic interest level is lower than a preset interest threshold, it uses the target user's emotions, which show the most dramatic emotional shifts during the conversation, as a basis for user emotion analysis. Based on the emotion analysis results, it determines and outputs the shifted topic content. This overcomes the shortcomings of traditional solutions, such as a crude user interest evaluation process and abrupt topic shifts that result in significant differences between the content before and after the shift, leading to a poor user experience. This method allows for the introduction of other topic content based on the current topic content, ensuring the continuity and coherence of the conversation, as well as the relevance and appropriateness of the topic content, thus optimizing the user experience.

[0069] Based on the above embodiments, step 120 includes:

[0070] Based on at least one of the first interest level, second interest level, and third interest level, determine the user's topic interest level for the current topic content;

[0071] The first level of interest is determined based on the response duration of the user's response content and a first threshold; the second level of interest is determined based on the richness of the user's response content and a second threshold; and the third level of interest is determined based on the relevance of the user's response content and a third threshold.

[0072] Specifically, step 120, which involves determining the user's level of interest in the current topic based on interaction data during the conversation, may include the following steps:

[0073] First, we need to use the response duration, richness, and relevance of user responses in the interaction data to determine the first, second, and third levels of interest in the current topic. Specifically, we can determine the user's first level of interest in the current topic based on the response duration and a first threshold. That is, if the response duration is less than the first threshold, we can determine that the user responded quickly, thus indicating a high level of interest in the current topic. Conversely, if the response duration is greater than or equal to the first threshold, we can determine that the user responded slowly, thus indicating a low level of interest in the current topic. Here, the specific value of the first level of interest can be determined based on the difference between the response duration and the first threshold.

[0074] Alternatively, the user's second level of interest in the current topic can be determined based on the richness of the user's response content and a second threshold. That is, if the response richness is greater than the second threshold, it can be determined that the user's response content is rich, and therefore the user's second level of interest in the current topic is high. Conversely, if the response richness is less than or equal to the second threshold, it can be determined that the user's response content is monotonous and scarce, and therefore the user's second level of interest in the current topic is low. Here, the specific value of the second level of interest can be determined based on the difference between the response richness and the second threshold.

[0075] Furthermore, the user's third level of interest in the current topic can be determined based on the relevance of the user's response content and the third threshold. That is, if the relevance of the response is greater than the third threshold, it can be determined that the relevance between the user's response content and the current topic content is high, and therefore the user's third level of interest in the current topic content is high. Conversely, if the relevance of the response is less than or equal to the third threshold, it can be determined that the user's response content is monotonous and lacking, and therefore the user's third level of interest in the current topic content is low. Here, the specific value of the third level of interest can be determined based on the difference between the relevance of the response and the third threshold.

[0076] Here, the first threshold, the second threshold, and the third threshold are all preset and can be used as reference values ​​from the corresponding perspectives to assess the user's level of interest in the current topic content. The specific values ​​can be set according to actual needs. For example, the first threshold can be 8 seconds, 10 seconds, 15 seconds, etc., the second threshold can be 50%, 60%, 75%, etc., and the third threshold can be 60%, 70%, 75%, etc.

[0077] Subsequently, the user's interest level in the current topic can be determined based on any one or more of the first, second, and third interest levels. That is, any one of the first, second, and third interest levels can be directly used as the user's interest level in the current topic, or at least two of the first, second, and third interest levels can be combined, or all three can be combined simultaneously to determine the user's interest level in the current topic. In other words, the user's interest status can be evaluated from multiple levels, and a comprehensive analysis can be conducted by combining interest levels from different perspectives to obtain the overall user's interest level in the current topic.

[0078] Based on the above embodiments, the richness of the response is determined based on the length of the user's response content in characters and / or the duration of content collection.

[0079] The relevance of a response is determined based on the logical and / or content relevance between the user's response and the current topic content.

[0080] Specifically, the richness of user response content in the interaction data can include the character length of the user response content and / or the content collection time of the user response content. Here, the character length refers to the number of characters in the user response content, and the content collection time refers to the duration of collecting the user response content. For both voice and text, the collection time can be the input time of the response content, as well as the presentation time of the response content presented in voice form.

[0081] The longer the content character length, the richer the user response; conversely, the shorter the content character length, the lower the richness of the user response. Similarly, the longer the content collection time, the richer the user response; conversely, the shorter the content collection time, the lower the richness of the user response.

[0082] The relevance of user responses in the interaction data includes the logical relevance between the user response and the current topic content, and / or the content relevance between the user response and the current topic content. Logical relevance refers to the degree of logical connection between the user response and the current topic content, while content relevance indicates the degree of semantic connection between the user response and the current topic content.

[0083] The stronger the logical connection, the higher the relevance of the user's response; conversely, the weaker the logical connection, the weaker the relevance of the user's response. Similarly, the stronger the content connection, the higher the relevance of the user's response; conversely, the weaker the content connection, the weaker the relevance of the user's response.

[0084] In this embodiment of the invention, combining data from different levels to determine the richness and relevance of user responses ensures the accuracy and comprehensiveness of interaction data, thereby contributing to the improvement of the accuracy of subsequent interest assessment processes and laying the foundation for optimizing user experience.

[0085] Based on the above embodiments, step 140 includes:

[0086] If the topic interest level is lower than the preset interest level threshold, and the interval between the conversation time corresponding to the target user's emotion and the current time exceeds the preset duration threshold, the topic content to be shifted is determined based on the target user's emotion, and the shifted topic content is output.

[0087] Considering that the topic shifting process in the traditional solution is executed immediately, that is, the topic content is adjusted immediately when it is determined that the user's interest in the current topic content is not good, the instantaneous topic shift will cause the user to not be able to react in time, and the topic shift is too fast, which will make it impossible for the user to detach from the previous topic content in time, thus making it impossible to engage in the next topic content, resulting in a very poor user experience.

[0088] Therefore, this embodiment of the invention proposes a topic shifting process after a preset time period. Specifically, in step 140, when the topic interest level is lower than a preset interest threshold, the process of determining the shifted topic content based on the target user's emotion and outputting the shifted topic content can be further included: when the topic interest level is lower than the preset interest threshold, and the interval between the conversation time corresponding to the target user's emotion and the current time exceeds a preset duration threshold (i.e., the user is not interested in the current topic content, and the interval between this time and the conversation time where the user's emotion drastically changes exceeds the preset duration threshold), user emotion analysis is performed based on the target user's emotion, and the shifted topic content is determined and output based on the emotion analysis results. The preset duration threshold can be pre-set according to actual needs, for example, it could be 4 seconds, 5 seconds, etc.

[0089] Correspondingly, if the interval between the conversation time corresponding to the target user's emotion and the current time is less than or equal to a preset duration threshold, the topic will not be shifted until the interval between the current time and the conversation time corresponding to the target user's emotion exceeds the preset duration threshold. Then, the topic content to be shifted will be determined and output based on the target user's emotion.

[0090] Based on the above embodiments, Figure 2 This is a flowchart illustrating the process of determining the content of a shifted topic provided by the present invention, as shown below. Figure 2 As shown, in step 140, determining the topic content for shifting the focus based on the target user's sentiment includes:

[0091] Step 210: Based on the temporal relationship between various user emotions, determine the neighboring user emotions of the target user emotion from among the various user emotions;

[0092] Step 220: Based on the target user's sentiment and the sentiments of neighboring users, perform user sentiment analysis, and determine the topic content to be shifted based on the sentiment analysis results.

[0093] Specifically, step 140, which involves determining the topic content to be shifted based on the target user's emotions, may include the following steps:

[0094] Step 210: First, the temporal relationship between various user emotions can be used as a reference to determine the user emotions adjacent to the target user's emotions. That is, the user emotions before and after the target user's emotions can be selected from the user emotion sequence arranged with reference to the temporal relationship as the user emotions adjacent to the target user's emotions.

[0095] Step 220: Then, based on the target user's emotions and the emotions of neighboring users, user emotion analysis can be performed to obtain emotion analysis results. That is, based on the target user's emotions and the emotions of neighboring users, the user's emotion shift can be analyzed. On this basis, combined with the device output content and / or the current topic content, the reasons for the user's emotion shift and the key points of the user's emotion shift in terms of time and content can be estimated to obtain emotion analysis results.

[0096] Furthermore, based on the user's sentiment analysis results, the topic can be shifted accordingly. This means asking questions about the reasons for the user's emotional shift and the key points of that shift, or avoiding the user's "sensitive points" by asking questions that are unrelated to the reasons and / or key points of the emotional shift but are relevant to the current topic. This can, to some extent, prevent secondary topic shifts during the conversation, maximizing the continuity and coherence of the conversation, as well as the relevance and appropriateness of the topic content.

[0097] Based on the above embodiments, Figure 3 This is a schematic diagram of the secondary topic shifting process provided by the present invention, as shown below. Figure 3 As shown, in step 140, the topic shift content is output, followed by:

[0098] Step 310: Determine the user's interest in the topic being moved, and the user's emotions during the conversation corresponding to the topic being moved.

[0099] Step 320: Based on the difference between the emotions of each transferred user and the emotions of the target user, determine the emotions of the target transferred user from the emotions of each transferred user;

[0100] Step 330: If the interest level of the transferred topic is lower than the preset interest level threshold, determine the secondary transferred topic content based on the target user's emotions and output the secondary transferred topic content.

[0101] Specifically, in step 140, after determining and outputting the topic to be transferred based on the target user's emotions, if the topic transfer conditions are met again, a second transfer can be performed. The specific process includes the following steps:

[0102] Step 310: First, it is necessary to determine the user's interest in the topic content after the initial topic shift during the conversation, i.e., the interest in the shifted topic. At the same time, it is also necessary to determine the user's multiple emotions during the conversation after the initial topic shift, i.e., the multiple user emotions corresponding to the shifted topic content during the conversation. Here, the process of determining the interest in the shifted topic and the user emotions is basically the same as the process of determining the interest in the topic and the user emotions of the initial topic shift described above, and will not be repeated here.

[0103] Step 320: Then, based on the difference between the emotions of each transferred user and the emotions of the target user, the target transferred user emotion can be determined from the emotions of each transferred user. Specifically, based on the emotions of the target user, a difference analysis is performed on the emotions of each transferred user to determine the difference between the emotions of each transferred user and the emotions of the target user. Using this difference as a reference, the target transferred user emotion is selected from the emotions of each user. Here, the target transferred user emotion is the transferred user emotion with the greatest difference from the emotions of the target user among the multiple transferred user emotions.

[0104] Step 330: Afterwards, the topic transfer interest level and preset interest level threshold can be used to determine whether the topic transfer conditions are met. If the topic transfer conditions are met, the topic transfer can be performed again. Specifically, if the topic transfer interest level is lower than the preset interest level, that is, if the user has a low interest in the topic transfer content, in other words, if the user is not interested in the topic transfer content, the secondary topic transfer content is determined based on the target user's emotions, and the secondary topic transfer content is output. That is, user emotion analysis is performed based on the target user's emotions, and the secondary topic transfer content is determined and output based on the emotion analysis results.

[0105] Here, by using the target shift of user emotions as a link, connecting the initial shift topic content and the secondary shift topic content, the continuity of the conversation and the relevance between topic content can be guaranteed. This overcomes the shortcomings of traditional solutions, such as the relatively crude user interest assessment process and the abrupt topic shifts, which result in large differences in content before and after the shift and a poor user experience. It achieves the introduction of secondary shift topic content on the basis of the initial shift topic content, ensuring the continuity of the conversation and the relevance between topic content, and optimizing the user experience.

[0106] Based on the above embodiments, Figure 4 This is a schematic diagram illustrating the process of determining user emotions provided by the present invention, as shown below. Figure 4 As shown, multiple user emotions are determined based on the following steps:

[0107] Step 410: Obtain multiple user images during the session;

[0108] Step 420: Determine the face region and / or body movement region in each user image;

[0109] Step 430: Perform facial expression recognition on each face region to obtain multiple facial expressions and emotions, and / or perform action recognition on each body movement region to obtain multiple action emotions.

[0110] Step 440: Determine multiple user emotions based on multiple facial expressions and / or multiple action emotions.

[0111] Specifically, the process of determining the emotions of multiple users during a conversation includes the following steps:

[0112] Step 410: First, it is necessary to acquire multiple user images during the session. These user images can be user images acquired by the image acquisition device at a certain frame rate during the session. While acquiring multiple user images, it is also necessary to determine the temporal relationship between the multiple user images. This temporal relationship is the order in which each user image appears during the session, i.e., the acquisition order of each user image.

[0113] Step 420: Since facial expressions and body movements in user images can represent user emotions, after obtaining multiple user images, the user's face region and / or body movement region can be determined from the multiple user images. That is, each user image can be divided into regions to determine the user's face region and / or body movement region.

[0114] Step 430: Subsequently, facial expression recognition can be performed on each facial region to identify the emotions corresponding to the user's facial expressions, thereby obtaining multiple emotional expressions of the user; for example, when the user's facial expression corresponding to the facial region is a hearty laugh, it can be determined that the user's emotional expression is happiness; correspondingly, when the user's facial expression is a frown and / or a downturned mouth, it can be determined that the user's emotional expression is anger, sadness, dissatisfaction, etc.

[0115] It is also possible to perform motion recognition on each body movement area to identify the user's body movements and the emotions they represent, and finally obtain the user's emotional state. For example, when the body movement area of ​​the user corresponds to clapping, it can be determined that the user's emotional state is happy; correspondingly, when the body movement is stomping, it can be determined that the user's emotional state is dissatisfied.

[0116] It can also perform facial expression recognition on various facial areas of the user and motion recognition on various body movement areas, thereby obtaining multiple facial expressions and multiple motion emotions of the user.

[0117] Step 440: After this step, multiple user emotions can be determined based on the user's multiple facial expressions and / or multiple action emotions. That is, facial expressions can be directly used as user emotions, or action emotions can be used as user emotions, or each user emotion can be determined by combining the corresponding action emotions with each facial expression. This embodiment of the invention does not make specific limitations in this regard.

[0118] Based on the above embodiments, the overall process of the topic shifting method provided by the present invention includes the following steps:

[0119] First, the interaction data during the conversation and multiple user emotions are determined. The interaction data includes at least one of the following: response duration, response richness, and response relevance of user responses. The multiple user emotions are determined based on the following steps: acquiring multiple user images during the conversation; identifying facial regions and / or body movement regions in each user image; performing facial expression recognition on each facial region to obtain multiple facial expression emotions, and / or performing action recognition on each body movement region to obtain multiple action emotions; and determining multiple user emotions based on the multiple facial expression emotions and / or multiple action emotions.

[0120] Subsequently, based on the interaction data, the user's interest level in the current topic content is determined. Specifically, the user's interest level in the current topic content can be determined based on at least one of the first interest level, the second interest level, and the third interest level. The first interest level is determined based on the response duration of the user's response content and a first threshold. The second interest level is determined based on the richness of the user's response content and a second threshold. The third interest level is determined based on the relevance of the user's response content and a third threshold.

[0121] Among them, response richness is determined based on the character length of the user's response content and / or the content collection time; response relevance is determined based on the logical and / or content relevance between the user's response content and the current topic content.

[0122] Subsequently, based on the temporal relationship between various user emotions and the degree of difference between adjacent user emotions in each user emotion, the target user emotion is determined from each user emotion.

[0123] Subsequently, if the topic interest level is lower than the preset interest level threshold, the topic content to be shifted is determined based on the target user's emotions, and the shifted topic content is output.

[0124] Furthermore, when the topic interest level is lower than a preset interest level threshold, the process of determining and outputting the topic content based on the target user's emotions can be as follows: when the topic interest level is lower than a preset interest level threshold, and the interval between the conversation time corresponding to the target user's emotions and the current time exceeds a preset duration threshold, the process of determining and outputting the topic content based on the target user's emotions can be as follows.

[0125] The process of determining the topic content to be shifted based on the target user's emotions includes the following steps: determining the neighboring user emotions of the target user's emotions based on the temporal relationship between the emotions of various users; conducting user emotion analysis based on the target user's emotions and the neighboring user emotions of the target user's emotions; and determining the topic content to be shifted based on the emotion analysis results obtained from the user emotion analysis.

[0126] After outputting the topic shift content, it is also possible to determine the user's interest level in the topic shift content, as well as multiple user emotions during the conversation corresponding to the topic shift content; based on the difference between each user emotion and the target user emotion, the target user emotion is determined from the various user emotions; if the interest level in the topic shift is lower than a preset interest threshold, a secondary topic shift content is determined based on the target user emotion, and the secondary topic shift content is output.

[0127] The method provided in this invention evaluates a user's interest in the current topic from multiple perspectives using interaction data at different levels during a conversation. If the interest level is below a preset threshold, it uses the target user's emotions, which show the most dramatic shifts during the conversation, as a basis for user emotion analysis. Based on the emotion analysis results, it determines and outputs the new topic content. This overcomes the shortcomings of traditional solutions, such as a crude user interest assessment process and abrupt topic transitions leading to significant differences in content before and after the transition, resulting in a poor user experience. The method allows for the introduction of other topics based on the current topic content, ensuring the continuity and coherence of the conversation, as well as the relevance and appropriateness of the topic content, thus optimizing the user experience.

[0128] The topic shifting device provided by the present invention will be described below. The topic shifting device described below can be referred to in correspondence with the topic shifting method described above.

[0129] Figure 5 This is a schematic diagram of the topic-shifting device provided by the present invention, as shown below. Figure 5 As shown, the device includes:

[0130] The data determination unit 510 is used for interaction data during the conversation and multiple user emotions. The interaction data includes at least one of the following: response duration, response richness, and response relevance.

[0131] The topic transfer unit 520 is used to determine the user's topic interest level for the current topic content based on the interaction data.

[0132] Based on the temporal relationship between various user emotions and the degree of difference between adjacent user emotions, the target user emotion is determined from the various user emotions.

[0133] If the interest level of the topic is lower than the preset interest level threshold, the topic content to be shifted is determined based on the target user's emotions, and the shifted topic content is output.

[0134] The topic-switching device provided by this invention evaluates the user's interest in the current topic from multiple perspectives through interaction data at different levels during the conversation. When the topic interest is lower than a preset interest threshold, it uses the target user's emotion with the most dramatic emotional shift during the conversation as a basis for user emotion analysis. Based on the emotion analysis results, it determines and outputs the topic to be switched. This overcomes the shortcomings of traditional solutions, such as the relatively crude user interest evaluation process and the abrupt topic switching, which leads to significant differences in content before and after the switch and a poor user experience. It enables the introduction of other topic content based on the current topic content, ensuring the continuity and coherence of the conversation, as well as the relevance and relevance of the topic content, thus optimizing the user experience.

[0135] Based on the above embodiments, the topic transfer unit 520 is used for:

[0136] The user's topic interest in the current topic content is determined based on at least one of the first interest level, the second interest level, and the third interest level.

[0137] The first interest level is determined based on the response duration of the user's response content and a first threshold; the second interest level is determined based on the response richness of the user's response content and a second threshold; and the third interest level is determined based on the response relevance of the user's response content and a third threshold.

[0138] Based on the above embodiments, the richness of the response is determined based on the length of the user's response content and / or the duration of content collection;

[0139] The response relevance is determined based on the logical and / or content relevance between the user's response content and the current topic content.

[0140] Based on the above embodiments, the topic transfer unit 520 is used for:

[0141] If the topic interest level is lower than a preset interest level threshold, and the interval between the conversation time corresponding to the target user's emotion and the current time exceeds a preset duration threshold, the topic content to be transferred is determined based on the target user's emotion, and the transferred topic content is output.

[0142] Based on the above embodiments, the topic transfer unit 520 is used for:

[0143] Based on the temporal relationship between various user emotions, the adjacent user emotions of the target user emotion are determined from the various user emotions;

[0144] Based on the target user's sentiment and the sentiments of neighboring users, user sentiment analysis is performed, and the topic content to be shifted is determined based on the sentiment analysis results.

[0145] Based on the above embodiments, the topic transfer unit 520 is also used for:

[0146] Determine the user's interest level in the shifted topic content, and the multiple shifted user emotions in the conversation process corresponding to the shifted topic content;

[0147] Based on the difference between the emotions of each transferred user and the emotions of the target user, the emotions of the target transferred user are determined from the emotions of each transferred user.

[0148] If the interest level of the transferred topic is lower than the preset interest level threshold, the secondary transferred topic content is determined based on the target user's emotions, and the secondary transferred topic content is output.

[0149] Based on the above embodiments, the data determination unit 510 is used for:

[0150] Acquire multiple user images during the session;

[0151] Identify the facial regions and / or limb movement regions in each user's image;

[0152] Perform facial expression recognition on each face region to obtain multiple facial expressions and emotions, and / or perform motion recognition on each body movement region to obtain multiple motion emotions;

[0153] Multiple user emotions are determined based on multiple facial expressions and / or multiple action emotions.

[0154] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a topic shifting method. This method includes: determining interaction data during the conversation and multiple user emotions, wherein the interaction data includes at least one of the following: response duration, response richness, and response relevance; determining the user's topic interest in the current topic content based on the interaction data; determining a target user emotion from the user emotions based on the temporal relationship between the various user emotions and the difference between adjacent user emotions; and, if the topic interest is lower than a preset interest threshold, determining and outputting the shifted topic content based on the target user emotion when the shifted topic interest is lower than a preset interest threshold.

[0155] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the topic transfer method provided by the above methods, the method comprising: determining interaction data during a conversation and multiple user emotions, the interaction data including at least one of response duration, response richness, and response relevance of user response content; determining the user's topic interest in the current topic content based on the interaction data; determining a target user emotion from the user emotions based on the temporal relationship between the user emotions and the difference between adjacent user emotions; and, if the topic interest is lower than a preset interest threshold, determining the topic content to be transferred based on the target user emotion and outputting the transferred topic content.

[0157] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the topic-shifting method provided by the above methods. The method includes: determining interaction data during a conversation and multiple user emotions, wherein the interaction data includes at least one of the following: response duration, response richness, and response relevance of user responses; determining the user's topic interest in the current topic content based on the interaction data; determining a target user emotion from the user emotions based on the temporal relationship between the user emotions and the difference between adjacent user emotions; and, if the topic interest is lower than a preset interest threshold, determining the topic content to be shifted based on the target user emotion and outputting the shifted topic content.

[0158] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A topic shift method characterized by, The method comprises: determining interaction data in a conversation process and a plurality of user emotions, the interaction data comprising at least one of response duration, response richness, and response relevance of user response content; determining a topic interest degree of a user for current topic content based on the interaction data; determining a target user emotion from the plurality of user emotions based on a time sequence relationship between the plurality of user emotions and a difference degree between adjacent user emotions in the plurality of user emotions; in a case where the topic interest degree is lower than a preset interest degree threshold, determining a transfer topic content based on the target user emotion and outputting the transfer topic content; the determination of the transfer topic content based on the target user emotion comprises: determining adjacent user emotions of the target user emotion from the plurality of user emotions based on the time sequence relationship between the plurality of user emotions; performing user emotion analysis based on the target user emotion and the adjacent user emotions of the target user emotion, and determining the transfer topic content based on an emotion analysis result obtained by the user emotion analysis.

2. The topic shift method according to claim 1, characterized by, the determination of the topic interest degree of the user for the current topic content based on the interaction data comprises: determining the topic interest degree of the user for the current topic content based on at least one of a first interest degree, a second interest degree, and a third interest degree; the first interest degree is determined based on the response duration of the user response content and a first threshold, the second interest degree is determined based on the response richness of the user response content and a second threshold, and the third interest degree is determined based on the response relevance of the user response content and a third threshold.

3. The topic transfer method according to claim 2, wherein: the response richness is determined based on a content character length and / or a content collection duration of the user response content; the response relevance is determined based on a logical relevance and / or a content relevance between the user response content and the current topic content.

4. The topic shift method according to any one of claims 1 to 3, characterized by, the determination of the transfer topic content based on the target user emotion in the case where the topic interest degree is lower than the preset interest degree threshold, and the output of the transfer topic content comprises: in a case where the topic interest degree is lower than the preset interest degree threshold, and an interval duration between a conversation time corresponding to the target user emotion and a current time exceeds a preset duration threshold, the transfer topic content is determined based on the target user emotion, and the transfer topic content is output.

5. The topic shift method according to any one of claims 1 to 3, characterized by, the output of the transfer topic content further comprises: determining a transfer topic interest degree of the user for the transfer topic content, and a plurality of transfer user emotions in a conversation process corresponding to the transfer topic content; determining a target transfer user emotion from the plurality of transfer user emotions based on a difference degree between each transfer user emotion and the target user emotion; in a case where the transfer topic interest degree is lower than the preset interest degree threshold, determining a secondary transfer topic content based on the target transfer user emotion and outputting the secondary transfer topic content.

6. The topic shift method according to any one of claims 1 to 3, characterized by, the plurality of user emotions are determined based on the following steps: obtaining a plurality of user images in a conversation process; determining a face region and / or a body movement region in each user image; perform facial expression recognition on each face region to obtain a plurality of expression emotions, and / or perform action recognition on each body action region to obtain a plurality of action emotions; determine a plurality of user emotions based on the plurality of expression emotions and / or the plurality of action emotions.

7. A topic shift apparatus characterized by comprising: comprise: a data determination unit configured to determine, during a conversation process, interaction data and a plurality of user emotions, the interaction data comprising at least one of a response duration, a response richness, and a response relevance of a user response content; a topic shift unit configured to determine, based on the interaction data, a topic interest degree of a user for a current topic content; determine, based on a time sequence relationship between the plurality of user emotions and a difference degree between adjacent user emotions in the plurality of user emotions, a target user emotion from the plurality of user emotions; in a case where the topic interest degree is lower than a preset interest degree threshold, determine a shift topic content based on the target user emotion and output the shift topic content; the determination of the shift topic content based on the target user emotion comprises: determine, based on a time sequence relationship between the plurality of user emotions, an adjacent user emotion of the target user emotion from the plurality of user emotions; perform user emotion analysis based on the target user emotion and the adjacent user emotion of the target user emotion, and determine the shift topic content based on an emotion analysis result obtained by the user emotion analysis.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, the processor implements the topic shift method according to any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, the computer program implements the topic shift method according to any one of claims 1 to 6 when executed by the processor.

Images