A human-computer interaction control method and related equipment

Abstract

The present application discloses a human-computer interaction control method and related equipment, the method comprising: after obtaining human-computer interaction state data in the current cycle, first determining the human-computer intimacy in the current cycle based on the human-computer interaction state data in the current cycle; then updating a reinforcement learning model based on the human-computer intimacy in the previous cycle and the human-computer intimacy in the current cycle, so that the updated reinforcement learning model can better make device control parameter decisions; finally, inputting the human-computer intimacy in the previous cycle, the device control parameters in the current cycle, and the human-computer intimacy in the current cycle into the updated reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model, so that the human-computer interaction device can be controlled in the next cycle using the device control parameters in the next cycle, which is conducive to improving user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a human-computer interaction control method and related equipment. Background Art

[0002] With the continuous development of artificial intelligence, human-computer interaction devices with different functions are gradually entering people's lives.

[0003] However, since the control process of a human-computer interaction device is usually invariable, the interaction mode between the human-computer interaction device and the user is fixed, which results in a poor user experience. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a human-computer interaction control method and related equipment, which can enable the control process of the human-computer interaction device to be adaptively adjusted as the human-computer intimacy changes, which is conducive to improving the user experience.

[0005] The present application provides a human-computer interaction control method, the method comprising:

[0006] Determining the human-machine intimacy within the current cycle according to the human-machine interaction state data within the current cycle;

[0007] updating the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle; wherein the human-machine intimacy in the previous cycle is determined according to the human-machine interaction state data in the previous cycle;

[0008] The human-machine intimacy in the previous cycle, the device control parameters in the current cycle, and the human-machine intimacy in the current cycle are input into the reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model.

[0009] In a possible implementation, if the current period includes N days, the process of determining the human-machine intimacy within the current period includes:

[0010] Determine the human-machine intimacy on the nth day in the current cycle according to the human-machine interaction state data on the nth day in the current cycle; wherein n is a positive integer, n≤N, and N is a positive integer;

[0011] The set of the human-machine intimacy from the first day in the current cycle to the human-machine intimacy on the Nth day in the current cycle is determined as the human-machine intimacy in the current cycle.

[0012] In a possible implementation, if the human-computer interaction state data includes interactive audio and interactive images, the process of determining the human-computer intimacy on the nth day in the current cycle includes:

[0013] Determining the interactive content within the nth day in the current cycle according to the interactive audio within the nth day in the current cycle;

[0014] Determining a content sentiment score for the nth day in the current cycle according to the interactive content within the nth day in the current cycle;

[0015] Determine the audio emotion score of the nth day in the current cycle according to the interactive audio on the nth day in the current cycle;

[0016] Determine, according to the interactive images within the nth day in the current cycle, the image emotion score of the nth day in the current cycle;

[0017] The human-machine intimacy on the nth day in the current cycle is determined according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

[0018] In a possible implementation, determining the human-machine intimacy on the nth day in the current cycle according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle includes:

[0019] The content emotion score of the nth day in the current cycle, the content emotion score in the historical time period corresponding to the nth day in the current cycle, the audio emotion score of the nth day in the current cycle, the audio emotion score in the historical time period corresponding to the nth day in the current cycle, the image emotion score of the nth day in the current cycle, and the image emotion score in the historical time period corresponding to the nth day in the current cycle are spliced ​​to obtain the emotion score splicing data of the nth day in the current cycle;

[0020] The emotion score splicing data of the nth day in the current cycle is input into a pre-constructed intimacy determination model to obtain the human-computer intimacy of the nth day in the current cycle output by the intimacy determination model; wherein the intimacy determination model is constructed based on the sample score splicing data and the actual human-computer intimacy corresponding to the sample score splicing data.

[0021] In a possible implementation, if the human-computer interaction state data also includes the number of active user interactions, the number of active device interactions, and the interaction duration, then determining the human-computer intimacy on the nth day in the current cycle according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle includes:

[0022] The human-computer intimacy on the nth day in the current cycle is determined based on the number of active user interactions on the nth day in the current cycle, the number of active device interactions on the nth day in the current cycle, the interaction duration on the nth day in the current cycle, the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

[0023] In a possible implementation, if there are M human-computer interaction processes within the nth day in the current cycle, the process of determining the content emotion score of the nth day in the current cycle includes:

[0024] Input the interactive content of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed content emotion scoring model, and obtain the content emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the content emotion scoring model; wherein the content emotion scoring model is constructed based on sample content and the actual emotion score of the sample content; m is a positive integer, m≤M, and M is a positive integer; average the content emotion scores of the first human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle, and obtain the content emotion score of the nth day in the current cycle;

[0025] The process of determining the audio sentiment score of the nth day in the current cycle includes:

[0026] Input the interactive audio of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed audio emotion scoring model, and obtain the audio emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the audio emotion scoring model; wherein the audio emotion scoring model is constructed based on sample audio and the actual emotion score of the sample audio; m is a positive integer, m≤M, and M is a positive integer; average the audio emotion scores of the 1st human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle, and obtain the audio emotion score of the nth day in the current cycle;

[0027] The process of determining the image sentiment score on the nth day in the current cycle includes:

[0028] The interactive image of the mth human-computer interaction process within the nth day in the current cycle is input into a pre-constructed image emotion scoring model to obtain the image emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the image emotion scoring model; wherein the image emotion scoring model is constructed based on sample images and actual emotion scores of the sample images; m is a positive integer, m≤M, and M is a positive integer; the image emotion scores of the first human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle are averaged to obtain the image emotion score of the nth day in the current cycle.

[0029] In a possible implementation, updating the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle includes:

[0030] Input the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle into a pre-constructed intimacy change analysis model, and obtain the intimacy change data corresponding to the current cycle output by the intimacy change analysis model; wherein the intimacy change analysis model is constructed based on the human-machine intimacy in a sample time period and the actual intimacy change data corresponding to the sample time period;

[0031] The reinforcement learning model is updated according to the intimacy change data corresponding to the current period.

[0032] The embodiment of the present application also provides a human-computer interaction control device, comprising:

[0033] An intimacy determination unit, configured to determine the human-computer intimacy in the current cycle according to the human-computer interaction state data in the current cycle;

[0034] A model updating unit, used to update the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle; wherein the human-machine intimacy in the previous cycle is determined according to the human-machine interaction state data in the previous cycle;

[0035] A parameter decision unit is used to input the human-machine intimacy in the previous cycle, the device control parameters in the current cycle and the human-machine intimacy in the current cycle into the reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model.

[0036] The present application also provides a device, the device comprising a processor and a memory:

[0037] The memory is used to store computer programs;

[0038] The processor is used to execute any implementation of the human-computer interaction control method provided in the embodiment of the present application according to the computer program.

[0039] The embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute any implementation of the human-computer interaction control method provided in the embodiment of the present application.

[0040] The embodiment of the present application also provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any implementation of the human-computer interaction control method provided in the embodiment of the present application.

[0041] Based on the above technical solution, this application has the following beneficial effects:

[0042] In the human-computer interaction control method provided by the present application, after obtaining the human-computer interaction status data in the current cycle, the human-computer intimacy in the current cycle is first determined based on the human-computer interaction status data in the current cycle; then, based on the human-computer intimacy in the previous cycle and the human-computer intimacy in the current cycle, the reinforcement learning model is updated so that the updated reinforcement learning model can better make device control parameter decisions; finally, the human-computer intimacy in the previous cycle, the device control parameters in the current cycle, and the human-computer intimacy in the current cycle are input into the updated reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model, so that the human-computer interaction device can be controlled in the next cycle using the device control parameters in the next cycle.

[0043] It can be seen that because the reinforcement learning model can learn in real time how to better make device control parameter decisions from the human-computer interaction process that has been experienced based on human-computer intimacy, the device control parameters in the next cycle determined by the reinforcement learning model based on human-computer intimacy are more suitable for the human-computer interaction process in the next cycle. In this way, the control process of the human-computer interaction device can be adaptively adjusted as the human-computer intimacy changes, which is beneficial to improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A flowchart of a human-computer interaction control method provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of the structure of a content sentiment scoring model provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of the structure of an audio emotion scoring model provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of an image emotion scoring model provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of sentiment score splicing data for the nth day in the current cycle provided by an embodiment of the present application;

[0050] Figure 6 A schematic diagram of the structure of an intimacy determination model provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of state splicing data of the nth day in the current cycle provided by an embodiment of the present application;

[0052] Figure 8 A schematic diagram of the structure of an intimacy generation model provided in an embodiment of the present application;

[0053] Fig. 9 A schematic diagram of the structure of an intimacy change analysis model provided in an embodiment of the present application;

[0054] Fig.10 A schematic diagram of the structure of a human-computer interaction control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] Method Embodiment

[0057] See also Figure 1 , which is a flow chart of a human-computer interaction control method provided in an embodiment of the present application.

[0058] The human-computer interaction control method provided in the embodiment of the present application includes S1-S3:

[0059] S1: Determine the human-computer intimacy in the current cycle based on the human-computer interaction status data in the current cycle.

[0060] In fact, the process of adjusting the device control parameters of the human-computer interaction device is implemented in a cycle; and the embodiment of the present application does not limit the cycle. For example, the device control parameters of the human-computer interaction device can be adjusted in a cycle of 10 days.

[0061] It can be seen that the "current cycle" in S1 can be used to represent the time period required for the adjustment process of the device control parameters of the current round of human-computer interaction devices. In addition, the embodiment of the present application does not limit the relationship between the "current cycle" in S1, the "previous cycle" below, and the "next cycle". For example, if the "current cycle" in S1 represents the time period [January 11, 2021, January 20, 2021], then the "previous cycle" below can represent the time period [January 1, 2021, January 10, 2021], and the "next cycle" below can represent the time period [January 21, 2021, January 30, 2021].

[0062] The human-computer interaction state data is used to describe the human-computer interaction process between the human-computer interaction device and the user of the human-computer interaction device. In addition, the embodiments of the present application do not limit the human-computer interaction state data. For example, the human-computer interaction state data may include at least one of the interactive audio, interactive image, number of active user interactions, number of active device interactions, and interaction duration.

[0063] Among them, interactive audio refers to audio data recorded by the human-computer interaction device during the human-computer interaction process. Interactive images refer to image data recorded by the human-computer interaction device during the human-computer interaction process (such as photographs or recorded videos, etc.). The number of user active interactions refers to the number of times the user of the human-computer interaction device actively initiates the human-computer interaction process. The number of device active interactions refers to the number of times the human-computer interaction process is actively initiated by the human-computer interaction device. The interaction duration refers to the length of time the human-computer interaction process is carried out between the human-computer interaction device and the user of the human-computer interaction device.

[0064] The human-computer intimacy in the current cycle is used to describe the degree of intimacy between the human-computer interaction device and the user of the human-computer interaction device during the human-computer interaction process in the current cycle, so that the human-computer intimacy can be used to characterize the degree of liking of the user of the human-computer interaction device for the human-computer interaction device in the current cycle.

[0065] In addition, the embodiment of the present application does not limit the determination process of the human-machine intimacy in the current period (that is, the implementation method of S1). For example, in a possible implementation method, if the current period includes N days, S1 may specifically include S11-S12:

[0066] S11: Determine the human-machine intimacy on the nth day in the current cycle according to the human-machine interaction state data on the nth day in the current cycle, where n is a positive integer, n≤N, and N is a positive integer.

[0067] Among them, the human-computer intimacy on the nth day in the current cycle is used to describe the degree of intimacy between the human-computer interaction device and the user of the human-computer interaction device during the human-computer interaction process on the nth day in the current cycle, so that the human-computer intimacy can be used to characterize the degree of liking of the user of the human-computer interaction device for the human-computer interaction device on the nth day in the current cycle.

[0068] In addition, the embodiment of the present application does not limit the implementation of S11. For example, in a possible implementation, if the human-computer interaction state data includes interactive audio and interactive images, S11 may specifically include S111-S115:

[0069] S111: Determine the interactive content within the nth day in the current cycle according to the interactive audio within the nth day in the current cycle.

[0070] In an embodiment of the present application, after obtaining the interactive audio within the nth day in the current cycle, the interactive audio within the nth day in the current cycle can be subjected to voice recognition processing to obtain the interactive content within the nth day in the current cycle, so that the interactive content within the nth day in the current cycle is used to represent the voice information carried by the interactive audio within the nth day in the current cycle.

[0071] It should be noted that the embodiments of the present application are not limited to the above-mentioned "speech recognition processing". For example, it can be implemented using a pre-built speech recognition model.

[0072] S112: Determine the content sentiment score of the nth day in the current cycle according to the interactive content within the nth day in the current cycle.

[0073] The content sentiment score of the nth day in the current cycle is used to represent the user sentiment carried by the interactive content within the nth day in the current cycle.

[0074] In addition, the embodiment of the present application does not limit the implementation of S112. For example, if there are M human-computer interaction processes within the nth day in the current cycle, S112 may specifically include S1121-S1122:

[0075] S1121: Input the interaction content of the mth human-computer interaction process within the nth day in the current cycle into a pre-built content sentiment scoring model, and obtain the content sentiment score of the mth human-computer interaction process within the nth day in the current cycle output by the content sentiment scoring model, where m is a positive integer, m≤M, and M is a positive integer.

[0076] The content sentiment scoring model is used to perform sentiment scoring processing on the input data of the content sentiment scoring model; moreover, the embodiment of the present application is not limited to the content sentiment scoring model, and can be any existing or future model that can perform sentiment scoring processing on text content (such as Figure 2 The neural network model 200 shown is implemented.

[0077] In addition, the content sentiment scoring model can be constructed based on the sample content and the actual sentiment score of the sample content. The sample content can be any text content; and the embodiment of the present application does not limit the method for obtaining the actual sentiment score of the sample content (for example, it can be implemented by manual annotation).

[0078] In addition, the embodiments of the present application do not limit the construction process of the content sentiment scoring model, and can be implemented using any existing or future model construction method.

[0079] The content sentiment score of the mth human-computer interaction process within the nth day in the current cycle is used to characterize the user emotions carried by the interaction content of the mth human-computer interaction process within the nth day in the current cycle.

[0080] Based on the relevant content of S1121 above, it can be known that after obtaining the interactive content of the mth human-computer interaction process within the nth day in the current cycle, the interactive content can be input into a pre-built content emotion scoring model, so that the content emotion scoring model performs emotion scoring processing on the interactive content, and obtains and outputs the content emotion score of the mth human-computer interaction process within the nth day in the current cycle, so that the content emotion score can accurately represent the user emotion carried by the interactive content of the mth human-computer interaction process within the nth day in the current cycle. Wherein, m is a positive integer, m≤M, and M is a positive integer.

[0081] S1122: Averaging the content emotion score of the first human-computer interaction process within the nth day in the current cycle to the content emotion score of the Mth human-computer interaction process within the nth day in the current cycle to obtain the content emotion score of the nth day in the current cycle.

[0082] In an embodiment of the present application, after obtaining the content emotion scores of M human-computer interaction processes within the nth day in the current cycle, the average value of the content emotion scores of the M human-computer interaction processes can be determined as the content emotion score of the nth day in the current cycle, so that the content emotion score of the nth day in the current cycle can accurately represent the user emotions carried by the interactive content within the nth day in the current cycle.

[0083] Based on the relevant content of S112 above, it can be known that after obtaining the interactive content within the nth day in the current cycle, the content emotion score of the nth day in the current cycle can be determined based on the interactive content, so that the content emotion score can accurately represent the user emotions carried by the interactive content within the nth day in the current cycle, so that the content emotion score can be referred to in the subsequent determination of the human-computer intimacy on the nth day in the current cycle.

[0084] S113: Determine the audio emotion score of the nth day in the current cycle according to the interactive audio within the nth day in the current cycle.

[0085] The audio sentiment score of the nth day in the current cycle is used to represent the user sentiment carried by the interactive audio within the nth day in the current cycle.

[0086] In addition, the embodiment of the present application does not limit the implementation of S113. For example, if there are M human-computer interaction processes within the nth day in the current cycle, S113 may specifically include S1131-S1132:

[0087] S1131: Input the interactive audio of the mth human-computer interaction process within the nth day in the current cycle into a pre-built audio emotion scoring model, and obtain the audio emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the audio emotion scoring model. Wherein, m is a positive integer, m≤M, and M is a positive integer.

[0088] The audio emotion scoring model is used to perform emotion scoring processing on the input data of the audio emotion scoring model; moreover, the embodiment of the present application is not limited to the audio emotion scoring model, and can be any existing or future model that can perform emotion scoring processing on audio data (such as Figure 3 The neural network model 300 shown is implemented.

[0089] In addition, the audio emotion scoring model can be constructed based on the sample audio and the actual emotion score of the sample audio. The sample audio can be any type of audio data; and the embodiment of the present application does not limit the method for obtaining the actual emotion score of the sample audio (for example, it can be implemented by manual annotation).

[0090] In addition, the embodiments of the present application do not limit the construction process of the audio emotion scoring model, and can be implemented using any existing or future model construction method.

[0091] The audio emotion score of the mth human-computer interaction process within the nth day in the current cycle is used to characterize the user emotion carried by the interaction audio of the mth human-computer interaction process within the nth day in the current cycle.

[0092] Based on the relevant content of S1131 above, it can be known that after obtaining the interactive audio of the mth human-computer interaction process within the nth day in the current cycle, the interactive audio can be input into a pre-built audio emotion scoring model, so that the audio emotion scoring model performs emotion scoring processing on the interactive audio, and obtains and outputs the audio emotion score of the mth human-computer interaction process within the nth day in the current cycle, so that the audio emotion score can accurately represent the user emotion carried by the interactive audio of the mth human-computer interaction process within the nth day in the current cycle. Wherein, m is a positive integer, m≤M, and M is a positive integer.

[0093] S1132: Averaging the audio emotion score of the 1st human-computer interaction process within the nth day in the current cycle to the audio emotion score of the Mth human-computer interaction process within the nth day in the current cycle to obtain the audio emotion score of the nth day in the current cycle.

[0094] In an embodiment of the present application, after obtaining the audio emotion scores of M human-computer interaction processes within the nth day in the current cycle, the average value of the audio emotion scores of the M human-computer interaction processes can be determined as the audio emotion score of the nth day in the current cycle, so that the audio emotion score of the nth day in the current cycle can accurately represent the user emotions carried by the interactive audio within the nth day in the current cycle.

[0095] Based on the relevant audio of S113 above, it can be known that after obtaining the interactive audio within the nth day in the current cycle, the audio emotion score of the nth day in the current cycle can be determined based on the interactive audio, so that the audio emotion score can accurately represent the user emotions carried by the interactive audio within the nth day in the current cycle, so that the audio emotion score can be referred to in the subsequent determination of the human-computer intimacy on the nth day in the current cycle.

[0096] S114: Determine the image emotion score on the nth day in the current cycle according to the interactive images on the nth day in the current cycle.

[0097] The image emotion score of the nth day in the current cycle is used to represent the user emotion carried by the interactive image within the nth day in the current cycle.

[0098] In addition, the embodiment of the present application does not limit the implementation of S114. For example, if there are M human-computer interaction processes within the nth day in the current cycle, S114 may specifically include S1141-S1142:

[0099] S1141: Input the interaction image of the mth human-computer interaction process within the nth day in the current cycle into a pre-built image emotion scoring model, and obtain the image emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the image emotion scoring model, where m is a positive integer, m≤M, and M is a positive integer.

[0100] The image emotion scoring model is used to perform emotion scoring processing on the input data of the image emotion scoring model; moreover, the embodiment of the present application is not limited to the image emotion scoring model, and can be any existing or future model that can perform emotion scoring processing on images (such as Figure 4 The neural network model 400 shown is implemented.

[0101] In addition, the image sentiment scoring model can be constructed based on the sample image and the actual sentiment score of the sample image. The sample image can be any image (such as a face image, etc.); and the embodiment of the present application does not limit the method for obtaining the actual sentiment score of the sample image (such as, it can be implemented by manual annotation).

[0102] In addition, the embodiments of the present application do not limit the construction process of the image emotion scoring model, and can be implemented using any existing or future model construction method.

[0103] The image emotion score of the mth human-computer interaction process within the nth day in the current cycle is used to characterize the user emotion carried by the interaction image of the mth human-computer interaction process within the nth day in the current cycle.

[0104] Based on the relevant content of S1141 above, it can be known that after obtaining the interactive image of the mth human-computer interaction process within the nth day in the current cycle, the interactive image can be input into the pre-constructed image emotion scoring model, so that the image emotion scoring model performs emotion scoring processing on the interactive image, and obtains and outputs the image emotion score of the mth human-computer interaction process within the nth day in the current cycle, so that the image emotion score can accurately represent the user emotion carried by the interactive image of the mth human-computer interaction process within the nth day in the current cycle. Wherein, m is a positive integer, m≤M, and M is a positive integer.

[0105] S1142: Averaging the image emotion scores of the first human-computer interaction process within the nth day in the current cycle to the image emotion scores of the Mth human-computer interaction process within the nth day in the current cycle to obtain the image emotion score of the nth day in the current cycle.

[0106] In an embodiment of the present application, after obtaining the image emotion scores of M human-computer interaction processes within the nth day in the current cycle, the average value of the image emotion scores of the M human-computer interaction processes can be determined as the image emotion score of the nth day in the current cycle, so that the image emotion score of the nth day in the current cycle can accurately represent the user emotions carried by the interactive images within the nth day in the current cycle.

[0107] Based on the relevant images of S114 above, it can be known that after obtaining the interactive image within the nth day in the current cycle, the image emotion score of the nth day in the current cycle can be determined based on the interactive image, so that the image emotion score can accurately represent the user emotions carried by the interactive image within the nth day in the current cycle, so that the image emotion score can be referred to in the subsequent determination of the human-computer intimacy on the nth day in the current cycle.

[0108] S115: Determine the human-machine intimacy on the nth day in the current cycle according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

[0109] The embodiment of the present application does not limit the implementation of S115. For example, in a possible implementation, S115 may specifically include S115A1-S115A2:

[0110] S115A1: Concatenate the content emotion score of the nth day in the current cycle, the content emotion score in the historical time period corresponding to the nth day in the current cycle, the audio emotion score of the nth day in the current cycle, the audio emotion score in the historical time period corresponding to the nth day in the current cycle, the image emotion score of the nth day in the current cycle, and the image emotion score in the historical time period corresponding to the nth day in the current cycle to obtain the emotion score concatenation data of the nth day in the current cycle.

[0111] The historical time period corresponding to the nth day in the current cycle can be determined based on the nth day in the current cycle and the preset number of days; and the historical time period corresponding to the nth day in the current cycle can be expressed as [the nth day in the current cycle - the preset number of days, the nth day in the current cycle - 1]. For example, if the nth day in the current cycle is January 15, 2021, and the preset number of days is 20 days, then the historical time period corresponding to the nth day in the current cycle can be [December 26, 2020, January 14, 2021].

[0112] The content sentiment score in the historical time period corresponding to the nth day in the current cycle may include the content sentiment score of each day in the historical time period corresponding to the nth day in the current cycle.

[0113] The audio emotion score in the historical time period corresponding to the nth day in the current cycle may include the audio emotion score of each day in the historical time period corresponding to the nth day in the current cycle.

[0114] The image emotion score in the historical time period corresponding to the nth day in the current cycle may include the image emotion score of each day in the historical time period corresponding to the nth day in the current cycle.

[0115] In addition, the present application embodiment does not limit the implementation method of "splicing" in S115A1. For example, Figure 5 The splicing method shown is implemented.

[0116] The sentiment score concatenation data of the nth day in the current cycle is used to describe the user sentiment in the nth day in the current cycle and its corresponding historical time period.

[0117] S115A2: Input the emotion score splicing data of the nth day in the current cycle into a pre-built intimacy determination model to obtain the human-computer intimacy of the nth day in the current cycle output by the intimacy determination model.

[0118] The intimacy determination model is used to calculate the intimacy for the input data of the intimacy determination model; and the embodiment of the present application does not limit the intimacy determination model, and any existing model that can calculate the human-computer intimacy based on multiple emotion scores can be used (such as Figure 6 The neural network model 600 shown is implemented.

[0119] It should be noted that Figure 6 The “continuous T+1 days” includes the historical time period corresponding to the nth day in the current cycle and the nth day in the current cycle.

[0120] In addition, the intimacy determination model is constructed based on the sample score splicing data and the actual human-machine intimacy corresponding to the sample score splicing data. The sample score splicing data is obtained by splicing the sample content emotion scores of consecutive (preset days + 1) days, the sample audio emotion scores of consecutive (preset days + 1) days, and the sample image emotion scores of consecutive (preset days + 1) days.

[0121] Based on the relevant contents of S115A1 to S115A2 above, it can be known that after obtaining the emotion score data of the nth day in the current cycle (such as the content emotion score, audio emotion score and image emotion score), the intimacy determination model can be used to analyze the human-computer intimacy of the nth day in the current cycle from the emotion score data of the nth day in the current cycle and the emotion score data of the corresponding historical time period, which is conducive to improving the accuracy of human-computer intimacy.

[0122] In some cases, in order to further improve the accuracy of human-computer intimacy, the embodiments of the present application also provide another possible implementation of S115. In this implementation, if the human-computer interaction status data includes the number of user active interactions, the number of device active interactions, the interaction duration, interaction audio and interaction images, then S115 may specifically include: determining the human-computer intimacy on the nth day in the current cycle based on the number of user active interactions on the nth day in the current cycle, the number of device active interactions on the nth day in the current cycle, the interaction duration on the nth day in the current cycle, the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle and the image emotion score on the nth day in the current cycle.

[0123] In order to facilitate understanding of the above implementation, the following description is given with reference to examples.

[0124] As an example, S115 may specifically include S115B1-S115B5:

[0125] S115B1: Add the number of active user interactions within the nth day in the current cycle and the number of active user interactions within the historical time period corresponding to the nth day in the current cycle to obtain the sum of the number of active user interactions corresponding to the nth day in the current cycle.

[0126] The number of active user interactions in the historical time period corresponding to the nth day in the current cycle may be obtained by adding up the number of active user interactions on each day in the historical time period corresponding to the nth day in the current cycle.

[0127] S115B2: Add the number of active device interactions within the nth day in the current cycle and the number of active device interactions within the historical time period corresponding to the nth day in the current cycle to obtain the sum of the number of active device interactions corresponding to the nth day in the current cycle.

[0128] The number of active device interactions in the historical time period corresponding to the nth day in the current cycle may be obtained by adding up the number of active device interactions on each day in the historical time period corresponding to the nth day in the current cycle.

[0129] S115B3: Add the interaction duration within the nth day in the current cycle and the interaction duration in the historical time period corresponding to the nth day in the current cycle to obtain the sum of the interaction durations corresponding to the nth day in the current cycle.

[0130] The interaction duration in the historical time period corresponding to the nth day in the current cycle may be obtained by adding up the interaction duration of each day in the historical time period corresponding to the nth day in the current cycle.

[0131] S115B4: Concatenate the sum of the number of active user interactions corresponding to the nth day in the current cycle, the sum of the number of active device interactions corresponding to the nth day in the current cycle, the sum of the interaction duration corresponding to the nth day in the current cycle, the content emotion score of the nth day in the current cycle, the content emotion score of the historical time period corresponding to the nth day in the current cycle, the audio emotion score of the nth day in the current cycle, the audio emotion score of the historical time period corresponding to the nth day in the current cycle, the image emotion score of the nth day in the current cycle, and the image emotion score of the historical time period corresponding to the nth day in the current cycle to obtain the state concatenated data of the nth day in the current cycle.

[0132] The state splicing data of the nth day in the current cycle is used to describe the user emotions of the nth day in the current cycle and its corresponding historical time period.

[0133] It should be noted that the embodiment of the present application does not limit the implementation method of "splicing" in S115B4. For example, Figure 7 The splicing method shown is implemented.

[0134] S115B5: Input the state splicing data of the nth day in the current cycle into a pre-built intimacy generation model to obtain the human-machine intimacy of the nth day in the current cycle output by the intimacy generation model.

[0135] The intimacy generation model is used to calculate the intimacy for the input data of the intimacy generation model; and the embodiment of the present application does not limit the intimacy generation model, and any existing model that can calculate the human-computer intimacy based on a variety of reference information can be used (such as Figure 8 The neural network model 800 shown is implemented.

[0136] It should be noted that Figure 8 The “continuous T+1 days” includes the historical time period corresponding to the nth day in the current cycle and the nth day in the current cycle. In addition, Figure 8 The “number of interactions in consecutive T+1 days” includes the number of active user interactions in consecutive T+1 days and the number of active device interactions in consecutive T+1 days.

[0137] In addition, the intimacy generation model is constructed based on the sample state splicing data and the actual human-machine intimacy corresponding to the sample state splicing data. Among them, the sample state splicing data is obtained by splicing the number of sample user active interactions for consecutive (preset days + 1) days, the number of sample device active interactions for consecutive (preset days + 1) days, the sample interaction time for consecutive (preset days + 1) days, the sample content emotion score for consecutive (preset days + 1) days, the sample audio emotion score for consecutive (preset days + 1) days, and the sample image emotion score for consecutive (preset days + 1) days.

[0138] Based on the relevant contents of S115B1 to S115B2 above, it can be known that after obtaining the human-computer interaction status data of the nth day in the current cycle (such as the number of user active interactions, the number of device active interactions and the interaction duration) and the emotion score data (such as the content emotion score, the audio emotion score and the image emotion score), the intimacy determination model can be used to analyze the human-computer intimacy of the nth day in the current cycle from the emotion score data and part of the human-computer interaction status data of the nth day in the current cycle and the emotion score data and part of the human-computer interaction status data of the corresponding historical time period, which is conducive to improving the accuracy of human-computer intimacy.

[0139] Based on the relevant content of S11 above, it can be known that after obtaining the human-computer interaction status data on the nth day in the current cycle, the human-computer intimacy on the nth day in the current cycle can be determined based on the human-computer interaction status data, so that the human-computer intimacy can accurately represent the degree of liking of the user of the human-computer interaction device for the human-computer interaction device in the current cycle. Wherein, n is a positive integer, n≤N, and N is a positive integer.

[0140] S12: Determine the set of the human-machine intimacy from the 1st day in the current cycle to the human-machine intimacy on the Nth day in the current cycle as the human-machine intimacy in the current cycle.

[0141] In an embodiment of the present application, after obtaining the human-computer intimacy of N days in the current cycle, the set of the human-computer intimacy of the N days can be determined as the human-computer intimacy within the current cycle, so that the human-computer intimacy within the current cycle can include the human-computer intimacy of the 1st day in the current cycle, the human-computer intimacy of the 2nd day in the current cycle, ... and the human-computer intimacy of the Nth day in the current cycle.

[0142] Based on the relevant content of S1 above, it can be known that for the adjustment process of the device control parameters of the current round of human-computer interaction equipment, the human-computer interaction status data in the current cycle is first obtained; then, based on the human-computer interaction status data in the current cycle, the human-computer intimacy in the current cycle is determined, so that the human-computer intimacy in the current cycle can represent the degree of liking of the user of the human-computer interaction device for the human-computer interaction device in the current cycle.

[0143] S2: Update the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle.

[0144] Among them, the human-computer intimacy in the previous cycle is determined based on the human-computer interaction state data in the previous cycle. It should be noted that the determination process of "the human-computer intimacy in the previous cycle" can be implemented by adopting any of the implementation methods of the determination process of "the human-computer intimacy in the current cycle" above, and it is only necessary to replace "the current cycle" with "the previous cycle" in any of the implementation methods of the determination process of "the human-computer intimacy in the current cycle" above.

[0145] In addition, the reinforcement learning model can be updated according to the environmental reward value of its previous decision. Based on this, the embodiment of the present application provides a possible implementation of S2, which can specifically include S21-S22:

[0146] S21: Inputting the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle into a pre-built intimacy change analysis model, and obtaining the intimacy change data corresponding to the current cycle output by the intimacy change analysis model.

[0147] The intimacy change analysis model is used to perform data change analysis on the input data of the intimacy change analysis model; moreover, the embodiment of the present application is not limited to the intimacy change analysis model, and any existing or future model that can perform data change analysis (such as Fig. 9 The neural network model 900 shown is implemented.

[0148] In addition, the intimacy change analysis model is constructed based on the human-machine intimacy within the sample time period and the actual intimacy change data corresponding to the sample time period. The sample time period pair includes a first sample time period and a second sample time period, and the end time point of the first sample time period is earlier than the start time point of the second sample time period. The actual intimacy change data corresponding to the sample time period pair is used to describe the change in human-machine intimacy in the second sample time period compared to the human-machine intimacy in the first sample time period.

[0149] The intimacy change data corresponding to the current cycle is used to describe the change in human-computer intimacy in the current cycle relative to the human-computer intimacy in the previous cycle, so that the intimacy change data corresponding to the current cycle can be used to represent the impact of the device control parameters determined by the reinforcement learning model during the adjustment of the device control parameters of the previous round of human-computer interaction devices on the human-computer interaction process in the current cycle.

[0150] In addition, the embodiment of the present application does not limit the representation method of the intimacy change data corresponding to the current period. For example, it can be represented by any value in the numerical interval [-1, 1]; and its meaning may specifically include:

[0151] If the intimacy change data corresponding to the current cycle is a positive number, and the larger the intimacy change data corresponding to the current cycle is, the greater the improvement of the human-computer intimacy in the current cycle relative to the human-computer intimacy in the previous cycle is, which indicates that the device control parameters determined by the reinforcement learning model in the adjustment process of the device control parameters of the previous round of human-computer interaction devices have a greater positive impact on the human-computer interaction process in the current cycle; if the intimacy change data corresponding to the current cycle is a negative number, and the smaller the intimacy change data corresponding to the current cycle is, the greater the decrease of the human-computer intimacy in the current cycle relative to the human-computer intimacy in the previous cycle is, which indicates that the device control parameters determined by the reinforcement learning model in the adjustment process of the device control parameters of the previous round of human-computer interaction devices have a greater negative impact on the human-computer interaction process in the current cycle; if the intimacy change data corresponding to the current cycle is 0, it indicates that the human-computer intimacy in the current cycle has not changed relative to the human-computer intimacy in the previous cycle, which indicates that the device control parameters determined by the reinforcement learning model in the adjustment process of the device control parameters of the previous round of human-computer interaction devices have no impact on the human-computer interaction process in the current cycle.

[0152] S22: Update the reinforcement learning model according to the intimacy change data corresponding to the current cycle.

[0153] In an embodiment of the present application, since the intimacy change data corresponding to the current cycle can represent the impact of the device control parameters determined by the reinforcement learning model during the adjustment of the device control parameters of the previous round of human-computer interaction devices on the human-computer interaction process in the current cycle, the intimacy change data corresponding to the current cycle can be used as an environmental reward value. Therefore, the intimacy change data corresponding to the current cycle can be used to update the reinforcement learning model, so that the updated reinforcement learning model can better decide the device control parameters in the next cycle.

[0154] Based on the relevant content of S2 above, it can be known that after obtaining the human-computer intimacy in the current cycle, the human-computer intimacy in the current cycle can be compared with the human-computer intimacy in the previous cycle to determine the intimacy change data corresponding to the current cycle; then use the intimacy change data corresponding to the current cycle to update the reinforcement learning model, so that the updated reinforcement learning model can better decide the device control parameters in the next cycle, which is conducive to optimizing the control process of the human-computer interaction equipment.

[0155] S3: Input the human-machine intimacy in the previous cycle, the device control parameters in the current cycle, and the human-machine intimacy in the current cycle into the reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model.

[0156] Among them, the device control parameter is used to describe the control process of the human-computer interaction device. In addition, the embodiment of the present application does not limit the device control parameter. For example, the device control parameter may include the control parameter of the interaction process (such as the frequency of the device actively initiating the human-computer interaction process, etc.).

[0157] The device control parameters in the current cycle refer to the relevant parameters for controlling the human-machine interaction device in the current cycle. In addition, the device control parameters in the current cycle are determined during the adjustment process of the device control parameters of the human-machine interaction device in the previous round.

[0158] The device control parameters in the next cycle refer to relevant parameters for controlling the human-computer interaction device in the next cycle.

[0159] Based on the relevant contents of S1 to S3 above, it can be known that for the human-computer interaction control method provided in the present application, after obtaining the human-computer interaction status data in the current cycle, first determine the human-computer intimacy in the current cycle based on the human-computer interaction status data in the current cycle; then update the reinforcement learning model based on the human-computer intimacy in the previous cycle and the human-computer intimacy in the current cycle, so that the updated reinforcement learning model can better make device control parameter decisions; finally, input the human-computer intimacy in the previous cycle, the device control parameters in the current cycle, and the human-computer intimacy in the current cycle into the updated reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model, so that the human-computer interaction device can be controlled in the next cycle using the device control parameters in the next cycle.

[0160] It can be seen that because the reinforcement learning model can learn in real time how to better make device control parameter decisions from the human-computer interaction process that has been experienced based on human-computer intimacy, the device control parameters in the next cycle determined by the reinforcement learning model based on human-computer intimacy are more suitable for the human-computer interaction process in the next cycle. In this way, the control process of the human-computer interaction device can be adaptively adjusted as the human-computer intimacy changes, which is beneficial to improving the user experience.

[0161] In addition, the embodiments of the present application do not limit the execution subject of the human-computer interaction control method. For example, the human-computer interaction control method provided in the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (PDA) or a tablet computer. The server can be an independent server, a cluster server or a cloud server.

[0162] Based on the human-computer interaction control method provided by the above method embodiment, the embodiment of the present application also provides a human-computer interaction control device, which is explained and illustrated below in conjunction with the accompanying drawings.

[0163] Device Embodiment

[0164] The device embodiment introduces the human-computer interaction control device. For related content, please refer to the above method embodiment.

[0165] See also Fig.10 , which is a structural schematic diagram of a human-computer interaction control device provided in an embodiment of the present application.

[0166] The human-computer interaction control device 1000 provided in the embodiment of the present application includes:

[0167] The intimacy determination unit 1001 is used to determine the human-computer intimacy in the current cycle according to the human-computer interaction state data in the current cycle;

[0168] A model updating unit 1002, configured to update the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle; wherein the human-machine intimacy in the previous cycle is determined according to the human-machine interaction state data in the previous cycle;

[0169] The parameter decision unit 1003 is used to input the human-machine intimacy in the previous cycle, the device control parameters in the current cycle and the human-machine intimacy in the current cycle into the reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model.

[0170] In a possible implementation, the current period includes N days; and the intimacy determination unit 1001 includes:

[0171] The first determination subunit is used to determine the human-machine intimacy on the nth day in the current cycle according to the human-machine interaction state data on the nth day in the current cycle; wherein n is a positive integer, n≤N, and N is a positive integer;

[0172] The second determining subunit is used to determine the set of the human-machine intimacy on the first day in the current cycle to the human-machine intimacy on the Nth day in the current cycle as the human-machine intimacy in the current cycle.

[0173] In a possible implementation manner, the human-computer interaction state data includes interaction audio and interaction image; and the first determining subunit includes:

[0174] A third determining subunit is used to determine the interactive content within the nth day in the current cycle according to the interactive audio within the nth day in the current cycle;

[0175] A fourth determination subunit, configured to determine a content sentiment score for the nth day in the current cycle according to the interactive content on the nth day in the current cycle;

[0176] A fifth determining subunit, configured to determine an audio emotion score on the nth day in the current cycle according to the interactive audio on the nth day in the current cycle;

[0177] a sixth determining subunit, configured to determine an image emotion score on the nth day in the current cycle according to the interactive images on the nth day in the current cycle;

[0178] The seventh determination subunit is used to determine the human-computer intimacy on the nth day in the current cycle based on the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

[0179] In a possible implementation manner, the seventh determining subunit is specifically configured to:

[0180] The content emotion score of the nth day in the current cycle, the content emotion score in the historical time period corresponding to the nth day in the current cycle, the audio emotion score of the nth day in the current cycle, the audio emotion score in the historical time period corresponding to the nth day in the current cycle, the image emotion score of the nth day in the current cycle, and the image emotion score in the historical time period corresponding to the nth day in the current cycle are spliced ​​to obtain the emotion score splicing data of the nth day in the current cycle;

[0181] The emotion score splicing data of the nth day in the current cycle is input into a pre-constructed intimacy determination model to obtain the human-computer intimacy of the nth day in the current cycle output by the intimacy determination model; wherein the intimacy determination model is constructed based on the sample score splicing data and the actual human-computer intimacy corresponding to the sample score splicing data.

[0182] In a possible implementation manner, the human-computer interaction state data further includes the number of active user interactions, the number of active device interactions, and the interaction duration; and the seventh determination subunit is specifically used to:

[0183] The human-computer intimacy on the nth day in the current cycle is determined based on the number of active user interactions on the nth day in the current cycle, the number of active device interactions on the nth day in the current cycle, the interaction duration on the nth day in the current cycle, the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

[0184] In a possible implementation manner, there are M human-computer interaction processes on the nth day in the current cycle;

[0185] The fourth determination subunit is specifically used to: input the interactive content of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed content emotion scoring model, and obtain the content emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the content emotion scoring model; wherein the content emotion scoring model is constructed based on sample content and the actual emotion score of the sample content; m is a positive integer, m≤M, and M is a positive integer; average the content emotion scores of the first human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle, and obtain the content emotion score of the nth day in the current cycle;

[0186] The fifth determination subunit is specifically used to: input the interactive audio of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed audio emotion scoring model, and obtain the audio emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the audio emotion scoring model; wherein the audio emotion scoring model is constructed based on the sample audio and the actual emotion score of the sample audio; m is a positive integer, m≤M, and M is a positive integer; average the audio emotion scores of the 1st human-computer interaction process within the nth day in the current cycle to the audio emotion scores of the Mth human-computer interaction process within the nth day in the current cycle, and obtain the audio emotion score of the nth day in the current cycle;

[0187] The sixth determination subunit is specifically used to: input the interactive image of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed image emotion scoring model, and obtain the image emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the image emotion scoring model; wherein the image emotion scoring model is constructed based on sample images and actual emotion scores of the sample images; m is a positive integer, m≤M, and M is a positive integer; average the image emotion scores of the 1st human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle, and obtain the image emotion score of the nth day in the current cycle.

[0188] In a possible implementation manner, the model updating unit 1002 is specifically configured to:

[0189] Input the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle into a pre-constructed intimacy change analysis model, and obtain the intimacy change data corresponding to the current cycle output by the intimacy change analysis model; wherein the intimacy change analysis model is constructed based on the human-machine intimacy in a sample time period and the actual intimacy change data corresponding to the sample time period;

[0190] The reinforcement learning model is updated according to the intimacy change data corresponding to the current period.

[0191] Furthermore, an embodiment of the present application also provides a human-computer interaction control device, including: a processor, a memory, and a system bus;

[0192] The processor and the memory are connected via the system bus;

[0193] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned human-computer interaction control method.

[0194] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any implementation method of the above-mentioned human-computer interaction control method.

[0195] Furthermore, an embodiment of the present application also provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the implementation methods of the above-mentioned human-computer interaction control method.

[0196] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0197] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0198] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0199] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A human-computer interaction control method, characterized in that: The method comprises: Determining the human-machine intimacy within the current cycle according to the human-machine interaction state data within the current cycle; updating the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle; wherein the human-machine intimacy in the previous cycle is determined according to the human-machine interaction state data in the previous cycle; Inputting the human-machine intimacy in the previous cycle, the device control parameters in the current cycle, and the human-machine intimacy in the current cycle into the reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model; The updating of the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle includes: Input the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle into a pre-constructed intimacy change analysis model, and obtain the intimacy change data corresponding to the current cycle output by the intimacy change analysis model; wherein the intimacy change analysis model is constructed based on the human-machine intimacy in a sample time period and the actual intimacy change data corresponding to the sample time period; The reinforcement learning model is updated according to the intimacy change data corresponding to the current period.

2. The method according to claim 1, characterized in that If the current period includes N days, the process of determining the human-machine intimacy within the current period includes: Determine the human-machine intimacy on the nth day in the current cycle according to the human-machine interaction state data on the nth day in the current cycle; wherein n is a positive integer, n≤N, and N is a positive integer; The set of the human-machine intimacy from the first day in the current cycle to the human-machine intimacy on the Nth day in the current cycle is determined as the human-machine intimacy in the current cycle.

3. The method according to claim 2, characterized in that If the human-computer interaction state data includes interactive audio and interactive images, the process of determining the human-computer intimacy on the nth day in the current cycle includes: Determining the interactive content within the nth day in the current cycle according to the interactive audio within the nth day in the current cycle; Determining a content sentiment score for the nth day in the current cycle according to the interactive content within the nth day in the current cycle; Determine the audio emotion score of the nth day in the current cycle according to the interactive audio on the nth day in the current cycle; Determine, according to the interactive images within the nth day in the current cycle, the image emotion score of the nth day in the current cycle; The human-machine intimacy on the nth day in the current cycle is determined according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

4. The method according to claim 3, characterized in that The determining the human-machine intimacy on the nth day in the current cycle according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle includes: The content emotion score of the nth day in the current cycle, the content emotion score in the historical time period corresponding to the nth day in the current cycle, the audio emotion score of the nth day in the current cycle, the audio emotion score in the historical time period corresponding to the nth day in the current cycle, the image emotion score of the nth day in the current cycle, and the image emotion score in the historical time period corresponding to the nth day in the current cycle are spliced ​​to obtain the emotion score splicing data of the nth day in the current cycle; The emotion score splicing data of the nth day in the current cycle is input into a pre-constructed intimacy determination model to obtain the human-computer intimacy of the nth day in the current cycle output by the intimacy determination model; wherein the intimacy determination model is constructed based on the sample score splicing data and the actual human-computer intimacy corresponding to the sample score splicing data.

5. The method according to claim 3, characterized in that: If the human-computer interaction state data also includes the number of active user interactions, the number of active device interactions, and the interaction duration, then determining the human-computer intimacy on the nth day in the current cycle according to the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle includes: The human-computer intimacy on the nth day in the current cycle is determined based on the number of active user interactions on the nth day in the current cycle, the number of active device interactions on the nth day in the current cycle, the interaction duration on the nth day in the current cycle, the content emotion score on the nth day in the current cycle, the audio emotion score on the nth day in the current cycle, and the image emotion score on the nth day in the current cycle.

6. The method according to claim 3, characterized in that If there are M human-computer interaction processes within the nth day in the current cycle, the process of determining the content sentiment score of the nth day in the current cycle includes: Input the interactive content of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed content emotion scoring model, and obtain the content emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the content emotion scoring model; wherein the content emotion scoring model is constructed based on sample content and the actual emotion score of the sample content; m is a positive integer, m≤M, and M is a positive integer; average the content emotion scores of the first human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle, and obtain the content emotion score of the nth day in the current cycle; The process of determining the audio sentiment score of the nth day in the current cycle includes: Input the interactive audio of the mth human-computer interaction process within the nth day in the current cycle into a pre-constructed audio emotion scoring model, and obtain the audio emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the audio emotion scoring model; wherein the audio emotion scoring model is constructed based on sample audio and the actual emotion score of the sample audio; m is a positive integer, m≤M, and M is a positive integer; average the audio emotion scores of the 1st human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle, and obtain the audio emotion score of the nth day in the current cycle; The process of determining the image sentiment score on the nth day in the current cycle includes: The interactive image of the mth human-computer interaction process within the nth day in the current cycle is input into a pre-constructed image emotion scoring model to obtain the image emotion score of the mth human-computer interaction process within the nth day in the current cycle output by the image emotion scoring model; wherein the image emotion scoring model is constructed based on sample images and actual emotion scores of the sample images; m is a positive integer, m≤M, and M is a positive integer; the image emotion scores of the first human-computer interaction process within the nth day in the current cycle to the Mth human-computer interaction process within the nth day in the current cycle are averaged to obtain the image emotion score of the nth day in the current cycle.

7. A human-computer interaction control device, characterized in that: include: An intimacy determination unit, configured to determine the human-computer intimacy in the current cycle according to the human-computer interaction state data in the current cycle; A model updating unit, used to update the reinforcement learning model according to the human-machine intimacy in the previous cycle and the human-machine intimacy in the current cycle; wherein the human-machine intimacy in the previous cycle is determined according to the human-machine interaction state data in the previous cycle; A parameter decision unit, used to input the human-machine intimacy in the previous cycle, the device control parameters in the current cycle, and the human-machine intimacy in the current cycle into the reinforcement learning model to obtain the device control parameters in the next cycle output by the reinforcement learning model; The model updating unit is specifically used to: input the human-computer intimacy in the previous cycle and the human-computer intimacy in the current cycle into a pre-constructed intimacy change analysis model to obtain the intimacy change data corresponding to the current cycle output by the intimacy change analysis model; wherein the intimacy change analysis model is constructed based on the human-computer intimacy in a sample time period and the actual intimacy change data corresponding to the sample time period; and update the reinforcement learning model based on the intimacy change data corresponding to the current cycle.

8. A device, characterized in that: The device comprises a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 6 according to the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that When the computer program product is executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 6.

Images