Method and apparatus for interactive and privacy protection communication between server and user device

Abstract

To provide a method and an apparatus for interactive and privacy protection communication between a server and a user device.SOLUTION: A method and an apparatus for interactive communication between a user device and a server are described. A notification is provided to a user in the user device, and reaction data indicating the user's reaction to the notification is acquired. Further, based on the acquired reaction data, a sentiment score for transmission to the server is determined, and the sentiment score indicates user sentiment in reaction to the notification. Further, the sentiment score can be used by the server to improve interactive communication between the server and the user device.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure generally relates to the field of interactive communication between a server and a user device. In particular, the present disclosure relates to a computer-implemented method for interactive communication of a user device with a server, a computer-implemented method for interactive communication of a server with a user device, and a computer-implemented method for interactive communication between a server and a user device. The present disclosure further relates to a user device for interactive communication with a server, the use of such a user device, a server for interactive communication with a user device, and a system comprising such a user device and such a server. Furthermore, the present disclosure relates to one or more computer programs and one or more computer-readable media.

[0002] In an exemplary interactive communication, a query received from a user device may be processed by one or more servers, for example, using a natural language processing (NLP) engine or any other engine configured to process queries. The query may be sent or transmitted from the user device to the one or more servers via a communication link or data connection between the user device and the one or more servers. The query may then be processed by the one or more servers, and one or more notifications may be provided to the user device in response. As an example, such a query may include or indicate one or more of a user's text input via a user interface of the user device, a verbal input via a microphone of the user device, and a video-based input via a camera of the user device. Furthermore, the one or more notifications provided by the one or more servers in response to the query may include, for example, at least one of text, voice, an audio message, a message, an alert, an image, and a video.

[0003] Therein, identifying or selecting notifications that are appropriate responses to queries received from user devices can be a challenging task. To this end, for example, a deep reinforcement learning model or a reinforcement learning model can be implemented on one or more servers, and feedback from users regarding notifications provided in response to queries can be used as a reward to improve the reinforcement learning model with respect to providing notifications to queries with more appropriate, accurate, and / or correct answers or content.

[0004] However, the process of improving the reinforcement learning model may require a significant amount of computing power on one or more servers. Furthermore, the feedback from the users may contain personal information about the users, which may raise concerns about data protection on the user side and require additional security measures on the server side.

[0005] It may therefore be desirable to provide an improved method and apparatus for interactive communication between a server and a user device. This is achieved by the subject matter of the independent claims, with further embodiments incorporated in the dependent claims and the following description.

[0006] Various aspects and embodiments and examples of those aspects of the present disclosure are described below. It should be noted that various aspects of the present disclosure are at least partially, primarily, or completely related to one or more of at least one server, at least one user device, at least one user-side device, at least one user-side sensor, and at least one system including one or more of such components or devices. In other words, the present invention can be considered to be distributed among one or more of at least one server, at least one user device, at least one user-side device, at least one user-side sensor, and at least one system including one or more of such components or devices.

[0007] However, it is emphasized that any feature, step, function, element, technical effect and / or advantage described hereinabove and below with reference to one aspect of the present disclosure, as described hereinabove and below, applies to any other aspect of the present disclosure as well.

[0008] According to a first aspect of the present disclosure, a computer-implemented method for interactive communication of a user device with a server is provided. The method according to the first aspect may alternatively or additionally relate to a computer-implemented method for operating a user device, e.g., for interactive communication with a server, and optionally for operating one or more user-side devices. The method includes: - on the user device, providing a notification to a user of the user device; - obtaining reaction data indicative of a user's reaction to the notification; - determining a sentiment score for transmission to the server based on the obtained reaction data, the sentiment score indicating a sentiment of the user in reaction to the notification.

[0009] As described and discussed in more detail below, determining the sentiment score for the transmission to the server on the user device may enable a significant improvement in interactive communication between the server and the user device, or between the server and the user of the user device. In particular, by determining the sentiment score on the user device, the computing resources used by the server for interactive communication may be significantly reduced. Alternatively or additionally, the process or method of selecting, identifying, and / or generating notifications on the server (or an engine or module configured to do so) may be improved in terms of efficiency, quality, and accuracy. Alternatively or additionally, determining the sentiment score on the user device may enable the privacy or anonymity of the user of the user device to be protected.

[0010] In the context of the present disclosure, interactive communication between a server and a user device may include, for example, one or more notifications sent or transmitted from the server to the user device. Alternatively or additionally, interactive communication between a server and a user device may include, for example, one or more queries sent or transmitted from the user device to the server in response to one or more user inputs at the user device. Therein, one or more notifications may be sent or transmitted from the server to the user device in response to the one or more queries. Alternatively or additionally, one or more notifications may be sent or transmitted from the server to the user device proactively or automatically. For example, one or more notifications or at least some of the notifications may be time-triggered and / or event-triggered.

[0011] Additionally, a notification provided on a user device may refer to a notification having or including content that may be brought to the user's attention via the user device, for example. For example, a notification may include one or more of at least one letter, at least one symbol, at least one number, text, a voice, an audio message, a question, an answer to a question, a statement, a request, a message, a text message, at least one icon, an alarm, an audio alarm, an optical alarm, at least one image, and a video. In general, a notification may trigger and / or be configured to trigger a user response.

[0012] Similarly, in the context of this disclosure, a query may include one or more of at least one letter, at least one symbol, at least one number, text, a rating, a question, an answer to a question, a statement, a request, a voice, an audio message, a message, a text message, at least one icon, at least one image, and a video. For example, a query may include a text input received at the user device via a user interface, a verbal input received at the user device via a microphone, and / or a video-based input received via a camera on the user device.

[0013] In a non-limiting example, the interactive communication between the user device and the server may be based on Frequently Asked Questions ("FAQ"), where a query may relate to or include a user question transmitted to the server via the user device, and the question or query may be answered using one or more notifications of the server.

[0014] Further, the user's reaction to the notification may be or may refer to, for example, user movement, user gestures, user movements, user mimics, user facial expressions, head movements, emotions, emotional expressions, shaking heads, nodding, smiling, mouth movements, changes in user skin color, changes in skin color, verbal responses, acoustic responses, and physical responses such as, for example, goose skin, goose bumps, shaking body parts, waving hands, etc. Those skilled in the art will appreciate that these reactions may be sensed using readily available systems. For example, various software packages are provided for detecting emotions and emotional expressions from image data. One example of this software is the face detection package provided by Microsoft Azure™.

[0015] In the context of this disclosure, the term "reaction data" should be understood or interpreted broadly. For example, reaction data may include or refer to data that contains information about, indicates, represents, and / or is descriptive of one or more reactions of a user to one or more notifications. Alternatively or additionally, reaction data may include or refer to data that contains information about, indicates, represents, and / or is descriptive of one or more reactions of a user, where the reaction may not be related to one or more notifications and / or may not be caused or triggered by one or more notifications.

[0016] As an example, the response data may include or reference sensor data of a sensor configured to sense one or more responses of a user. Alternatively, or in addition, the response data may reference or include combined sensor data, such as, for example, sensor data from multiple sensors or independent sensors. Alternatively, or in addition, the response data may reference or include data derived from sensor data of one or more sensors.

[0017] As discussed further below, in at least some embodiments, the reaction data may be obtained based on or using one or more sensors of the user device. Alternatively, or in addition, the reaction data may be obtained based on or using one or more user-side sensors of one or more user-side devices.

[0018] Generally, a user's reaction may be associated with, indicative of, and / or representative of a user's sentiment in reaction and / or response to a notification provided on a user device, where the user's sentiment may refer to or be indicative of a user's feelings, moods, and / or emotions, for example. Thus, by obtaining reaction data that may indicate, describe, and / or represent a user's reaction to a notification, the user's sentiment may be determined and / or derived from the reaction data.

[0019] In the context of the present disclosure, a sentiment score may refer to or be a scale or indicator, e.g., a numerical scale, a numerical indicator, and / or a numerical score, indicative of a user's sentiment in response to a notification provided on a user device. Thus, the user's sentiment or actual sentiment may be encoded and / or converted into a sentiment score. In other words, the sentiment score may be determined, e.g., based on converting and / or encoding the user's sentiment into a sentiment score as determined based on the reaction data. Alternatively or additionally, the sentiment score may be derived from the reaction data, e.g., based on processing at least some or a portion of the reaction data. Alternatively or additionally, determining the sentiment score may include, e.g., calculating and / or computing the sentiment score based on the reaction data.

[0020] Additionally, the term sentiment score “for transmission to a server” may mean that the sentiment score may be configured for transmission and / or may be configured to be transmitted from a user device to a server.

[0021] Because the sentiment score indicates the user's sentiment in response to the notification, the sentiment score may function as and / or indicate the user's feedback on the notification. Thus, the sentiment score and / or the feedback indicated therewith may reflect the accuracy, level of appropriateness, and / or quality of the notification. Because the notification may be transmitted to the user device, for example, in response to one or more queries transmitted from the user device to a server, the sentiment score and / or the feedback indicated therewith may alternatively or additionally reflect the accuracy, level of appropriateness, and / or quality of the notification in response to the one or more queries.

[0022] Further, the determined sentiment score may be used by the server, may be usable by the server, and / or may be configured to be used by the server to improve, personalize, adapt, and / or modify interactive communications between the server and the user device. For example, the sentiment score may be usable by the server to modify, personalize, and / or adapt an engine or module of the server configured to select, identify, determine, and / or generate notifications, e.g., notifications responsive to one or more queries from a user device.

[0023] In the context of this disclosure, a server may comprise at least one computing device, e.g., including one or more processors. It should be noted, however, that a server may refer to a server system, a computer system, a server network, a computing network, a cloud computing network, etc. Thus, any reference to "a server" or "the server" should be interpreted as including multiple servers.

[0024] Generally, the server may comprise or include an engine or module for identifying, selecting, and / or generating notifications to be sent or transmitted to a user device, e.g., in response to one or more queries from the user device. The server may comprise, e.g., a natural language processing ("NLP") module or engine configured to process, analyze, and / or resolve one or more queries received from the user device. The server may be further configured to determine one or more notifications to be transmitted to the user device, e.g., in a proactive, automated, time-triggered, and / or event-triggered manner, in response to one or more queries.

[0025] Further, the server may comprise an artificial intelligence (also referred to as a recommendation engine) that allows for determining, identifying, selecting, and / or generating a notification that may represent an appropriate or most appropriate notification, for example, from among or based on a plurality of potential notifications that may be transmitted to the user device. As an example, the server may comprise a knowledge base that includes a plurality of notifications that may be transmitted to the user device. The artificial intelligence module of the server may be configured to determine, select, identify, and / or generate an appropriate or most appropriate notification from among or based on a plurality of notifications in the knowledge base. For example, one or more queries received from the user device may be classified using the artificial intelligence module of the server to determine, select, identify, and / or generate an appropriate or most appropriate notification from among or based on a plurality of notifications in the knowledge base.

[0026] For example, the sentiment score may be advantageously used by the server to train an artificial intelligence module to improve the quality, accuracy, level of appropriateness, and / or relevance of notifications selected by the artificial intelligence module of the server in response to one or more queries from the user device. Thus, by using one or more sentiment scores to train the artificial intelligence module, it may be possible to continuously improve the artificial intelligence module over time.

[0027] In general, an artificial intelligence module (and / or recommendation engine) may refer to a classifier arrangement or classifier circuitry of a server configured to classify an input, e.g., one or more queries, and determine an output, e.g., one or more notifications to be transmitted to a user device based on the classification. For example, the artificial intelligence module may include a neural network, may refer to a machine learning module, and / or may relate to a deep learning module. However, any other type of artificial intelligence module may be used.

[0028] As an example, the server, the artificial intelligence module, and / or the recommendation engine may include a reinforcement learning model, and the sentiment score may be used by the server to train the server's reinforcement learning model. In general, the reinforcement learning model may be configured to be trained based on maximizing and / or optimizing a reward function over time, for example, continuously and / or iteratively in multiple training steps. In each training step, a reinforcement learning reward (also referred to as a "reward") reflecting and / or indicating the level of correctness or quality of the decision of the reinforcement learning module performed may be provided or dispensed to the reinforcement learning model to penalize it when it makes an incorrect decision and reward it when it makes a correct decision.

[0029] Thus, the sentiment score may be advantageously used as a reinforcement learning reward. Alternatively, or additionally, a reinforcement learning reward may be derived from and / or determined based on the sentiment score, and the reinforcement learning reward may be used by or granted on the server to train the server's reinforcement learning model. For example, the sentiment score may correlate with and / or be indicative of a reinforcement learning reward for training the server's reinforcement learning model.

[0030] Thus, based on the sentiment score, the server's reinforcement learning model and / or artificial intelligence module can be efficiently and accurately trained, thereby enabling efficiently improving interactive communication between the server and the user device, e.g., in terms of improving the level of accuracy, the level of appropriateness, the quality, and / or the relevance of one or more notifications selected, identified, and / or generated by the server in response to one or more queries from the user device.

[0031] Furthermore, the interaction between the server and the user device may be personalized over time based on the sentiment score. Therefore, it may be possible to meet user demand while requiring minimal user effort to ensure a sufficient level of accuracy to provide correct notifications in response from the user device. Such efforts may involve, for example, requesting explicit feedback from the user based on evaluating the correctness of the notifications provided in response to the query. Thus, based on the sentiment score, the interaction between the server and the user device may be improved in an automated manner while keeping user effort, such as explicit feedback from the user, to a minimum or eliminating such user effort.

[0032] In the context of the present disclosure, a user device may refer to any device or apparatus configured to communicate with a server, e.g., based on an exchange of data or information. Generally, a user device may be a handheld or portable device. Alternatively, a user device may be a stand-alone or fixedly installed device. By way of example, a user device may refer to a handheld, a smartphone, a personal computer ("PC"), a tablet PC, a laptop, or a computer. A user device may be configured to receive input from a user, e.g., one or more queries, e.g., via a user interface of the user device.

[0033] The user device may include one or more processors for data processing. Additionally, the user device may include a data storage device for storing data, such as, for example, reaction data. The user device may have software instructions or computer programs stored in the data storage device that, when executed by the one or more processors, instruct the user device to perform one or more steps of the method according to the first aspect and / or any other aspect of the present disclosure. Alternatively or additionally, an app or computer program may be stored on the user device configured to interactively communicate with the server, for example, based on transmitting one or more queries to the server and receiving one or more notifications in response thereto.

[0034] The user device may be configured to communicate with the server via a communication link or connection between the server and the user device, for example, using a communication protocol, standard or technology. To this end, the user device may be equipped with a communication circuitry that can be coupled with a corresponding communication arrangement of the server to enable data exchange between the user device and the server. As an example, the user device may be coupled and / or configured to communicate with the server (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a radio connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0035] The method may further include transmitting the determined sentiment score to a server. In other words, the sentiment score may be transmitted from the user device to the server. In which, transmitting the sentiment score to the server may include providing and / or sending the sentiment score to the server. Optionally, transmitting the sentiment score to the server may include establishing a communication link or connection between the user device and the server. Optionally, transmitting the sentiment score may include confirming receipt of the sentiment score at the server, for example, based on a signal or confirmation transmitted by the server to the user device upon receipt of the sentiment score. By transmitting the sentiment score to the server, the sentiment score is provided to the server, thereby enabling the server to use the sentiment score to, for example, improve, modify, personalize, and / or adapt interactive communication between the server and the user device.

[0036] The sentiment score may be transmitted by the user device to the server in an automated manner. This may mean that user interaction with and / or user input on the user device is not required to transmit the sentiment score. Thus, the determination of the sentiment score may trigger the transmission of the sentiment score to the user device. Alternatively, or additionally, the sentiment score may be transmitted upon or in response to user input at the user device.

[0037] The sentiment score may be determined by the user device, for example, based on processing at least a portion of the reaction data with control circuitry of the user device. Alternatively or additionally, the sentiment score may be determined at and / or on at least one user-side device communicatively coupled or couplable with the user device. For example, the user device may instruct the user-side device to determine the sentiment score based on processing at least a portion of the reaction data. Alternatively or additionally, determining the sentiment score may include determining the sentiment score on the user device and providing the sentiment score to the user-side device, for example, for transmission to a server or further processing of the sentiment score. Alternatively or additionally, determining the sentiment score may include determining the sentiment score on the user-side device and providing the sentiment score to the user device, for example, for transmission to a server or further processing of the sentiment score.

[0038] Determining the sentiment score on the user device and / or the user-side device may advantageously allow for reducing the computing resources used on the server, since the computing load of determining the sentiment score may be distributed to the user device and / or the user-side device. The overall interactive communication between the server and the user device may then be significantly improved in terms of a more efficient and faster reaction of the server, for example, in response to a query from the user device. Apart from the effort and cost, for example, the hardware cost and / or maintenance cost for the server may be advantageously reduced.

[0039] In the context of the present disclosure, a user-side device may refer to a device or apparatus located near, around, in the environment, and / or remotely from a user device. Alternatively or additionally, a user-side device may refer to a device or apparatus that is under at least partial control of a user of the user device. For example, a user device may be configured to transmit one or more control signals to a user-side device, and / or a user-side device may be configured to receive one or more control signals from a user device. Alternatively or additionally, a user-side device may refer to a device or apparatus that is connected to a user device via a common communication network, for example, the same local area network (LAN) as the user device.

[0040] The user side device may be configured to communicate with the user device via a communication link or connection between the user side device and the user device, for example, using a communication protocol, standard or technology. For this purpose, the user side device may comprise a communication circuit that can be coupled with a corresponding communication circuit of the user device to enable data exchange between the user side device and the user side device. As an example, the user side device may be coupled and / or configured to communicate with the user device (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0041] Furthermore, the user-side device may be configured to communicate with the server via a communication link or connection between the user-side device and the server, for example, using a communication protocol, standard or technology. For this purpose, the user-side device may be equipped with a communication circuit that can be coupled with a corresponding communication arrangement of the server to enable data exchange between the server and the user-side device. As an example, the user-side device may be coupled with and / or configured to communicate with the server (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, a LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol. The user-side device may be configured to communicate with the server via the user device.

[0042] The user-side device may be configured to sense, monitor, and / or detect the vicinity, environment, and / or surroundings of the user-side device. For this purpose, the user-side device may comprise one or more user-side sensors. The user-side device may be configured, for example, to capture and / or acquire at least a portion of the reaction data. Alternatively or additionally, the user-side device may comprise a controller or control circuitry configured to process data, for example, reaction data and / or user-side sensor data acquired by one or more user-side sensors of the user-side device.

[0043] As an example, the user-side device may refer to or be one or more of a smart TV, a smart speaker, a smart watch, a smart device health monitor, and an electric aerosol generating device. However, it should be noted that the present disclosure is not limited to the above-mentioned exemplary user-side devices.

[0044] The determined sentiment score may be transmitted from the user device to the server and / or from at least one user-side device communicatively coupled to the user device. As an example, the user device may, for example, upon determination of the sentiment score at the user device or the user-side device, instruct the user-side device to transmit the sentiment score to the server. Alternatively or additionally, transmitting the sentiment score to the server may include transmitting the sentiment score from the user device to the user-side device and transmitting the sentiment score from the user-side device to the server. Vice versa, transmitting the sentiment score to the server may include transmitting the sentiment score from the user-side device to the user device and transmitting the sentiment score from the user device to the server.

[0045] The reaction data or at least a part of it may be acquired at and / or by the user device. Alternatively or additionally, the reaction data or at least a part of it may be acquired at and / or by at least one user-side device communicatively coupled or capable of being coupled to the user device. Alternatively or additionally, a part of the reaction data may be acquired at the user device and a further part of the reaction data may be acquired at the at least one user-side device. Various parts of the reaction data may be further merged and / or combined, for example, on the user device and / or on the at least one user-side device. It should be noted that alternatively or additionally, the reaction data may be completely acquired at the user device and / or on the user-side device. In other words, a complete set of reaction data may be acquired at the user device and / or on the user-side device. Thus, for example, it may be possible to perform a plausibility check of the reaction data based on comparing reaction data acquired with the user device with reaction data acquired at the user-side device.

[0046] The method may further include receiving a query at the user device, where the notification is provided by the server in response to the query, where the query may be received at the user device based on one or more user inputs, for example, via a user interface, a microphone, a camera, and / or a sensor of the user device. Alternatively, or additionally, the notification may be provided on the user device by the server in response to the query.

[0047] The method may further include transmitting the query received at the user device to a server, where transmitting the query may include transmitting the query to the server, for example in the form of one or more data packages. Optionally, transmitting the query to the server may include establishing a communication link or connection between the user device and the server.

[0048] The query may include text input received at the user device via a user interface, verbal input received at the user device via a microphone, and / or video-based input received via a camera on the user device. As noted above, the query may include one or more of at least one letter, at least one symbol, at least one number, text, a question, an answer to a question, a statement, a request, a voice, an audio message, a message, a text message, at least one icon, at least one image, and a video.

[0049] The notification may include one or more of at least one letter, at least one symbol, at least one number, text, voice, an audio message, a question, an answer to a question, a statement, a request, a message, a text message, at least one icon, an alarm, an audio alarm, an optical alarm, at least one image, and a video. However, it should be noted that the notification may be any other type of notification. The notification may include any suitable content.

[0050] The sentiment score may be an anonymized numerical measure (or score) indicative of a user's reaction to the notification. Alternatively, or in addition, the sentiment score may correlate with and / or be indicative of a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0051] Therein, the term "anonymized" may mean that the sentiment score does not contain, is devoid of, and / or is devoid of personal and / or private information about a user of a user device. In general, the sentiment score may be considered or may refer to a numerical measure of a user's reaction to a notification provided on a user device.

[0052] Further, the sentiment score may be measured on a scale that may be an absolute or relative value scale between a minimum sentiment score value and a maximum sentiment score value, and the actual value of the sentiment score may be indicative of and / or reflect the user's sentiment, current sentiment, and / or actual sentiment in response to the notification.

[0053] By way of example, the sentiment score may be measured on a scale between "-10" and "+10," where a sentiment score of "-10" may indicate a user's most negative reaction to the notification, a sentiment score of "0" may indicate a user's neutral reaction to the notification, and a sentiment score of "+10" may indicate a user's most positive reaction to the notification. However, it is emphasized that any other scale may be used for the sentiment score, including scales with non-integer values ​​for the sentiment score.

[0054] Determining the sentiment score determining an intermediate sentiment score based on the obtained reaction data; and anonymizing the intermediate sentiment score, thereby generating a sentiment score.

[0055] Therein, the intermediate sentiment score may include information or data related to the user's personal information and / or personal data, while the sentiment score may not contain or include the user's personal information and / or personal data. Thus, the intermediate sentiment score may be considered as an intermediate quantity or measure that may be determined, for example, temporarily determined, to determine the sentiment score. In general, the intermediate sentiment score may be determined and anonymized on the user device and / or on at least one user-side device.

[0056] In general, anonymizing the intermediate sentiment score to generate it as an anonymized score or measure advantageously allows protecting the privacy or anonymity of the interactive communication between the server and the user device. This may mean that the user or any personal information or personal data related to the user is not transmitted from the user device to the server. On the other hand, the (anonymized) sentiment score may be used to improve and / or personalize the interactive communication between the user device and the server, regardless of the anonymization, for example, based on training an artificial intelligence module and / or a reinforcement learning model on the server based on the sentiment score, even if data such as personal information or reaction data is transmitted to the server. Thus, determining the sentiment score based on anonymizing the intermediate sentiment score may allow selectively providing the server with information or data that may be useful for improving the overall interactive communication, without providing any personal information or personal data to the server.

[0057] Additionally, users' privacy concerns may be respected in a comprehensive manner, which may in turn reduce the likelihood of secure data relating to the users being compromised.

[0058] Furthermore, from the server's perspective, since none of the personal data or information may be processed by or stored on the server, the processing load and cost associated with security or safety-related measures may be effectively reduced. Therefore, any computing resources may not be required for additional safety measures on the server, which may lead to more efficient and quicker reaction of the server in response to queries from user devices. Also, the overall performance of the server may be improved, and the processing load and cost, e.g., hardware cost and / or maintenance cost, that may otherwise be required to provide comprehensive safety measures on the server may be effectively reduced.

[0059] Alternatively, by determining anonymized sentiment scores and transmitting the sentiment scores to a server to improve interactive communication, the amount or volume of data transmitted from the user device and / or user-side device to the server may be significantly reduced, for example, when compared to transmitting reaction data to the server and determining the sentiment scores on the server. This may further improve the overall performance of the server and improve the overall interactive communication between the server and the user device, for example, in terms of a more efficient and faster reaction of the server in response to queries from the user device.

[0060] The sentiment score may be anonymized based on normalizing the intermediate sentiment score with a reference sentiment score (and / or reference sentiment score value). Therein, for example, the reference sentiment score may be selected depending on the scale on which the sentiment score is determined. For example, the reference sentiment score may refer to and / or indicate the user's most negative reaction to the notification, the user's neutral reaction to the notification, the user's most positive reaction to the notification, or any reaction of the user in between. Furthermore, normalizing the intermediate sentiment score may include converting the intermediate sentiment score to a sentiment score based on the reference sentiment score. For example, the intermediate sentiment score may be divided by or multiplied with the reference sentiment score. However, any other mathematical operation may be applied. Furthermore, it is noted that the sentiment score may be given in absolute or relative values.

[0061] Optionally, further security measures and / or privacy protection schemes may be implemented on the user device and / or user-side device to protect the intermediate sentiment score and / or the sentiment score. For example, secret sharing, encryption, and / or homomorphic encryption may be used on the user device and / or user-side device.

[0062] The method may further include removing the intermediate sentiment score from the user device upon anonymizing the intermediate sentiment score and / or upon generating the sentiment score. Alternatively or additionally, the intermediate sentiment score may be removed from the at least one user-side device. By removing the intermediate sentiment score from the user device and / or the at least one user device, the user's private or personal information may be further protected. In which, removing the intermediate sentiment score may include deleting, overwriting, and / or destroying the intermediate sentiment score.

[0063] The method may further include removing the sentiment score from the user device when transmitting the sentiment score to the server. Alternatively or additionally, the sentiment score may be removed from at least one user-side device when transmitting the sentiment score to the server. In other words, for example, the transmission of the sentiment score from the user device and / or the at least one user-side device to the server may trigger the removal of the sentiment score from the user device and / or the user-side device. Thus, it may be ensured that no storage resources or memory are used or blocked by the sentiment score when the sentiment score is being transmitted to the server.

[0064] Providing the notification on the user device may include displaying the notification on a user interface of the user device. Displaying the notification on the user device may enable ensuring that the user's attention is directed to the notification. Alternatively or additionally, providing the notification on the user device may include playing or outputting the notification via a microphone.

[0065] Obtaining the reaction data may include obtaining the reaction data from a data storage device of the user device. Alternatively or additionally, obtaining the reaction data may include obtaining the reaction data from a data storage device or memory of one or more user-side devices. Alternatively or additionally, obtaining the reaction data may include requesting, by the user device, the reaction data to be transmitted from one or more user-side devices to the user device. Alternatively or additionally, obtaining the reaction data may include instructing, by the user device, one or more user-side devices to transmit the reaction data to the user device.

[0066] Obtaining the reaction data may include capturing sensor data using at least one sensor of the user device. In other words, the reaction data, or at least a portion of the reaction data, may be obtained based on sensor data of one or more sensors of the user device.

[0067] The method may further include deriving, with the user device, reaction data from the captured sensor data of at least one sensor of the user device. In which, deriving the reaction data may include processing the sensor data, for example, by control circuitry of the user device. Such processing may include one or more of filtering, transforming, and converting the captured sensor data, for example, to generate the reaction data. Alternatively or additionally, deriving the reaction data may include selecting at least a portion of the sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include using at least a portion of the sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include combining at least a portion of the sensor data with further sensor data of further sensors of the user device and / or with user-side sensor data of at least one user-side sensor.

[0068] The at least one sensor of the user device may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitive sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor. Such a sensor may advantageously enable detection or sensing of a user's reaction in response to a notification, and thus enable reliable determination of reaction data indicative of the user's reaction. However, it should be noted that any other type of sensor of the user device may be used to capture sensor data and / or to determine reaction data based thereon.

[0069] The reaction data may be obtained based on sensor data from multiple sensors of the user device, e.g., different types of sensors. Thus, obtaining the reaction data may include capturing sensor data using multiple sensors of the user device. In which, the sensor data of different sensors may be combined, merged, and / or fused to determine the reaction data. Thus, it may be possible to provide accurate and comprehensive reaction data that may accurately and comprehensively reflect the user's reaction to the notification and / or may contain comprehensive and accurate information regarding the user's reaction to the notification. Alternatively or additionally, using sensor data of multiple sensors to determine the reaction data may enable a plausibility check of the sensor data and / or the reaction data.

[0070] For example, first sensor data of a first sensor of a user device may be captured, and the first sensor data may be complemented, merged, combined, and / or fused with second sensor data of a second sensor of the user device to generate response data, where the first sensor may be different from the second sensor.

[0071] Obtaining the reaction data may include receiving user-side sensor data from at least one user-side sensor communicatively coupled to the user device by and / or in the user device. In other words, the reaction data or at least a part thereof may be obtained based on user-side sensor data of one or more user-side sensors. Therein, a "user-side sensor" may indicate or refer to a sensor of the user-side device. Furthermore, receiving the user-side sensor data may include obtaining the user-side sensor data from the at least one user-side sensor. Alternatively or additionally, receiving the user-side sensor data may include instructing the at least one user-side sensor to transmit the user-side sensor data to the user device.

[0072] In general, a user-side sensor may refer to a sensor located near, around, in the environment, and / or remotely from a user device. Alternatively or additionally, a user-side sensor may refer to a sensor coupled to, communicatively coupled to, capable of being coupled to, and / or configured to communicate with a user device. For example, a user-side sensor may be included in or part of a user-side device. Alternatively or additionally, a user-side sensor may be a stand-alone sensor. However, it should be noted that various or multiple different user-side sensors may be used to acquire reaction data, such as, for example, at least one user-side sensor included in at least one user-side device and at least one further user-side sensor that may be a stand-alone user-side sensor. When a user-side sensor is included in a user-side device, communication between the user device and the user-side sensor (or the user-side device) may be implemented via a communication circuit of the user-side device. If the user-side sensor is a standalone sensor, communication between the user-side sensor and the user device may be performed via corresponding communication circuitry of the user-side sensor, e.g., using a communication link, communication connection, and / or communication protocol, e.g., as described above with respect to the user-side device.

[0073] The method may further include deriving, with the user device, reaction data from the received user-side sensor data of the at least one user-side sensor. In which, deriving the reaction data may include, for example, processing the user-side sensor data by a control circuit of the user device. Such processing may include, for example, one or more of filtering, transforming, and converting the user-side sensor data to generate the reaction data. Alternatively or additionally, deriving the reaction data may include selecting at least a portion of the user-side sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include using at least a portion of the user-side sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include combining at least a portion of the user-side sensor data with further user-side sensor data of further user-side sensors and / or with sensor data of one or more sensors of the user device.

[0074] The at least one user-side sensor may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a near-field sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor. However, it should be noted that any other type of user-side sensor may be used.

[0075] The at least one user-side sensor may be included in at least one user-side device positioned in the vicinity of the user device, where the at least one user-side device is one or more of a smart television (television), a smart speaker, a smart watch, a health monitor, an IoT (Internet of Things) device, and an aerosol-generating device.

[0076] In the context of the present disclosure, an aerosol generating device may refer to a device that interacts with an aerosol-forming substrate to generate an aerosol. The aerosol-forming substrate may be part of an aerosol-generating article, for example, a smoking article. The aerosol generating device may include one or more components that are used to provide energy from a power source to the aerosol-forming substrate to generate an aerosol. The aerosol generating device and all its components may be portable and mobile. Therein, the heating element may be provided in different shapes, sizes, and numbers. For example, the heating element may be shaped as a needle, pin, rod, or blade that can be inserted into the smoking article to contact the aerosol-forming substrate. The aerosol generating device may comprise more than one heating element, and in the following description, reference to a heating element means one or more heating elements. The aerosol generating device may also comprise an electronic circuit arranged to control the supply of current to the heating element to control its temperature. The aerosol generating device may optionally include a means for sensing the temperature of the heating element.

[0077] The method may further include deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment that affects the user's sentiment. In which, a sentiment score may be determined based on the at least one environmental parameter. Generally, by determining the at least one environmental parameter and determining a sentiment score based thereon, one or more reactions of the user to the at least one environmental parameter, to the user's environment, and / or to the context of the user's environment may be differentiated and / or differentiated from the user's actual reaction in response to the notification. This may ensure that the sentiment score that may be used by the server to improve interactive communication preferably only reflects or at least mainly indicates the user's sentiment or reaction in direct consequence to the notification. Thus, it may be possible to improve interactive communication in a more precise, specific, targeted and goal-oriented manner. This may be particularly advantageous if the sentiment scores are used by the server to train the server's artificial intelligence modules and / or reinforcement learning models, since erroneous training may be effectively avoided or at least reduced to a minimum.

[0078] In the context of the present disclosure, at least one environmental parameter indicative of the user's environment affecting the user's sentiment may refer to any factor or situation that, for example, overlaps or interferes with the user's reaction to the notification, affects the user's sentiment, and / or (potentially) triggers the user's reaction. Examples of such environmental parameters may be the temperature in the user's environment, the noise in the user's environment, family or friends present in the user's environment, the user's location, etc.

[0079] In the following, an illustrative example for the determination of at least one environmental parameter and the associated benefit is provided. The sentiment score may be implicitly calculated from multiple sources, such as, for example, one or more sensors of the user device and / or one or more user-side sensors or devices, such as, for example, a smart TV, a thermostat, a GPS sensor, etc. Thus, it may be possible to validate the determined sentiment score in terms of ensuring that the sentiment score is at least primarily indicative of the user's reaction to the notification. This may, for example, allow for training a reinforcement learning model on the server based primarily or solely on reactions that are the result of the notification provided by the server and / or the determination of the reinforcement learning model to select a particular notification, as opposed to reactions based on external factors. For example, the server may provide a notification to the user device, and a negative sentiment or negative reaction from the user may be determined based on one or more sensor data, such as, for example, sensor data or image data of the camera of the user device. However, this negative sentiment or negative reaction of the user may be the result of low temperatures in the user's environment, noise in the user's environment, or any other environmental parameters that negatively affect the user's sentiment. Thus, the user's negative sentiment or negative reaction may be caused by one or more environmental parameters, rather than by the notification. By determining the sentiment score based on the determined at least one environmental parameter, it can be determined whether the sentiment score primarily reflects the user's sentiment or reaction in direct consequence to the notification, or whether the user's reaction or sentiment is primarily caused by one or more environmental parameters. Thus, the quality of the sentiment score can be improved by determining the sentiment score based on at least one environmental parameter.

[0080] The reaction data may be acquired by the user device based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor disposed in the environment of the user device. Using multiple sources to determine the sentiment score, such as one or more sensors of the user device and / or one or more user-side sensors, may enable checking the plausibility of the determined sentiment score. As a result, the quality of the sentiment score may be further improved.

[0081] Determining the sentiment score may include providing at least a portion of the reaction data to a classifier circuit of the user device. Alternatively, or in addition, the sentiment score may be determined based on classifying at least a portion of the reaction data with a classifier circuit of the user device. The classifier circuit may refer to, for example, an artificial intelligence module of the user device and / or a machine learning classifier, such as, for example, a neural network, that may enable determining a user's reaction to determine the sentiment score.

[0082] The method is: - capturing, with a camera of the user device, image data indicative of one or more images of a user of the user device; - determining response data based on the captured image data; and - determining a sentiment score based on providing at least a portion of the reaction data to a classifier circuit of the user device.

[0083] Thus, a camera of the user device may be used to obtain sensor data in the form of image data. However, the image data is not transmitted from the user device to the server, and instead, determining the sentiment score may mean that an anonymized sentiment score may be transmitted to the server without the image data leaving the user device. The reaction data may be determined based on the sensor data, for example, by selecting one or more images of the image data. Furthermore, the sentiment score may then be determined based on processing these reaction data. Therein, providing at least a portion of the reaction data to a classifier circuit may include providing at least a portion of the reaction data to the classifier circuit. The classifier circuit may reliably determine many different reactions of the user by the classifier circuit, so that the sentiment score may be determined quickly and accurately.

[0084] Determining the sentiment score determining, with a classifier circuitry of the user device, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - deriving a sentiment score from the determined response patterns.

[0085] In particular, the emotional response patterns may reflect the user's reaction to the notification, and thus, by using the classifier circuit to determine the response patterns, the sentiment score may be determined with high accuracy and precision.

[0086] The sentiment score may be determined based on computing and / or calculating a current sentiment score with the user device and deriving the sentiment score from the computed current sentiment score.

[0087] In the context of this disclosure, a "current sentiment score" may refer to a sentiment score that is determined or computed locally on a user device. For example, the current sentiment score may be determined based on sensor data of one or more sensors of the user device. In particular, the current sentiment score may be determined based solely on sensor data of one or more sensors of the user device.

[0088] Determining the sentiment score may include receiving, with the user device, at least one user-side sentiment score from at least one user-side device communicatively coupleable to and / or communicatively coupled to the user device, and the sentiment score may be determined based on the user-side sentiment score received from the at least one user-side device.

[0089] In the context of this disclosure, a "user-side sentiment score" may refer to a sentiment score that is determined or computed based on user-side sensor data of one or more user-side sensors. The user-side sentiment score may be determined by a user device and / or by one or more user-side devices based on user-side sensor data of one or more user-side sensors. For example, the user-side sentiment score may be determined based solely on user-side sensor data of one or more user-side sensors.

[0090] The sentiment score may be determined based on (a) a current sentiment score computed with the user device and (b) at least one user-side sentiment score received by the user device from at least one user-side device communicatively coupleable to the user device, or (b) at least one user-side sentiment score computed by the user device based on user-side sensor data received from at least one user-side device. In other words, the sentiment score may be determined based on multiple sentiment scores, i.e., based on the current sentiment score and based on at least one user-side sentiment score.

[0091] In other words, the sentiment score may be determined based on selecting either the current sentiment score or the at least one user-side sentiment score as the sentiment score. Alternatively, the current sentiment score and the at least one user-side sentiment score may be combined to determine the sentiment score. For example, a mean value or an average value of the current sentiment score and the at least one user-side sentiment score may be computed to determine the sentiment score.

[0092] Determining the sentiment score may include comparing the current sentiment score and at least one user-side sentiment score. By comparing the current sentiment score with one or more user-side sentiment scores, it may be determined whether the current sentiment score and the one or more user-side sentiment scores match and / or are consistent with each other. Thus, the sentiment score may be determined based on multiple sources. Thus, for example, it may be possible to verify the sentiment score that may be transmitted to the server to improve interactive communication, and in at least certain embodiments, may be used to train an artificial intelligence module of the server. Furthermore, it may be possible to perform a plausibility check of the current sentiment score and the user-side sentiment score.

[0093] Determining the sentiment score may include determining a deviation between the current sentiment score and at least one user-side sentiment score, and comparing the determined deviation to a threshold value for the deviation. By comparing the deviation to the threshold value, a match, consistency, inconsistency, and / or inconsistency between the current sentiment score and the at least one user-side sentiment score may be efficiently determined. Therein, the threshold value may be a predetermined threshold value. Optionally, the threshold value may be stored in a data storage device of the user device or may be obtained from another source, for example, from a server. Furthermore, it is noted that the threshold value for the deviation may refer to or indicate a range of the deviation.

[0094] The method may further include discarding and / or ignoring at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if the determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a threshold for deviation, where discarding the current sentiment score, the at least one user-side sentiment score, and the sentiment score may include, for example, removing and / or deleting the current sentiment score, the at least one user-side sentiment score, and the sentiment score from the user device and / or the at least one user-side device.

[0095] For this reason, it may be possible to ensure that only the sentiment score is used by the server to improve interactive communication, and the sentiment score is based on the current sentiment score and one or more user-side sentiment scores that are consistent and / or match each other. Meanwhile, the sentiment score determined based on the current sentiment score and one or more user-side sentiment scores that are different from another may be ignored and may not be used by the server. Overall, it may be possible to effectively improve interactive communication between the server and the user device, for example, by using the sentiment score to train an artificial intelligence module and / or a reinforcement learning model of the server.

[0096] The method may further include preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a threshold for deviation. Alternatively or additionally, the method may further include transmitting the sentiment score to the server only if the current sentiment score and the at least one user-side sentiment score substantially match each other. Thus, it may be ensured that the sentiment score is only transmitted to the server and / or used by the server to improve interactive communication if the sentiment score is based on the current sentiment score and the at least one user-side sentiment score that match each other and / or are consistent with each other.

[0097] In the context of the present disclosure, "substantially" matching (or differing) may mean that the respective quantities or measures, e.g., the relative deviation between the current sentiment score and at least one user-side sentiment score, are within a particular range. For example, substantially matching or differing may mean that the respective measures differ from each other by less than 40%, in particular less than 30%, less than 25%, less than 20%, less than 15%, or less than 10%.

[0098] The method may further include aggregating the plurality of sentiment scores, e.g., one or more of the current sentiment score and at least the user-side sentiment score, to determine a sentiment score for transmission to the server. Thus, the sentiment score for transmission to the server may refer to an integrated sentiment score determined based on the plurality of sentiment scores.

[0099] Aggregating multiple sentiment scores may include applying an aggregation function, such as, for example, a weighted average.

[0100] Additionally, the plurality of sentiment scores may be aggregated according to a deviation between at least two of the plurality of sentiment scores.

[0101] As an example, if the deviation between at least two sentiment scores reaches and / or exceeds a threshold for the deviation, different weightings may be applied to the different sentiment scores to determine the sentiment score for transmission to the server.

[0102] Alternatively or additionally, weighting may be applied depending on the source of the aggregated sentiment score, e.g., the user device and / or the user-side device. For example, a current sentiment score determined based on sensor data of a user device may have a higher weighting than one or more user-side sentiment scores determined based on user-side sensor data of one or more user-side sensors.

[0103] The method is: - preventing and / or delaying transmission of the sentiment score to the server if a determined deviation between the current sentiment score and at least one user-side sentiment score reaches and / or exceeds a deviation threshold value; receiving a further notification from the server to the user, the further notification substantially matching and / or equal to the notification; - obtaining further response data indicative of a further response of the user to the further notification; - determining a further sentiment score based on the obtained further reaction data, the further sentiment score indicating a further sentiment of the user in reaction to the further notification.

[0104] In other words, the transmission of the determined sentiment score to the server may be delayed, and the further sentiment score may be determined before transmitting the sentiment score to the server. Thus, it may be possible to use the further sentiment score to verify the sentiment score when a similar or identical reaction of the user, which is substantially consistent with each other, equal to each other, and / or identical to each other, should be triggered by the notification and the further notification. As a result, it may be possible to detect whether the sentiment score or the further sentiment score actually indicates the user's reaction to the notification and does not indicate the user's reaction to an environmental parameter or factor, such as, for example, low temperature or noise in the user's environment. Thus, the quality of the sentiment score may be improved and / or it may be ensured that the sentiment score at least mainly reflects and / or indicates the user's sentiment in the reaction to the notification. Furthermore, for example, if the sentiment score and the further sentiment score were different from each other, it may be possible to effectively improve interactive communication, since such sentiment scores may be effectively identified based on comparing the sentiment score and the further sentiment score, and thus erroneous sentiment scores may not be used by the server.

[0105] The method may further include comparing a further sentiment score determined based on further reaction data indicative of a further reaction of the user to the further notification to the sentiment score or the current sentiment score determined based on the reaction data indicative of the user's reaction to the notification, wherein the comparing may include determining a deviation between the sentiment score and the further sentiment score. Optionally, a deviation between the further sentiment score and the sentiment score or the current sentiment score may be determined.

[0106] The method may further include transmitting at least one of the sentiment score, the current sentiment score, and the further sentiment score to a server only if the further sentiment score, determined based on the further reaction data indicative of the user's further reaction to the further notification, substantially matches the sentiment score or the current sentiment score, determined based on the reaction data indicative of the user's reaction to the notification.

[0107] As an example, determining the sentiment score and the further sentiment score before transmitting the sentiment score to the server and comparing these sentiment scores may allow to avoid misinterpretation of the user's reaction. For example, the determined sentiment score may not be considered to be correlated with a negative reaction if the sentiment score and the further sentiment score do not match each other, e.g., one is positive and one is negative. Instead, a further notification, which may be identical or similar to the notification, may be provided on the user device, e.g., at a different time than the notification. The user's further reaction to the further notification and / or the further sentiment score may then make it possible to confirm that the sentiment score at least mainly indicates the user's reaction to the notification (and not to any environmental parameters) if the sentiment score and the further sentiment score substantially match each other. Thus, determining the sentiment score and the further sentiment score and comparing these sentiment scores may make it possible to confirm the correctness of the sentiment score and / or the further sentiment score. As a result, interactive communication can be effectively improved since only the confirmed sentiment scores can be used by the server, for example, to train an artificial intelligence module and / or a reinforcement learning model.

[0108] The method may further include determining a privacy level for at least one of the reaction data, the sensor data of one or more sensors of the user device, and the user-side sensor data of the one or more user-side sensors, the privacy level being indicative of one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data, which one or more features are to be manipulated in and / or removed from one or more of the reaction data, the sensor data, and the user-side sensor data to determine and / or before determining the sentiment score. Such manipulation may be performed on or by one or more of the user device, the one or more user-side sensors, and the one or more user-side devices.

[0109] The privacy level may be, for example, at least one of definable, user-settable, and programmable. For example, a user may define one or more privacy levels for one or more feature groups, such as, for example, "location, person, health, activity." Therein, the privacy level may be set, for example, to "low, medium, high." Furthermore, the feature group may define one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data.

[0110] Based on the defined privacy level for one or more of the feature groups, one or more processing operations for processing one or more of the reaction data, the sensor data, and the user-side sensor data may be selected by the user device for one or more features defined by the one or more feature groups.

[0111] The one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data may include and / or refer to a reaction feature and / or a reaction pattern of the user related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expressions, changes in the user's skin color, etc. Alternatively or additionally, the one or more features may include and / or refer to one or more environmental parameters that affect the user's sentiment, such as, for example, people in the user's vicinity, the location of the user and / or the user device, the temperature in the environment of the user device, etc.

[0112] The one or more processing operations selected for processing one or more of the response data, the sensor data, and the user-side sensor data may depend on a data type of the response data, the sensor data, and / or the user-side sensor data.

[0113] For example, when the response data, sensor data, and / or user-side sensor data include one or more images or image data, a processing operation associated with the privacy level may leave the image intact for a "low" privacy level, blur a person's face for a "medium" privacy level, and crop the person from the image for a "high" privacy level.

[0114] Similarly, when the response data, sensor data, and / or user-side sensor data includes text input or audio, for example, received via a user interface or via a microphone, depending on the privacy level set for a particular feature, the text input or audio input may be trimmed and certain features or information, such as name, date of birth, city, address, etc., may be removed from the response data, sensor data, and / or user-side sensor data.

[0115] According to a second aspect of the present disclosure, there is provided a computer program comprising instructions or software instructions, which when executed on one or more processors of a user device, cause the user device to perform a method according to the first aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the user device.

[0116] According to a third aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the second aspect of the present disclosure.

[0117] According to a fourth aspect of the present disclosure, a computer-implemented method for interactive communication of a user device with a server is provided. The method according to the fourth aspect may alternatively or additionally relate to a computer-implemented method for operating a user device, e.g., for interactive communication with a server, and optionally for operating one or more user-side devices. The method includes: - on the user device, providing one or more notifications to a user of the user device; obtaining reaction data indicative of one or more reactions of a user to one or more notifications; - determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; - determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores to one another, wherein the final sentiment score is usable by the server to train an artificial intelligence module and / or a reinforcement learning model implemented on the server.

[0118] In other words, a plurality of sentiment scores may be determined for at least one notification provided on a user device, and a final sentiment score may be determined based on a comparison of at least two of the sentiment scores. Also, by comparing at least two sentiment scores, for example, as discussed above and below with reference to the first aspect of the present disclosure, deviations, agreements, consistency, inconsistencies, and / or contradictions between the at least two sentiment scores may be determined. Thus, it may be possible to verify the determined sentiment scores and / or plausibility checks. Thus, it may be possible to ensure that only the (final) sentiment score is used by the server to improve interactive communication, and the (final) sentiment score is based on at least two sentiment scores, e.g., the current sentiment score and one or more user-side sentiment scores, that are in agreement and / or consistency with each other.

[0119] It should be noted that the term "final sentiment score" may refer to a sentiment score that can be transmitted to a server and used by the server to train an artificial intelligence module and / or a reinforcement learning module.

[0120] In general, the final sentiment score may refer to, among other things, a "sentiment score," as described with reference to the first aspect of the present disclosure. Furthermore, each of the "at least two sentiment scores" and / or each of the "plurality of sentiment scores" may refer to, for example, at least one of a current sentiment score and a user-side sentiment score, as described with reference to the first aspect of the present disclosure.

[0121] At least one notification may be provided on the user device. Further, a plurality of sentiment scores are determined. The plurality of sentiment scores may be determined based on the same source or based on different sources. For example, all or a subset of the plurality of sentiment scores may be determined using the user device, for example, based on sensor data of one or more sensors of the user device and / or based on user-side sensor data of one or more user-side sensors. Alternatively or additionally, all or a subset of the plurality of sentiment scores may be determined using one or more user-side devices, for example, based on user-side sensor data of the one or more user-side devices. Therein, at least one of the determined sentiment scores indicates a user's reaction to the at least one notification. One or more of the plurality of sentiment scores may indicate a user's reaction to at least one environmental parameter or factor. Alternatively, all of the plurality of sentiment scores may indicate a user's reaction to the at least one notification. Alternatively or additionally, at least a subset of the sentiment scores may indicate a user's reaction to different notifications, for example, notifications provided at different times.

[0122] For example, if the first sentiment score determined based on the user-side sensor data of at least one user-side device and the second sentiment score determined based on the sensor data of the user device do not match each other or do not substantially match each other, this may indicate that the user's reaction to at least one notification may be triggered or caused by one or more environmental parameters or factors, such as a low room temperature or noise in the user's environment. Thus, by comparing these two sentiment scores, it can be reliably determined whether the final sentiment score primarily or correctly indicates or reflects the user's sentiment in reaction to the notification, or whether the user's reaction was caused by one or more environmental parameters or factors that affect the user's sentiment.

[0123] Alternatively or additionally, for example, if a first sentiment score indicative of a user's first reaction to a first notification is determined and a second sentiment score indicative of a user's second reaction to a second notification is determined and do not match or do not substantially match each other, this may indicate that at least one of the user's first and second reactions may be triggered or caused by one or more environmental parameters or factors, such as a cold room temperature or a noise in the user's environment, in which the first and second notifications may be provided on the user device at different times, for example, before or after each other.

[0124] It may therefore be possible to verify the sentiment score and / or the final sentiment score based on comparing at least two different sentiment scores, where the two different sentiment scores may be based on different sensor data and / or reaction data, such as, for example, user-side sensor data of at least one user-side sensor and / or sensor data of at least one sensor of the user device. Alternatively or additionally, the at least two sentiment scores may indicate different or multiple reactions of the user in response to multiple notifications, for example, provided at different times. Comparing the at least two sentiment scores may enable reliable determination of whether the at least two sentiment scores are consistent with each other, which allows comprehensive verification of the determined sentiment scores and / or plausibility checks. It may therefore be possible to provide a final sentiment score that can be used to train an artificial intelligence module and / or a reinforcement learning model on the server that accurately reflects or indicates the user's sentiment in reaction to the at least one notification.

[0125] It is emphasized that any feature, step, element and / or example described herein above and below with reference to one aspect of the present disclosure applies to any other aspect of the present disclosure, and vice versa. In particular, any embodiment of the first aspect may be combined with any embodiment of the fourth aspect, and vice versa.

[0126] At least one of the plurality of sentiment scores may be indicative of the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that influences the user's sentiment.

[0127] At least two of the plurality of sentiment scores may indicate a user's sentiment in the user's reaction to at least two notifications received from the server, for example, at different times.

[0128] The final sentiment score may be determined based on selecting at least one of the plurality of sentiment scores as the final sentiment score. Alternatively or additionally, the plurality of sentiment scores, e.g., the at least two sentiment scores, may be combined to determine the final sentiment score. For example, the final sentiment score may be determined as a mean value or average value of the at least two sentiment scores.

[0129] The method may further include receiving, with the user device, at least one of the plurality of sentiment scores from at least one user-side device communicatively coupleable and / or communicatively coupled to the user device. In other words, at least one of the plurality of sentiment scores may be determined at one or more user-side devices and transmitted to the user device. In which, receiving the sentiment score may include obtaining the sentiment score from the at least one user-side device and / or instructing the at least one user-side device to transmit or send the sentiment score to the user device.

[0130] Obtaining the reaction data may include capturing sensor data using at least one sensor of the user device. In other words, the reaction data, or at least a portion of the reaction data, may be obtained based on sensor data of one or more sensors of the user device.

[0131] The method may further include deriving, with the user device, reaction data from the captured sensor data of at least one sensor of the user device. In which, deriving the reaction data may include processing the sensor data, for example, by control circuitry of the user device. Such processing may include one or more of filtering, transforming, and converting the captured sensor data, for example, to generate the reaction data. Alternatively or additionally, deriving the reaction data may include selecting at least a portion of the sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include using at least a portion of the sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include combining at least a portion of the sensor data with further sensor data of further sensors of the user device and / or with user-side sensor data of at least one user-side sensor.

[0132] The at least one sensor of the user device may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitive sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor. Such a sensor may advantageously enable reliable detection or sensing of a user's reaction in response to a notification, and consequently, reliable determination of reaction data indicative of the user's reaction. However, it should be noted that any other type of sensor of the user device may be used to capture sensor data and / or to determine reaction data based thereon.

[0133] The reaction data may be obtained based on sensor data from multiple sensors of the user device, e.g., different types of sensors. Thus, obtaining the reaction data may include capturing sensor data using multiple sensors of the user device. In which, the sensor data of different sensors may be combined, merged, and / or fused to determine the reaction data. Thus, it may be possible to provide accurate and comprehensive reaction data that may accurately and comprehensively reflect the user's reaction to one or more notifications, or may contain comprehensive and accurate information regarding the user's reaction to one or more notifications. Alternatively or additionally, using sensor data of multiple sensors to determine the reaction data may enable a check of the plausibility of all sensor data and / or reaction data.

[0134] For example, first sensor data of a first sensor of a user device may be captured, and the first sensor data may be complemented, merged, combined, and / or fused with second sensor data of a second sensor of the user device to generate response data, where the first sensor may be different from the second sensor.

[0135] Obtaining the reaction data may include receiving user-side sensor data from at least one user-side sensor communicatively coupled to the user device by and / or in the user device. In other words, the reaction data, or at least a portion thereof, may be obtained based on user-side sensor data of one or more user-side sensors.

[0136] A "user-side sensor" may indicate or refer to a sensor of a user-side device. Furthermore, receiving user-side sensor data may include obtaining user-side sensor data from at least one user-side sensor. Alternatively or additionally, receiving user-side sensor data may include instructing at least one user-side sensor to transmit the user-side sensor data to a user device.

[0137] As described herein above with reference to the first aspect of the present disclosure, a user-side sensor may refer to a sensor located near, around, in the environment, and / or remotely from a user device. Alternatively or additionally, a user-side sensor may refer to a sensor coupled to, communicatively coupled to, capable of being coupled to, and / or configured to communicate with a user device via a common communication network, e.g., the same local area network (LAN) to which the user device is connected. For example, a user-side sensor may be included in or part of a user-side device. Alternatively or additionally, a user-side sensor may be a stand-alone sensor. However, it should be noted that various or multiple different user-side sensors may be used to acquire reaction data, such as, for example, at least one user-side sensor included in at least one user-side device and at least one further user-side sensor that may be a stand-alone user-side sensor. When a user-side sensor is included in a user-side device, communication between the user device and the user-side sensor (or the user-side device) may be implemented via a communication circuit of the user-side device. If the user-side sensor is a standalone sensor, communication between the user-side sensor and the user device may be performed via corresponding communication circuitry of the user-side sensor, e.g., using a communication link, communication connection, and / or communication protocol, e.g., as described above with respect to the user-side device.

[0138] The method may further include deriving, with the user device, reaction data from the received user-side sensor data of the at least one user-side sensor. In which, deriving the reaction data may include, for example, processing the user-side sensor data by a control circuit of the user device. Such processing may include, for example, one or more of filtering, transforming, and converting the user-side sensor data to generate the reaction data. Alternatively or additionally, deriving the reaction data may include selecting at least a portion of the user-side sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include using at least a portion of the user-side sensor data as reaction data. Alternatively or additionally, deriving the reaction data may include combining at least a portion of the user-side sensor data with further user-side sensor data of further user-side sensors and / or with sensor data of one or more sensors of the user device.

[0139] The at least one user-side sensor may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a near-field sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor. However, it should be noted that any other type of user-side sensor may be used.

[0140] The at least one user-side sensor may be included in at least one user-side device positioned in the vicinity of the user device, where the at least one user-side device is one or more of a smart television (television), a smart speaker, a smart watch, a health monitor, an IoT (Internet of Things) device, and an aerosol-generating device.

[0141] The method may further include: receiving, with the user device, user-side sensor data from the at least one user-side device; and computing at least one of the plurality of sentiment scores based on deriving, with the user device, at least a portion of the reaction data and / or further reaction data from the received user-side sensor data. In other words, the user-side sensor data may be obtained from at least one user-side sensor, and the further reaction data may be derived therefrom by the user device to determine one or more of the plurality of sentiment scores.

[0142] The method may further include receiving at least one query at the user device, where the at least one notification is provided by the server in response to the at least one query, where the at least one query may be received at the user device based on one or more user inputs, for example, via a user interface, a microphone, a camera, and / or a sensor of the user device. Alternatively, or additionally, the one or more notifications may be provided on the user device by the server in response to the at least one query.

[0143] The method may further include transmitting the at least one query received at the user device to a server, where transmitting the at least one query may include transmitting the at least one query to the server, for example in the form of one or more data packages. Optionally, transmitting the at least one query to the server may include establishing a communication link or connection between the user device and the server.

[0144] The at least one query may include text input received at the user device via a user interface, verbal input received at the user device via a microphone, and / or video-based input received via a camera of the user device. As noted above, the at least one query may include one or more of at least one letter, at least one symbol, at least one number, text, a question, an answer to a question, a statement, a request, a voice, an audio message, a message, a text message, at least one icon, an alert, an audio alert, an optical alert, at least one image, and a video.

[0145] Each of the one or more notifications may include one or more of at least one letter, at least one symbol, at least one number, text, voice, an audio message, a question, an answer to a question, a statement, a request, a message, a text message, at least one icon, an alarm, an audio alarm, an optical alarm, at least one image, and a video. However, it should be noted that any other type of notification or content of any notification is contemplated.

[0146] The determined final sentiment score may be transmitted to a server, where the final sentiment score may be transmitted to the server from the user device and / or from one or more user-side devices.

[0147] The method may further include displaying one or more notifications, for example on a user interface of the user device. Alternatively, or in addition, one or more of the notifications may be played or output on a microphone of the user device.

[0148] Determining the final sentiment score determining an intermediate sentiment score based on the obtained reaction data; and anonymizing the intermediate sentiment score, thereby generating a final sentiment score.

[0149] As described with reference to the first aspect of the present disclosure, the intermediate sentiment score may include information or data related to the user's personal information and / or personal data, whereas the final sentiment score may not contain or include the user's personal information and / or personal data. Thus, the intermediate sentiment score may be considered as an intermediate quantity or measure that may be determined, e.g., temporarily determined, to determine the final sentiment score. In general, the intermediate sentiment score may be determined and anonymized on the user device and / or on at least one user-side device. In general, anonymizing the intermediate sentiment score to generate it as an anonymized score or measure advantageously allows for protecting the privacy or anonymity of the interactive communication between the server and the user device, as described herein above with respect to the first aspect of the present disclosure.

[0150] The final sentiment score may be anonymized based on normalizing the intermediate sentiment score with a reference sentiment score (and / or reference sentiment score value). Therein, for example, the reference sentiment score may be selected depending on the scale on which the sentiment score is determined. For example, the reference sentiment score may refer to and / or indicate the user's most negative reaction to the notification, the user's neutral reaction to the notification, the user's most positive reaction to the notification, or any reaction of the user in between. Furthermore, normalizing the intermediate sentiment score may include converting the intermediate sentiment score to a final sentiment score based on the reference sentiment score. For example, the intermediate sentiment score may be divided by or multiplied with the reference sentiment score. However, any other mathematical operation may be applied. Furthermore, it is noted that the final sentiment score may be given in absolute or relative values.

[0151] The method may further include removing the intermediate sentiment scores from the user devices when anonymizing the intermediate sentiment scores and / or when generating the final sentiment scores. Alternatively or additionally, the intermediate sentiment scores may be removed from at least one user-side device. By removing the intermediate sentiment scores from the user devices and / or the at least one user device, the user's private or personal information may be further protected. In which, removing the intermediate sentiment scores may include deleting, overwriting, and / or destroying the intermediate sentiment scores.

[0152] The method may further include removing the final sentiment score from the user device when transmitting the final sentiment score to the server. Alternatively or additionally, the final sentiment score may be removed from the at least one user-side device when transmitting the final sentiment score to the server. In other words, for example, the transmission of the final sentiment score from the user device and / or the at least one user-side device to the server may trigger the removal of the final sentiment score from the user device and / or the user-side device. In this way, it may be ensured that no storage resources or memory are used or blocked by the final sentiment score when it is being transmitted to the server.

[0153] Providing one or more notifications may include receiving a first notification and a second notification at the user device, where the second notification substantially matches and / or is equal to the first notification. In which, obtaining the reaction data may include obtaining first reaction data indicative of a first reaction of the user to the first notification, and obtaining second reaction data indicative of a second reaction of the user to the second notification. Further, determining the plurality of sentiment scores may include determining a first sentiment score based on the first reaction data, and determining a second sentiment score based on the second reaction data. The first and second reaction data may be determined based on at least one of sensor data of one or more sensors of the user device and user-side sensor data of one or more user-side sensors. By determining independent or separate sentiment scores, e.g., the first and second sentiment scores, based on independent, separate, and / or different notifications, e.g., the first and second notifications that substantially match each other, it can be ensured that the final sentiment score primarily or correctly indicates the user's sentiment in response to one of the notifications. As a result, the interactive communication can be accurately improved and / or personalized, e.g., based on training an artificial intelligence module and / or a reinforcement learning model on a server. For example, the final sentiment score can be determined based on the first and second sentiment scores.

[0154] Determining the final sentiment score may include determining a deviation between at least two of the multiple sentiment scores, e.g., a first sentiment score and a second sentiment score, and comparing the determined deviation to a threshold for the deviation. The threshold may be predefined and / or may be stored in a data storage or memory of the user device. Alternatively or additionally, the threshold may be obtained from another source, e.g., a server. Furthermore, the threshold for the deviation may refer to a range of deviation, e.g., a percentage of relative deviation between the at least two sentiment scores.

[0155] The method may further include discarding and / or ignoring at least one of the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, if a determined deviation between the at least two sentiment scores, e.g., between the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold value for the deviation.

[0156] The method may further include preventing the final sentiment score from being transmitted to the server if a determined deviation between the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold for deviation. Alternatively, or additionally, the method may further include transmitting the final sentiment score to the server only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0157] The method may further include aggregating the multiple sentiment scores to determine a final sentiment score for transmission to the server. Thus, the final sentiment score for transmission to the server may refer to a consolidated sentiment score determined based on the multiple sentiment scores.

[0158] Aggregating multiple sentiment scores may include applying an aggregation function, such as, for example, a weighted average.

[0159] Additionally, the plurality of sentiment scores may be aggregated according to a deviation between at least two of the plurality of sentiment scores.

[0160] As an example, if the deviation between at least two sentiment scores reaches and / or exceeds a threshold for the deviation, different weightings may be applied to the different sentiment scores to determine the final sentiment score.

[0161] Alternatively or additionally, weighting may be applied depending on the source of the aggregated sentiment score, e.g., the user device and / or the user-side device. For example, a sentiment score determined based on sensor data of a user device may have a higher weighting than one or more (user-side) sentiment scores determined based on user-side sensor data of one or more user-side sensors.

[0162] The method may further include determining a privacy level for at least one of the reaction data, the sensor data of one or more sensors of the user device, and the user-side sensor data of the one or more user-side sensors, the privacy level being indicative of one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data, which one or more features are to be manipulated in and / or removed from one or more of the reaction data, the sensor data, and the user-side sensor data to determine and / or before determining the sentiment score. Such manipulation may be performed on or by one or more of the user device, the one or more user-side sensors, and the one or more user-side devices.

[0163] The privacy level may be, for example, at least one of definable, user-settable, and programmable. For example, a user may define one or more privacy levels for one or more feature groups, such as, for example, "location, person, health, activity." Therein, the privacy level may be set, for example, to "low, medium, high." Furthermore, the feature group may define one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data.

[0164] Based on the defined privacy level for one or more of the feature groups, one or more processing operations for processing one or more of the reaction data, the sensor data, and the user-side sensor data may be selected by the user device for one or more features defined by the one or more feature groups.

[0165] The one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data may include and / or refer to a reaction feature and / or a reaction pattern of the user related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expression, etc. Alternatively, or additionally, the one or more features may include and / or refer to one or more environmental parameters that affect the user's sentiment, such as, for example, people in the user's vicinity, the location of the user and / or the user device, the temperature in the environment of the user device, etc.

[0166] The one or more processing operations selected for processing one or more of the response data, the sensor data, and the user-side sensor data may depend on a data type of the response data, the sensor data, and / or the user-side sensor data.

[0167] For example, when the response data, sensor data, and / or user-side sensor data include one or more images or image data, a processing operation associated with the privacy level may leave the image intact for a "low" privacy level, blur a person's face for a "medium" privacy level, and crop the person from the image for a "high" privacy level.

[0168] Similarly, when the response data, sensor data, and / or user-side sensor data includes text input or audio, for example, received via a user interface or via a microphone, depending on the privacy level set for a particular feature, the text input or audio input may be trimmed and certain features or information, such as name, date of birth, city, address, etc., may be removed from the response data, sensor data, and / or user-side sensor data.

[0169] According to a fifth aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which when executed on one or more processors of a user device, cause the user device to perform a method according to the fourth aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the user device.

[0170] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the fifth aspect of the present disclosure.

[0171] According to a seventh aspect of the present disclosure, a computer-implemented method for interactive communication of a server with a user device is provided. The method according to the seventh aspect may alternatively or additionally relate to a computer-implemented method for operating a server, e.g., for interactive communication with a user device, and optionally for operating one or more user-side devices. The method includes: transmitting a notification from the server to the user device; receiving, by a server, a sentiment score, the sentiment score being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on the received sentiment scores.

[0172] Therein, the sentiment score may be received from the user device and / or from one or more user-side devices. Further, receiving the sentiment score may include obtaining the sentiment score from the user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or at least one user-side device to transmit or send the sentiment score to a server.

[0173] In general, the method according to the seventh aspect may correspond to one or more of the methods of the first and fourth aspects of the present disclosure, but from the perspective of the server, or at least primarily from the perspective of the server. Any of the disclosures, embodiments and examples described herein above and below with reference to any aspect of the present disclosure may equally apply to and be combined with the method according to the seventh aspect of the present disclosure, and vice versa.

[0174] The method may further include selecting, from the knowledge base of the server, a further notification to be transmitted to the user device based on the received sentiment score. The further notification may be selected, for example, upon or in response to receipt at the server of one or more queries from the user device.

[0175] The server may comprise, for example, a natural language processing ("NLP") module or engine configured to process, analyze, and / or resolve one or more queries received from the user device. Additionally, the server's knowledge base may include a plurality of notifications transmittable to the user device, and the server may be configured to determine, select, identify, and / or generate further notifications from among or based on the plurality of notifications, for example, based on or using the server's artificial intelligence module, recommendation engine, and / or reinforcement learning model.

[0176] The reinforcement learning model may be trained based on maximizing and / or optimizing a reward function of the reinforcement learning model implemented on the server.

[0177] Thus, it may be possible to improve the interactive communication and / or reinforcement learning model over time, for example, continuously or iteratively over multiple training steps, in each of which reinforcement learning rewards reflecting and / or indicative of the level of correctness or quality of decisions of the implemented reinforcement learning model may be provided or dispensed to the reinforcement learning model to penalize it when it makes an incorrect decision and reward it when it makes a correct decision.

[0178] The method further includes receiving at least one query from the user device, and the one or more notifications are transmitted from the server to the user device upon receiving the at least one query from the user device.

[0179] The method further includes receiving a further query from the user device, and the further notification is selected based on the received sentiment score and based on the received further query. By the sentiment score, the reinforcement learning model can be precisely trained so that the further notification is more appropriate for the further query, thereby improving the interactive communication. Alternatively, or additionally, the interactive communication can be personalized based on or using the sentiment score to reflect or indicate the user's sentiment in reaction to the notification, where the sentiment score can be provided, for example, earlier than the further notification during the interactive communication.

[0180] The method may further include transmitting a further notification to the user device in response to the further query.

[0181] The method may further include processing the query and / or the further query using a natural language processing engine implemented on the server.

[0182] The method may further include receiving at least one further sentiment score and selecting from the server's knowledge base a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score, wherein both the sentiment score and the further sentiment may be used by the server to train a reinforcement learning model of the server. Alternatively or additionally, the sentiment score and the further sentiment score may be used to validate one or both of the sentiment score and the further sentiment score, for example, as described with reference to the first and fourth aspects of the present disclosure.

[0183] The at least one further sentiment score may be received by the server, from the user device, and / or from the user-side device, where receiving the further sentiment score may include obtaining the further sentiment score from the user device and / or from the at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or the at least one user-side device to transmit or send the sentiment score to the server.

[0184] The method may further include comparing the sentiment score with at least one further sentiment score. Based on such comparison, the sentiment score and / or the further sentiment may be verified. Thus, it may be possible to effectively improve the reinforcement learning model or its training.

[0185] The method may further include determining a deviation of the sentiment score and the at least one further sentiment score, and comparing the determined deviation to a threshold for the deviation. The threshold may be predefined and / or may be stored in a data store or memory of the server. Alternatively or additionally, the threshold may be obtained from another source, e.g., a user device. Furthermore, the threshold for the deviation may refer to a range of deviation, e.g., a percentage of relative deviation between the sentiment score and the at least one further sentiment score.

[0186] The method may further include discarding at least one of the sentiment score and the at least one further sentiment score if a determined deviation between the sentiment score and the at least one further sentiment score reaches and / or exceeds a threshold value for deviation.

[0187] The method may further include training a reinforcement learning model implemented on the server based on the sentiment score and / or the at least one further sentiment score only if the sentiment score and the at least one further sentiment score substantially match one another.

[0188] According to an eighth aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which when the program is executed on one or more processors of a server, cause the server to perform the method according to the seventh aspect of the present disclosure. The computer program may for example be stored on a data storage device or memory of the server.

[0189] According to a ninth aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the ninth aspect of the present disclosure.

[0190] According to a tenth aspect of the present disclosure, a computer-implemented method for interactive communication of a server with a user device is provided. The method according to the tenth aspect may alternatively or additionally relate to a computer-implemented method for operating a server, e.g., for interactive communication with a user device, and optionally for operating one or more user-side devices. The method includes: transmitting one or more notifications from the server to the user device; receiving, by a server, a plurality of sentiment scores, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores.

[0191] Therein, each of the sentiment scores may be received from a user device and / or from one or more user-side devices. Further, receiving the sentiment scores may include obtaining the sentiment scores from the user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment scores may include instructing the user device and / or at least one user-side device to transmit or send the sentiment scores to a server.

[0192] In general, the method according to the tenth aspect may correspond to one or more of the methods of the first, fourth and seventh aspects of the present disclosure, but from the perspective of the server, or at least primarily from the perspective of the server. Any of the disclosures, embodiments and examples described herein above and below with reference to any aspect of the present disclosure may equally apply to and be combined with the method according to the tenth aspect of the present disclosure, and vice versa.

[0193] Thus, at least two sentiment scores, e.g., a first sentiment score and a second sentiment score, may be determined, and each of the first sentiment score and the second sentiment score may be correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server. Further, the reinforcement learning model implemented on the server may be trained based on a comparison of the first sentiment score and the second sentiment score.

[0194] The multiple sentiment scores, for example the first and / or second sentiment scores, may be received by the server, from the user device, or from a user-side device.

[0195] At least one of the plurality of sentiment scores, e.g., the first and / or second sentiment scores, may indicate a user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment. Alternatively or additionally, at least one of the plurality of sentiment scores, e.g., the first and / or second sentiment scores, may indicate a user's sentiment in response to one or more notifications, e.g., at least one of the first and / or second notifications, received by the user device from the server.

[0196] Transmitting one or more notifications from the server to the user device may include transmitting a first notification and a second notification from the server to the user device, the second notification substantially matching and / or equal to the first notification. In which, receiving a plurality of sentiment scores may include receiving a first sentiment score in response to transmitting the first notification to the user device, and receiving a second sentiment score in response to transmitting the second notification to the user device. The first and second notifications may be transmitted, for example, before or after and / or at different times. For this reason, as described herein above, it may be possible to validate the first and / or second sentiment scores and, as a result, effectively train a reinforcement learning model.

[0197] The method may further include determining a deviation between at least two of the plurality of sentiment scores, e.g., between a first sentiment score and a second sentiment score, and comparing the determined deviation to a threshold value for the deviation. The threshold value may be predefined and / or may be stored in a data store or memory of the server. Alternatively or additionally, the threshold value may be obtained from another source, e.g., a user device. Furthermore, the threshold value for the deviation may refer to a range of deviation, e.g., a percentage of relative deviation between the sentiment scores and the at least one.

[0198] The method may further include discarding and / or ignoring at least one of the plurality of sentiment scores, e.g., the first sentiment score and / or the second sentiment score, if a determined deviation between at least two of the plurality of sentiment scores, e.g., between the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold value for deviation.

[0199] The method may further include training a reinforcement learning model implemented on the server based on at least one of the received plurality of sentiment scores, e.g., based on at least one of the first sentiment score and the second sentiment score, only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0200] Training a reinforcement learning model is - determining a final sentiment score based on at least one of the plurality of sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score; - deriving a reinforcement learning reward for training a reinforcement learning model from the determined final sentiment scores and / or computing a reinforcement learning reward based on the final sentiment scores; - feeding the reinforcement learning reward to the reinforcement learning model.

[0201] Therein, the final sentiment score may be determined based on selecting at least one of the multiple sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score, as the final sentiment score. Alternatively, or additionally, the multiple sentiment scores, e.g., the first and second sentiment scores, may be combined to determine the final sentiment score. For example, the final sentiment score may be determined as a mean value or average value of the multiple sentiment scores, e.g., the first and second sentiment scores.

[0202] The method may further include aggregating the multiple sentiment scores to determine a final sentiment score for transmission to the server. Thus, the final sentiment score for transmission to the server may refer to a consolidated sentiment score determined based on the multiple sentiment scores.

[0203] Aggregating multiple sentiment scores may include applying an aggregation function, such as, for example, a weighted average.

[0204] Additionally, the plurality of sentiment scores may be aggregated according to a deviation between at least two of the plurality of sentiment scores.

[0205] As an example, if the deviation between at least two sentiment scores reaches and / or exceeds a threshold for the deviation, different weightings may be applied to the different sentiment scores to determine the final sentiment score.

[0206] Alternatively or additionally, weighting may be applied depending on the source of the aggregated sentiment score, e.g., the user device and / or the user-side device. For example, a sentiment score determined based on sensor data of a user device may have a higher weighting than one or more (user-side) sentiment scores determined based on user-side sensor data of one or more user-side sensors. A reinforcement learning reward may refer to a reinforcement learning reward value. A reinforcement learning reward may be determined, calculated, and / or computed, for example, based on at least the final sentiment score and a reward function of a reinforcement learning model.

[0207] Determining and / or computing the reinforcement learning reward may further include determining a trend of a plurality and / or series of previous sentiment scores, e.g., a plurality and / or series of previously determined final sentiment scores, determined at least in part based on one or more notifications provided to a user. Thus, a plurality and / or series of previous sentiment scores may refer to a series of past sentiment scores, e.g., past final sentiment scores, determined at least in part based on one or more notifications provided to a user device.

[0208] The method may further include determining a weighting based on the determined trends of the plurality and / or sequence of previous sentiment scores, and determining a reinforcement learning reward based on the determined weighting and the final sentiment score.

[0209] By determining a trend and determining a reinforcement learning reward based thereon, it may be possible, for example, to put the user's current sentiment in reaction to a notification against the historical background of ongoing or recurring interactive communication between the user device and a server, such that the reinforcement learning reward may accurately reflect the user's current sentiment.

[0210] The method may further include comparing the final sentiment score to a trend of a plurality and / or series of previous sentiment scores and determining a reinforcement learning reward based on the comparison.

[0211] The method may further include determining the reinforcement learning reward based on one or more previous sentiment scores, e.g., one or more previous reinforcement learning rewards determined based on one or more previous final sentiment scores.

[0212] According to an eleventh aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which when the program is executed on one or more processors of a server, cause the server to perform the method according to the tenth aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the server.

[0213] According to a twelfth aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the eleventh aspect of the present disclosure.

[0214] According to a thirteenth aspect of the present disclosure, a computer-implemented method for interactive communication between a user device and a server is provided. The method according to the thirteenth aspect may alternatively or additionally relate to a computer-implemented method for operating a system for interactive communication, the system including at least one server and at least one user device. Optionally, the system may include one or more user-side devices and / or one or more user-side sensors. The method includes: transmitting a notification from the server to the user device; - on the user device, providing a notification to a user of the user device; - obtaining, by means of the user device and / or by means of at least one user-side device, reaction data indicative of a reaction of the user to the notification; - determining, using the user device and / or using at least one user-side device, a sentiment score based on the acquired reaction data, the sentiment score indicating a user's sentiment in reaction to the notification, the sentiment score correlating with a reinforcement learning reward for training a reinforcement learning model implemented on a server; - transmitting the determined sentiment scores from the user device and / or from at least one user-side device to a server; receiving, by a server, a sentiment score; - training an enrichment model implemented on the server based on the sentiment scores received by the server.

[0215] In which, receiving the sentiment score may include obtaining the sentiment score from the user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or the at least one user-side device to transmit or send the sentiment score to a server.

[0216] In general, the method according to the thirteenth aspect may correspond to one or more of the methods of the first, fourth, seventh and tenth aspects of the present disclosure, but from a systems perspective, or at least primarily from a systems perspective. Any of the disclosures, embodiments and examples described herein above and below with reference to any aspect of the present disclosure apply equally to and may be combined with the method according to the thirteenth aspect of the present disclosure, and vice versa.

[0217] According to a fourteenth aspect of the present disclosure, a computer-implemented method for interactive communication between a user device and a server is provided. The method according to the fourteenth aspect may alternatively or additionally relate to a computer-implemented method for operating a system for interactive communication, the system including at least one server and at least one user device. Optionally, the system may include one or more user-side devices and / or one or more user-side sensors. The method includes: transmitting one or more notifications from the server to the user device; - on the user device, providing one or more notifications to a user of the user device; - obtaining, using the user device and / or using at least one user-side device, reaction data indicative of one or more reactions of a user to the one or more notifications; - determining, using the user device and / or using at least one user-side device, a plurality of sentiment scores based on the acquired reaction data, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on a server; comparing at least two of the determined sentiment scores to one another; transmitting at least one of the sentiment scores from the user device and / or from at least one user-side device to a server; receiving, by a server, at least one of a plurality of sentiment scores; training an enrichment model implemented on the server based on at least one of the plurality of sentiment scores received by the server.

[0218] In which, receiving the at least one sentiment score may include obtaining the sentiment score from a user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or the at least one user-side device to transmit or send the sentiment score to a server.

[0219] In general, the method according to the fourteenth aspect may correspond to one or more of the methods of the first, fourth, seventh and tenth aspects of the present disclosure, but from a systems perspective, or at least primarily from a systems perspective. Any of the disclosures, embodiments and examples described herein above and below with reference to any aspect of the present disclosure may equally apply to and be combined with the method according to the fourteenth aspect of the present disclosure, and vice versa.

[0220] According to a fifteenth aspect of the present disclosure, a computer program is provided that includes instructions and / or software instructions, which when executed on one or more processors of a system for interactive communication, cause the system to perform a method according to one or both of the thirteenth and fourteenth aspects of the present disclosure. The computer program may be stored, for example, on a data storage device or memory of a server, a user device, and / or one or more user-side devices.

[0221] According to a sixteenth aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the fifteenth aspect of the present disclosure.

[0222] According to a seventeenth aspect of the present disclosure, a user device configured to interactively communicate with a server is provided. The user device may be configured to perform one or more steps of a method according to, for example, any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The user device comprises a communication circuit communicatively coupling the user device to the server and configured to receive notifications from the server. The user device further comprises a user interface configured to provide notifications to a user of the user device, and a control circuit including one or more processors, the control circuitry: - obtaining reaction data indicative of a user's reaction to the notification; - determining a sentiment score for transmission to a server based on the obtained reaction data, the sentiment score indicating a sentiment of the user in reaction to the notification.

[0223] The communication circuitry of the user may be configured to be coupled with a corresponding communication arrangement of the server to enable data exchange between the user device and the server. As an example, the user device may be coupled and / or configured to communicate with the server (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0224] The control circuitry of a user device may be referred to as a controller, a control module, and / or a control arrangement.

[0225] Further, the user device may comprise a data storage or memory for storing, for example, sentiment scores, notifications, and / or reaction data. Alternatively, or in addition, computer programs, instructions, apps, applications, and / or software instructions may be stored in the data storage or memory.

[0226] The control circuitry may be further configured to transmit the determined sentiment score to a server via the communication circuitry of the user device.

[0227] The sentiment score may be an anonymized numerical measure indicative of a user's reaction to the notification. Alternatively, or in addition, the sentiment score may correlate with and / or be indicative of a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0228] The control circuit includes: determining an intermediate sentiment score based on the obtained reaction data; and anonymizing the intermediate sentiment score to generate a sentiment score.

[0229] The control circuitry may be configured to anonymize the sentiment score based on normalizing the intermediate sentiment score with a reference sentiment score.

[0230] The control circuitry may be configured to remove the intermediate sentiment scores from the user device when anonymizing the intermediate sentiment scores and / or when generating the sentiment scores.

[0231] The control circuitry may be configured to remove the sentiment score from the user device upon transmitting the sentiment score to the server.

[0232] The user device may further include at least one sensor, and the control circuitry may be further configured to capture sensor data with the at least one sensor of the user device.

[0233] The control circuitry can be configured to derive the response data from the captured sensor data of the at least one sensor.

[0234] The at least one sensor of the user device may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near field sensor, and a position sensor.

[0235] The communication circuitry may be further configured to receive user-side sensor data from at least one user-side sensor communicatively coupled with the user device via the communication circuitry.

[0236] The control circuitry may be configured to derive the response data from the received user-side sensor data of the at least one user-side sensor.

[0237] The at least one user-side sensor may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a close-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0238] The at least one user-side sensor may be included in at least one user-side device positioned in the vicinity of the user device, where the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, an IoT device, a health monitor, and an aerosol-generating device.

[0239] The control circuit - deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment influencing the user's sentiment; - determining a sentiment score based on the at least one environmental parameter.

[0240] The control circuitry may be configured to acquire reaction data based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor disposed in the environment of the user device.

[0241] The control circuitry may include a classifier circuit configured to classify at least a portion of the response data to determine a sentiment score.

[0242] The user device may further include at least one camera configured to capture image data indicative of one or more images of a user of the user device, and the control circuitry may be further configured to determine reaction data based on the captured image data and determine, with the classifier circuitry, a sentiment score based on classifying at least a portion of the reaction data.

[0243] The control circuit includes: determining, with a classifier circuit, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - deriving a sentiment score from the determined reaction patterns.

[0244] The control circuitry may be configured to determine the sentiment score based on computing a current sentiment score and / or based on deriving the sentiment score from the computed current sentiment score.

[0245] The communication circuitry may be configured to receive at least one user-side sentiment score from at least one user-side device communicatively coupled to the user device, and the control circuitry may be configured to determine the sentiment score based on the user-side sentiment score received from the at least one user-side device.

[0246] The control circuit - computing a current sentiment score; - may be configured to determine the sentiment score based on at least one user-side sentiment score received from at least one user-side device communicatively coupled to the user device, or at least one user-side sentiment score computed by the control circuit based on user-side sensor data received from the at least one user-side device.

[0247] The control circuitry can be configured to compare the current sentiment score to at least one user-side sentiment score.

[0248] The control circuitry can be configured to determine a deviation between the current sentiment score and the at least one user-side sentiment score and to compare the determined deviation to a threshold value for the deviation.

[0249] The control circuitry may be configured to discard and / or ignore at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if a determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a threshold value for deviation.

[0250] The control circuit includes: - preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches and / or exceeds a deviation threshold; and / or - may be configured to transmit the sentiment score to the server only if the current sentiment score and at least one user-side sentiment score substantially match each other.

[0251] The control circuit includes: - preventing transmission of the sentiment score to the server if a determined deviation between the current sentiment score and at least one user-side sentiment score reaches and / or exceeds a deviation threshold value; receiving a further notification from the server, the further notification substantially matching or equal to the notification; - obtaining further response data indicative of a further response of the user to the further notification; - determining a further sentiment score based on the obtained further reaction data, the further sentiment score indicating a further sentiment of the user in reaction to the further notification.

[0252] The control circuitry may be configured to compare a further sentiment score determined based on further response data indicative of a further response of the user to the further notification to the sentiment score or a current sentiment score determined based on the response data indicative of the user's response to the notification.

[0253] The control circuitry may be configured to transmit at least one of the sentiment score, the current sentiment score, and the further sentiment score to the server only if the further sentiment score, determined based on the further reaction data indicative of the user's further reaction to the further notification, substantially matches the sentiment score or the current sentiment score, determined based on the reaction data indicative of the user's reaction to the notification.

[0254] According to an eighteenth aspect of the present disclosure, a user device configured to interactively communicate with a server is provided. The user device may be configured to perform one or more steps of a method according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure, for example. The user device may also refer to a user device described with reference to the seventeenth aspect of the present disclosure. The user device comprises a communication circuit communicatively coupling the user device to the server and configured to receive one or more notifications from the server. The user device further comprises a user interface configured to provide one or more notifications to a user of the user device, and a control circuit including one or more processors, the control circuit being configured to: obtaining reaction data indicative of one or more reactions of a user to one or more notifications; - determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; - determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores, e.g., the first sentiment score and the second sentiment score, to each other, wherein the final sentiment score can be used by the server to train a reinforcement learning model implemented on the server.

[0255] At least one of the multiple sentiment scores, e.g., at least one of the first and / or second sentiment scores, may be indicative of the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0256] At least two of the plurality of sentiment scores may indicate a user's sentiment in the user's reaction to at least two notifications received from the server, for example, at different times.

[0257] The control circuitry may be configured to determine a final sentiment score based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

[0258] The communications circuitry may be further configured to receive at least one of the plurality of sentiment scores, e.g., at least one of the first and second sentiment scores, from at least one user-side device communicatively coupleable to the user device.

[0259] The communication circuitry may be configured to receive user-side sensor data from the at least one user-side device, and the control circuitry may be further configured to compute at least one of a plurality of sentiment scores, e.g., at least one first and second sentiment scores, based on deriving the further reaction data from the received user-side sensor data.

[0260] The communication circuitry may be configured to receive a first notification and a second notification to the user from the server, the second notification being substantially consistent with and / or equal to the first notification. obtaining first reaction data indicative of a first reaction of a user to a first notification; obtaining second reaction data indicative of a second reaction of the user to the second notification; The method may be further configured to: determine a first sentiment score based on the first response data; and determine a second sentiment score based on the second response data.

[0261] The control circuitry can be configured to determine a deviation between at least two of the plurality of sentiment scores, e.g., between a first sentiment score and a second sentiment score, and compare the determined deviation to a threshold value for the deviation.

[0262] The control circuitry may be configured to discard and / or ignore at least one of the at least two sentiment scores, e.g., one of the first sentiment score and the second sentiment score, if the determined deviation reaches and / or exceeds a threshold for the deviation.

[0263] The control circuitry may be configured to prevent the final sentiment score from being transmitted to the server if a determined deviation between the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold deviation. Alternatively, or additionally, the control circuitry may be further configured to transmit the final sentiment score to the server only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0264] A nineteenth aspect of the present disclosure relates to the use of a user device, as described herein above and herein below, for interactive communication with a server and / or for determining a sentiment score.

[0265] A twentieth aspect of the present disclosure relates to the use of a user-side device, as described herein above and herein below, for interactively communicating with a server and / or for determining a sentiment score.

[0266] According to a twenty-first aspect of the present disclosure, a server configured to interactively communicate with a user device and / or a user-side device is provided. The server may be configured to perform one or more steps of a method according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure, for example. The server comprises a communication arrangement communicatively coupling the server to the user device and configured to transmit a notification to the user device, the communication arrangement further configured to receive a sentiment score from the user device, the sentiment score indicating the user's sentiment in response to the notification. The server further comprises a control arrangement including a reinforcement learning model, the sentiment score being correlated with a reinforcement learning reward for training the reinforcement learning model, and the control arrangement configured to train the reinforcement learning model based on the received sentiment score.

[0267] A server may comprise at least one computing device, for example including one or more processors. However, it should be noted that a server may refer to a server system, a computer system, a server network, a computing network, a cloud computing network, etc. Thus, when referred to as "a server" or "the server," this includes multiple servers.

[0268] Generally, the server may comprise or include an engine or module for identifying, selecting, and / or generating notifications to be sent or transmitted to the user device, e.g., in response to one or more queries from the user device. The server may comprise, e.g., a natural language processing ("NLP") module or engine configured to process, analyze, and / or resolve one or more queries received from the user device. The server may be further configured to determine, e.g., based on the processing of the one or more queries, notifications to be sent to the user device in response to the one or more queries.

[0269] Further, the server may comprise an artificial intelligence module and / or recommendation engine that enables the server to determine, identify, select, and / or generate a notification that may represent an appropriate or most appropriate notification from among or based on a plurality of notifications transmittable to the user device, for example, in response to one or more queries from the user device. By way of example, the server may include a knowledge base that includes a plurality of notifications transmittable to the user device, and the artificial intelligence module or recommendation engine of the server may be configured to determine, select, identify, and / or generate a suitable or most appropriate notification from among or based on a plurality of potential notifications, for example, based on categorizing one or more queries received from the user device.

[0270] The user's communication arrangement may be configured to be coupled with corresponding communication circuitry of the user device to enable data exchange between the user device and the server. As an example, the server may be coupled to and / or configured to communicate with the user device (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0271] The control arrangement of the server may refer to a controller, a control module, and / or a control arrangement.

[0272] Further, the server may comprise a data storage or memory for storing, for example, the sentiment scores, notifications, and / or queries. Alternatively, or in addition, computer programs, instructions, apps, applications, and / or software instructions may be stored in the data storage or memory.

[0273] The control arrangement may include a knowledge base and may be configured to select further notifications for transmission to the user device based on the received sentiment scores.

[0274] The control arrangement may include a reinforcement learning model, the sentiment score may be correlated with a reinforcement learning reward for training the reinforcement learning model, and the control arrangement may be configured to train the reinforcement learning model based on the received sentiment score.

[0275] The control arrangement may be configured to train the reinforcement learning model based on maximizing and / or optimizing a reward function of the reinforcement learning model.

[0276] The communication arrangement may be further configured to receive a query from the user device, and the control arrangement may be configured to transmit a notification from the server to the user device upon receiving the query from the user device.

[0277] The communication arrangement may be configured to receive a further query from the user device, and the control arrangement may be configured to select a further notification based on the received sentiment score and based on the received further query.

[0278] The control arrangement may be configured to transmit, via the communications arrangement, a further notification to the user device in response to the further query.

[0279] The control arrangement may further include a natural language processing engine configured to process the query and / or the further query using the natural language processing engine.

[0280] The communication arrangement may be further configured to receive at least one further sentiment score, and the control arrangement may be further configured to select, from the knowledge base, a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score.

[0281] At least one further sentiment score may be received by the server, from the user device, and / or from the user side device via the communication arrangement.

[0282] A control arrangement may be configured to compare the sentiment score to at least one further sentiment score.

[0283] The control arrangement may be configured to determine a deviation between the sentiment score and the at least one further sentiment score, and to compare the determined deviation to a threshold for the deviation.

[0284] The control arrangement may be configured to discard and / or ignore at least one of the sentiment score and the at least one further sentiment score if a determined deviation between the sentiment score and the at least one further sentiment score reaches and / or exceeds a threshold value for the deviation.

[0285] The control arrangement may include a reinforcement learning model, and the control arrangement may be configured to train the reinforcement learning model based on the sentiment score and / or the at least one further sentiment score only if the sentiment score and the at least one further sentiment score substantially match each other.

[0286] According to a twenty-second aspect of the present disclosure, a server configured to interactively communicate with a user device and / or a user-side device is provided. The server may be configured to perform one or more steps of a method according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure, for example. The server may also refer to the server of the twenty-first aspect of the present disclosure. The server comprises a communication arrangement communicatively coupling the server with the user device and configured to transmit one or more notifications from the server to the user device. The server comprises a control arrangement including a reinforcement learning model, the communication arrangement further configured to receive a plurality of sentiment scores, for example, a first sentiment score and a second sentiment score, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training the reinforcement learning model implemented on the server. Therein, the control arrangement is configured to train the reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores, for example, the first sentiment score and the second sentiment score, with each other.

[0287] The communication arrangement may be further configured to receive at least one of the plurality of sentiment scores, for example at least one of the first and second sentiment scores, from the user device or from a user side device.

[0288] At least one of the multiple sentiment scores, e.g., at least one of the first and / or second sentiment scores, may be indicative of the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0289] At least two of the plurality of sentiment scores, for example the first and second sentiment scores, may indicate a user sentiment in the user's reaction to the at least two notifications received by the user device from the server.

[0290] The communication arrangement may be configured to transmit a first notification and a second notification from a server to the user device, where the second notification may substantially match and / or be equal to the first notification, and the server may be configured to receive a first sentiment score in response to transmitting the first notification to the user device, and to receive a second sentiment score in response to transmitting the second notification to the user device.

[0291] The control arrangement may be configured to determine a deviation between at least two of the plurality of sentiment scores, e.g., between a first sentiment score and a second sentiment score, and the control arrangement may be configured to compare the determined deviation to a threshold value for the deviation.

[0292] The control circuitry may be configured to discard and / or ignore at least one of the plurality of sentiment scores, e.g., one of the first sentiment score and the second sentiment score, if a determined deviation between the at least two of the plurality of sentiment scores reaches and / or exceeds a threshold value for the deviation.

[0293] The control arrangement may be configured to train a reinforcement learning model implemented on the server based on at least one of the received plurality of sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score, only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0294] The control arrangement is - determining a final sentiment score based on and / or based on at least one of the plurality of sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score; Deriving a reinforcement learning reward from the determined final sentiment score for training a reinforcement learning model; and - feeding the reinforcement learning reward to the reinforcement learning model.

[0295] A twenty-third aspect of the present disclosure relates to the use of a server, as described hereinabove and hereinafter, for interactively communicating with a user device, and optionally with one or more user-side devices. Alternatively or additionally, the twenty-third aspect may refer to the use of a server, as described hereinabove and hereinafter, for determining a sentiment score.

[0296] According to a twenty-fourth aspect of the present disclosure, a system for interactive communication between a server and a user device, and optionally at least one user-side device, is provided. The system may be configured to perform one or more steps of a method, for example, according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The system comprises at least one user device, for example, according to one or more of the seventeenth and eighteenth aspects of the present disclosure, as described hereinabove and hereinbelow. The system further comprises a server, for example, according to one or more of the twenty-first and twenty-second aspects of the present disclosure, as described hereinabove and hereinbelow. Optionally, the system may comprise one or more user-side devices and / or one or more user-side sensors, as described hereinabove and hereinbelow.

[0297] A twenty-fifth aspect of the present disclosure relates to the use of a system, as described hereinabove and hereinbelow, for interactive communication between a server and a user device, and optionally one or more user-side devices. Alternatively or additionally, the twenty-fifth aspect may refer to the use of a system, as described hereinabove and hereinbelow, for determining a sentiment score.

[0298] Below is provided a non-exhaustive list of non-limiting examples, any one or more of the features of which may be combined with any one or more features of another example, embodiment, or aspect described herein.

[0299] A.1: A computer-implemented method for interactive communication of a user device with a server, the method comprising: - on the user device, providing a notification to a user of the user device; - obtaining reaction data indicative of a user's reaction to the notification; - determining a sentiment score for transmission to a server based on the obtained reaction data, the sentiment score indicating a sentiment of the user in reaction to the notification.

[0300] B.1: The method of example A.1, further comprising transmitting the determined sentiment score to a server.

[0301] C.1: The method of embodiment A.1 or B.1, wherein the sentiment score is transmitted from the user device to the server.

[0302] D.1: The method of any of embodiments A.1-C.1, wherein the determined sentiment score is transmitted from the user device and / or a user-side device communicatively coupled to the user device.

[0303] E.1: The method of any of embodiments A.1-D.1, further comprising receiving a query at the user device, wherein the notification is provided by the server in response to the query.

[0304] F.1: A method as described in any of Examples A.1 to E.1, wherein the query includes text input received via a user interface at the user device, verbal input received via a microphone at the user device, and / or video-based input received via a camera at the user device.

[0305] G.1: The method of any of embodiments A.1 to F.1, wherein the notification includes text, audio, at least one image, and / or video.

[0306] H.1: The sentiment score is an anonymized numerical measure that indicates a user's reaction to the notification, and / or The method of any of Examples A.1-G.1, wherein the sentiment score correlates with and / or indicates a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0307] I.1: Determining a sentiment score determining an intermediate sentiment score based on the obtained response data; The method of any of embodiments A.1-H.1.

[0308] J.1: The method of example I.1, wherein the sentiment score is anonymized based on normalizing the intermediate sentiment score with the reference sentiment score.

[0309] K.1: The method of example I.1 or J.1, further comprising removing the intermediate sentiment scores from the user device when anonymizing the intermediate sentiment scores and / or when generating the sentiment scores.

[0310] L.1: The method of any of embodiments A.1-K.1, further comprising removing the sentiment score from the user device when transmitting the sentiment score to the server.

[0311] M.1: The method of any of embodiments A.1-L.1, wherein providing a notification on the user device includes displaying the notification on a user interface of the user device.

[0312] N.1: The method of any of embodiments A.1-M.1, wherein obtaining the response data includes capturing sensor data using at least one sensor of the user device.

[0313] O.1: The method of embodiment N.1, further comprising deriving, using the user device, response data from captured sensor data of at least one sensor of the user device.

[0314] P.1: The method of embodiment N.1 or O.1, wherein at least one sensor of the user device is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near field sensor, and a position sensor.

[0315] Q.1: A method according to any of embodiments A.1 to P.1, wherein acquiring the response data includes receiving, using the user device, user-side sensor data from at least one user-side sensor communicatively coupled to the user device.

[0316] R.1: The method of embodiment Q.1, further comprising deriving, using the user device, response data from the received user-side sensor data of at least one user-side sensor.

[0317] S.1: The method of embodiment Q.1 or R.1, wherein at least one user-side sensor is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a close-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0318] T.1: A method described in any of Examples Q.1 to S.1, wherein at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, an IoT device, and an aerosol generating device.

[0319] U.1: deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment affecting the user's sentiment; The method of any of embodiments A.1-T.1, wherein the sentiment score is determined based on at least one environmental parameter.

[0320] V.1: A method according to any of embodiments A.1 to U.1, in which the reaction data is acquired by the user device based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor positioned in the environment of the user device.

[0321] W.1: The method of any of Examples A.1 to V.1, wherein determining the sentiment score includes providing at least a portion of the reaction data to a classifier circuit of the user device, and / or the sentiment score is determined based on classifying at least a portion of the reaction data using a classifier circuit of the user device.

[0322] X.1: - capturing, with a camera of the user device, image data indicative of one or more images of a user of the user device; - determining response data based on the captured image data; and The method of any of Examples A.1-W.1, further including: determining a sentiment score based on providing at least a portion of the reaction data to a classifier circuit of the user device.

[0323] Y.1: Determining a sentiment score determining, with a classifier circuitry of the user device, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - Deriving a sentiment score from the determined response patterns.

[0324] Z.1: A method as described in any of embodiments A.1 to Y.1, wherein the sentiment score is determined based on computing a current sentiment score using a user device and deriving a sentiment score from the computed current sentiment score.

[0325] AA.1: A method according to any of embodiments A.1 to Z.1, wherein determining the sentiment score includes receiving, using the user device, at least one user-side sentiment score from at least one user-side device communicatively coupled to the user device, and the sentiment score is determined based on the user-side sentiment score received from the at least one user-side device.

[0326] AB.1: A method according to any of embodiments A.1 to AA.1, in which the sentiment score is determined based on a current sentiment score computed using the user device and at least one user-side sentiment score received by the user device from at least one user-side device communicatively coupleable to the user device, or at least one user-side sentiment score computed by the user device based on user-side sensor data received from at least one user-side device.

[0327] AC.1: The method of example AB.1, wherein determining the sentiment score includes comparing a current sentiment score and at least one user-side sentiment score.

[0328] AD.1: Determining a sentiment score - determining a deviation between a current sentiment score and at least one user sentiment score; The method of embodiment AB.1 or AC.1, comprising: comparing the determined deviation to a threshold value for the deviation.

[0329] AF.1: The method of embodiment AD.1, further comprising discarding at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if a determined deviation between the current sentiment score and the at least one user-side sentiment score meets or exceeds a threshold for deviation.

[0330] AG.1: - preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a deviation threshold; and / or The method of any of embodiments AD.1-AF.1, further comprising transmitting the sentiment score to the server only if the current sentiment score and the at least one user-side sentiment score substantially match each other.

[0331] AH.1: - preventing transmission of the sentiment score to the server if a determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a threshold deviation; receiving a further notification from the server to the user, the further notification substantially matching or equal to the notification; - obtaining further response data indicative of a further response of the user to the further notification; The method of any of embodiments AD.1 to AG.1, further comprising: determining a further sentiment score based on the obtained further response data, the further sentiment score indicating a further sentiment of the user in response to the further notification.

[0332] AI.1: The method of example AH.1, further comprising comparing a further sentiment score determined based on further response data indicating the user's further response to the further notification with the sentiment score or the current sentiment score determined based on the response data indicating the user's response to the notification.

[0333] AJ.1: The method of embodiment AH.1 or AI.1, further comprising transmitting at least one of the sentiment score, the current sentiment score, and the further sentiment score to a server only if the further sentiment score, determined based on further reaction data indicative of the user's further reaction to the further notification, substantially matches the sentiment score or the current sentiment score, determined based on the reaction data indicative of the user's reaction to the notification.

[0334] AK.1: The method of any of A.1-AJ.1, further comprising aggregating a plurality of sentiment scores to determine a sentiment score for transmission to a server.

[0335] AL.1: The method of any of Examples A.1-AK.1, wherein aggregating the plurality of sentiment scores includes applying an aggregation function.

[0336] AM.1: A method as described in any of embodiments A.1 to AL.1, further comprising determining a privacy level for at least one of the reaction data, sensor data of one or more sensors of the user device, and user-side sensor data of one or more user-side sensors, wherein the privacy level is indicative of one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data, and the one or more features are to be manipulated in and / or removed from one or more of the reaction data, the sensor data, and the user-side sensor data to determine and / or before determining the sentiment score.

[0337] AN.1: The method of any of embodiments A.1-AM.1, wherein the privacy level is at least one of definable, user configurable, and programmable.

[0338] AO.4: A method as described in any of embodiments A.1 to AN.1, further comprising selecting one or more processing operations based on a defined privacy level for one or more of the feature groups to process one or more of the reaction data, sensor data, and user-side sensor data that may be selected by the user device for one or more features defined by the one or more feature groups.

[0339] AP.1: A method as described in any of Examples A.1 to AO.1, wherein one or more features in one or more of the reaction data, sensor data, and user-side sensor data may include and / or refer to a user's reaction features and / or reaction patterns related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expressions, changes in the user's skin color, and one or more environmental parameters that affect the user's sentiment.

[0340] AQ.1: A method as described in any of embodiments A.1 to AP.1, wherein the selected one or more processing operations for processing one or more of the reaction data, sensor data, and user-side sensor data depend on the data type of the reaction data, sensor data, and / or user-side sensor data.

[0341] A.2: A computer program comprising instructions, which when executed on one or more processors of a user device, cause the user device to perform the method described in any of Examples A.1 to AQ.1.

[0342] A.3: A non-transitory computer-readable medium storing a computer program according to embodiment A.2.

[0343] A.4: A computer-implemented method for interactive communication of a user device with a server, the method comprising: - on the user device, providing one or more notifications to a user of the user device; obtaining reaction data indicative of one or more reactions of a user to one or more notifications; - determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores to one another, wherein the final sentiment score is usable by the server to train a reinforcement learning model implemented on the server.

[0344] B.4: The method described in Example A.4, wherein at least one of the plurality of sentiment scores is related to and / or indicative of a user's environment affecting the user's sentiment, indicative of the user's sentiment in the user's reaction to at least one environmental parameter, and / or at least two of the plurality of sentiment scores indicative of the user's sentiment in the user's reaction to at least two notifications received from the server.

[0345] C.4. The method of example A.4 or B.4, wherein the final sentiment score is determined based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

[0346] D.4. The method of any of embodiments A.4-C.4, further comprising receiving, using the user device, at least one of the plurality of sentiment scores from at least one user-side device communicatively coupled to the user device.

[0347] E.4: The method of any of Examples A.4 to D.4, further comprising: receiving, using a user device, user-side sensor data from at least one user-side device; and computing at least one of a plurality of sentiment scores based on deriving, using the user device, at least a portion of the reaction data and / or further reaction data from the received user-side sensor data.

[0348] F.4: Obtaining reaction data receiving, by a user device, user-side sensor data from at least one user-side sensor communicatively coupleable to the user device; The method of any of embodiments A.4-E.4, comprising: - deriving, using a user device, reaction data from the received user-side sensor data of the at least one user-side sensor.

[0349] G.4: The method of embodiment F.4, wherein the at least one user-side sensor is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a near-field sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0350] H.4: The method described in embodiment F.4 or G.4, wherein at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, an IoT device, and an aerosol generating device.

[0351] I.4: A method as described in any of Examples A.4 to H.4, wherein obtaining the response data includes capturing sensor data using at least one sensor of the user device, and deriving the response data from the captured sensor data of the at least one sensor of the user device using the user device.

[0352] J.4: The method of Example I.4, wherein at least one sensor of the user device is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near field sensor, and a position sensor.

[0353] K.4: The method of any of embodiments A.4-J.4, further comprising receiving at least one query at the user device, wherein at least one notification is provided by the server in response to the at least one query.

[0354] L.4: The method described in example K.4, wherein at least the query includes text input received via a user interface at the user device, verbal input received via a microphone at the user device, and / or video-based input received via a camera at the user device.

[0355] M.4: The method of any of embodiments A.4-L.4, wherein each of the one or more notifications includes at least one of text, voice, an audio message, a message, an alert, an image, and a video.

[0356] N.4. The method of any of embodiments A.4-M.4, further comprising transmitting the determined final sentiment score to a server.

[0357] O.4: A method as described in any of embodiments A.4 to N.4, wherein the determined final sentiment score is transmitted from the user device and / or the determined final sentiment score is transmitted from a user-side device communicatively coupled to the user device.

[0358] P.4: The method of any of embodiments A.4-O.4, wherein the at least one notification includes at least one of text, voice, an audio message, a message, an alert, an image, and a video.

[0359] Q.4: The method of any of embodiments A.4 to P.4, wherein providing one or more notifications on the user device includes displaying one or more notifications on a user interface of the user device.

[0360] R.4: The method of any of Examples A.4-Q.4, wherein determining a final sentiment score includes determining an intermediate sentiment score based on the obtained response data and anonymizing the intermediate sentiment score, thereby generating a final sentiment score.

[0361] S.4: The method of example R.4, wherein the final sentiment score is anonymized based on normalizing the intermediate sentiment score with the reference sentiment score.

[0362] T.4: The method of any of embodiments R.4-S.4, further comprising removing the intermediate sentiment scores from the user device when anonymizing the intermediate sentiment scores.

[0363] U.4: The method of any of embodiments A.4-T.4, further comprising removing the final sentiment score from the user device upon transmitting the final sentiment score to the server.

[0364] V.4: The method of any of examples A.4 to U.4, wherein providing one or more notifications includes receiving at the user device a first notification and a second notification, wherein the second notification substantially matches and / or is equal to the first notification, obtaining the response data includes obtaining first response data indicative of a first response of the user to the first notification and obtaining second response data indicative of a second response of the user to the second notification, and determining the plurality of sentiment scores includes determining a first sentiment score based on the first response data and determining a second sentiment score based on the second response data.

[0365] W.4: The method of any of embodiments A.4 to V.4, wherein determining the final sentiment score includes determining a deviation between at least two of the plurality of sentiment scores and comparing the determined deviation to a threshold for the deviation.

[0366] X.4: The method of example W.4, further comprising discarding at least one of the at least two sentiment scores if the determined deviation meets or exceeds a threshold for the deviation.

[0367] Y.4: - preventing the final sentiment score from being transmitted to the server if the determined deviation between the at least two sentiment scores reaches or exceeds a deviation threshold; and / or The method of embodiment W.4 or X.4, further comprising transmitting a final sentiment score to a server only if the at least two sentiment scores substantially match one another.

[0368] Z.4: The method of any of A.4 to y.4, further comprising aggregating multiple sentiment scores to determine a final sentiment score for transmission to a server.

[0369] AA.4: The method of any of examples A.4-Z.4, wherein aggregating the plurality of sentiment scores includes applying an aggregation function.

[0370] AB.4: A method as described in any of embodiments A.4 to AA.4, further comprising determining a privacy level for at least one of the reaction data, sensor data of one or more sensors of the user device, and user-side sensor data of one or more user-side sensors, wherein the privacy level is indicative of one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data, and the one or more features are to be manipulated in and / or removed from one or more of the reaction data, the sensor data, and the user-side sensor data to determine and / or prior to determining the sentiment score.

[0371] AC.4: The method of any of embodiments A.4-AB.4, wherein the privacy level is at least one of definable, user configurable, and programmable.

[0372] AD.4: A method as described in any of embodiments A.4 to AC.4, further comprising selecting one or more processing operations based on a defined privacy level for one or more of the feature groups to process one or more of the reaction data, sensor data, and user-side sensor data, which may be selected by the user device, for one or more features defined by the one or more feature groups.

[0373] AE.4: A method as described in any of Examples A.4 to AD.1, wherein the one or more features in one or more of the reaction data, sensor data, and user-side sensor data may include and / or refer to a user's reaction features and / or reaction patterns related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expressions, changes in the user's skin color, and one or more environmental parameters that affect the user's sentiment.

[0374] AF.4: A method as described in any of embodiments A.4 to AE.1, wherein the selected one or more processing operations for processing one or more of the response data, sensor data, and user-side sensor data depend on the data type of the response data, sensor data, and / or user-side sensor data.

[0375] A.5: A computer program comprising instructions, which when executed on one or more processors of a user device, cause the user device to perform the method described in any of Examples A.4 to AF.4.

[0376] A.6: A non-transitory computer-readable medium storing a computer program according to example A.5.

[0377] A.7: A computer-implemented method for interactive communication of a server with a user device, the method comprising: transmitting a notification from the server to the user device; receiving, by a server, a sentiment score, the sentiment score being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on the received sentiment scores.

[0378] B.7: The method of example A.7, wherein the reinforcement learning model is trained based on maximizing a reward function of the reinforcement learning model implemented on the server.

[0379] C.7: The method of any one of embodiments A.7 to B.7, further comprising selecting, from the server's knowledge base, a further notification to be transmitted to the user device based on the received sentiment score.

[0380] D.7: The method of any of embodiments A.7-C.7, further comprising receiving a query from the user device, wherein a notification is transmitted from the server to the user device upon receiving the query from the user device.

[0381] E.7: The method of any of embodiments A.7 to D.7, further comprising receiving a further query from the user device, wherein the further notification is selected based on the received sentiment score and based on the received further query.

[0382] F.7: The method of any of embodiments E.7, further comprising transmitting a further notification to the user device in response to the further query.

[0383] G.7: The method of any of embodiments A.7-F.7, further comprising processing the query and / or the further query using a natural language processing engine implemented on the server.

[0384] H.7: receiving at least one further sentiment score; and A method as described in any of embodiments A.7 to G.7, further comprising: - selecting, from the server's knowledge base, a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score.

[0385] I.7: The method of example H.7, wherein at least one further sentiment score is received by the server, from the user device, and / or from at least one user-side device.

[0386] J.7: The method of example H.7 or I.7, further comprising comparing the sentiment score to at least one additional sentiment score.

[0387] K.7: - determining a deviation between the sentiment score and at least one further sentiment score; The method of any of embodiments H.7-J.7, further comprising: comparing the determined deviation to a threshold value for the deviation.

[0388] L.7: The method of example K.7, further comprising discarding at least one of the sentiment score and the at least one additional sentiment score if a determined deviation between the sentiment score and the at least one additional sentiment score meets or exceeds a threshold for deviation.

[0389] M.7: The method of any of Examples H.7 to L.7, further comprising training a reinforcement learning model implemented on the server based on the sentiment score and / or the at least one additional sentiment score only if the sentiment score and the at least one additional sentiment score substantially match each other.

[0390] A.8: A computer program comprising instructions, which when executed on one or more processors of a server, cause the server to perform the method described in any of embodiments A.7 to M.7.

[0391] A.9: A non-transitory computer-readable medium storing a computer program according to example A.8.

[0392] A.10: A computer-implemented method for interactive communication of a server with a user device, the method comprising: transmitting one or more notifications from the server to the user device; receiving, by a server, a plurality of sentiment scores, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores.

[0393] B.10: The method of example A.10, wherein at least one of the plurality of sentiment scores is received by a server, from the user device, or from a user-side device.

[0394] C.10: The method of any of Examples A.10 to B.10, wherein at least one of the plurality of sentiment scores indicates a user's sentiment in the user's response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0395] D.10: A method as described in any of embodiments A.10 to C.10, wherein at least one of the plurality of sentiment scores indicates a user's sentiment in the user's reaction to at least one of the one or more notifications received by the user device from the server.

[0396] E.10: A method as described in any of embodiments A.10 to DC.10, wherein transmitting one or more notifications from the server to the user device includes transmitting a first notification and a second notification from the server to the user device, wherein the second notification substantially matches or is equal to the first notification, and receiving a plurality of sentiment scores includes receiving a first sentiment score in response to transmitting the first notification to the user device, and receiving a second sentiment score in response to transmitting the second notification to the user device.

[0397] F.10: The method of any of examples A.10-E.10, comprising determining a deviation between at least two of the plurality of sentiment scores and comparing the determined deviation to a threshold for the deviation.

[0398] G.10: The method of embodiment F.10, further comprising discarding at least one of the plurality of sentiment scores if the determined deviation between the at least two of the plurality of sentiment scores meets or exceeds a threshold for deviation.

[0399] H.10: The method of embodiment F.10 or G.10, further comprising training a reinforcement learning model implemented on a server based on at least one of the received sentiment scores only if the at least two sentiment scores substantially match one another.

[0400] I.10: Training a reinforcement learning model determining a final sentiment score based on at least one of the plurality of sentiment scores; training a reinforcement learning model from the determined final sentiment score and / or deriving a reinforcement learning reward for computing the determined sentiment score based on the final sentiment score; - feeding the reinforcement learning reward to the reinforcement learning model.

[0401] J.10: The method of any of A.10 to I.10, further comprising aggregating a plurality of sentiment scores to determine a sentiment score for transmission to a server.

[0402] K.10: The method of any of examples A.10-J.10, wherein aggregating the plurality of sentiment scores includes applying an aggregation function.

[0403] L.10: The method of any of embodiments A.10-K.10, wherein a reinforcement learning reward is determined, calculated, and / or computed based on at least the final sentiment score and a reward function of the reinforcement learning model.

[0404] M.10: The method of any of Examples A.10-L.10, wherein determining and / or computing the reinforcement learning reward includes determining a trend for a plurality and / or series of prior sentiment scores.

[0405] N.10: The method of any of embodiments A.10-M.10, further including determining weights based on a determined trend of a plurality and / or sequence of previous sentiment scores, and determining a reinforcement learning reward based on the determined weights and the final sentiment score.

[0406] O.10: The method of any of embodiments A.10-N.10, further comprising comparing the final sentiment score to a trend of a plurality and / or series of previous sentiment scores and determining a reinforcement learning reward based on the comparison.

[0407] P.10: The method of any of embodiments A.10-O.10, wherein determining the reinforcement learning reward is determined based on one or more previous reinforcement learning rewards, determined based on one or more previous sentiment scores.

[0408] A.11: A computer program comprising instructions, which when executed on one or more processors of a server, cause the server to perform the method described in any of embodiments A.10 to P.10.

[0409] A.12: A non-transitory computer-readable medium storing the computer program of Example A.11.

[0410] A.13: A computer-implemented method for interactive communication between a user device and a server, the method comprising: transmitting a notification from the server to the user device; - on the user device, providing a notification to a user of the user device; - obtaining, by means of the user device and / or by means of at least one user-side device, reaction data indicative of a reaction of the user to the notification; - determining, using the user device and / or using at least one user-side device, a sentiment score based on the acquired reaction data, the sentiment score indicating a user's sentiment in reaction to the notification, the sentiment score correlating with a reinforcement learning reward for training a reinforcement learning model implemented on a server; - transmitting the determined sentiment score from the user device and / or from the at least one user-side device to a server; receiving, by a server, a sentiment score; training an enrichment model implemented on the server based on the sentiment scores received by the server.

[0411] A.14: A computer-implemented method for interactive communication between a user device and a server, the method comprising: transmitting one or more notifications from the server to the user device; - on the user device, providing one or more notifications to a user of the user device; - obtaining, using the user device and / or using at least one user-side device, reaction data indicative of one or more reactions of a user to the one or more notifications; - determining, using the user device and / or using at least one user-side device, a plurality of sentiment scores based on the acquired reaction data, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on a server; comparing at least two of the determined sentiment scores to one another; transmitting at least one of the sentiment scores from the user device and / or at least one user-side device to a server; receiving, by a server, at least one of a plurality of sentiment scores; training an enrichment model implemented on the server based on at least one of the plurality of sentiment scores received by the server.

[0412] A.15: A computer program comprising instructions, which when executed on one or more processors of a server, cause the server to perform the method of example A.13 or A.14.

[0413] A.16: A non-transitory computer-readable medium storing the computer program of Example A.15.

[0414] A.17: A user device configured for interactive communication with a server, the user device comprising: a communication circuit communicatively coupling the user device to a server and configured to receive notifications from the server; a user interface configured to provide a notification to a user of a user device; A control circuit including one or more processors, the control circuit comprising: - obtaining reaction data indicative of a user's reaction to the notification; a control circuit configured to: determine a sentiment score for transmission to a server based on the acquired reaction data, the sentiment score indicating a user's sentiment in reaction to the notification.

[0415] B.17. The user device of embodiment A.17, wherein the control circuitry is further configured to transmit the determined sentiment score to a server via the communications circuitry of the user device.

[0416] C.17: A user device as described in Example A.17 or B.17, wherein the sentiment score is an anonymized numerical measure indicating the user's reaction to the notification and / or the sentiment score correlates with or indicates a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0417] D.17: The control circuit determining an intermediate sentiment score based on the obtained reaction data; and The user device of example A.17 or B.17, further configured to: anonymize the intermediate sentiment score to generate a sentiment score.

[0418] E.17: The user device of embodiment D.17, wherein the control circuitry is configured to anonymize the sentiment score based on normalizing the intermediate sentiment score with a reference sentiment score.

[0419] F.17: The user device of embodiment E.17, wherein the control circuitry is configured to remove the intermediate sentiment scores from the user device when anonymizing the intermediate sentiment scores and / or when generating the sentiment scores.

[0420] G.17: The user device of any of embodiments A.17-F.17, wherein the control circuitry is configured to remove the sentiment score from the user device when transmitting the sentiment score to the server.

[0421] H.17: A user device described in any of Examples A.17 to G.17, wherein the user device further comprises at least one sensor and the control circuit is further configured to capture sensor data using the at least one sensor of the user device.

[0422] I.17: The user device of embodiment H.17, wherein the control circuitry is further configured to derive response data from captured sensor data of the at least one sensor.

[0423] J.17: A user device as described in embodiment H.17 or I.17, wherein at least one sensor of the user device is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near field sensor, and a position sensor.

[0424] K.17: A user device described in any of embodiments A.17 to J.17, wherein the communications circuitry is further configured to receive user-side sensor data from at least one user-side sensor communicatively coupled to the user device via the communications circuitry.

[0425] L.17: A user device as described in embodiment K.17, wherein the control circuitry is further configured to derive response data from received user-side sensor data of at least one user-side sensor.

[0426] M.17: A user device as described in embodiment K.17 or L.17, wherein at least one user-side sensor is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a close-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0427] N.17: A user device as described in Examples K.17 to M.17, wherein at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, and an aerosol generating device.

[0428] O.17: The control circuit is - deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment influencing the user's sentiment; The user device of any of embodiments A.17 to N.17, further configured to: determine a sentiment score based on at least one environmental parameter.

[0429] P.17: A user device described in any of embodiments A.17 to O.17, wherein the control circuit is configured to acquire reaction data based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor positioned in the environment of the user device.

[0430] Q.17: A user device of any of embodiments A.17 to P.17, wherein the control circuitry includes a classifier circuit configured to classify at least a portion of the response data to determine a sentiment score.

[0431] R.17: A user device as described in any of Examples A.17 to Q.17, wherein the user device further comprises at least one camera configured to capture image data indicating one or more images of a user of the user device, and the control circuitry is further configured to determine reaction data based on the captured image data and determine a sentiment score based on classifying at least a portion of the reaction data using a classifier circuit.

[0432] S.17: The control circuit determining, with a classifier circuit, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - The user device of embodiment R.17, further configured to: - derive a sentiment score from the determined response pattern.

[0433] T.17: A user device as described in any of Examples A.17 to S.17, wherein the control circuitry is configured to determine the sentiment score based on computing a current sentiment score and / or based on deriving the sentiment score from the computed current sentiment score.

[0434] U.17: A user device described in any of embodiments A.17 to T.17, wherein the communication circuitry is configured to receive at least one user-side sentiment score from at least one user-side device communicatively coupled to the user device, and the control circuitry is configured to determine a sentiment score based on the user-side sentiment score received from the at least one user-side device.

[0435] V.17: A control circuit comprising: - computing a current sentiment score; -A user device described in any of embodiments A.17 to U.17, configured to determine a sentiment score based on at least one user-side sentiment score received from at least one user-side device communicatively coupled to the user device, or at least one user-side sentiment score computed by a control circuit based on user-side sensor data received from at least one user-side device.

[0436] W.17: The user device of embodiment V.17, wherein the control circuitry is configured to compare the current sentiment score with at least one user-side sentiment score.

[0437] X.17: The control circuit - determining a deviation between a current sentiment score and at least one user sentiment score; The user device of embodiment W.17, further configured to: compare the determined deviation with a threshold value for the deviation.

[0438] Y.17: The user device of embodiment X.17, wherein the control circuitry is further configured to discard at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if a determined deviation between the current sentiment score and the at least one user-side sentiment score reaches or exceeds a threshold for the deviation.

[0439] Z.17: The control circuit - preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a deviation threshold; and / or The user device of embodiment X.17 or Y.17, further configured to transmit the sentiment score to the server only if the current sentiment score and the at least one user-side sentiment score substantially match each other.

[0440] AA.17. The control circuit - preventing transmission of the sentiment score to the server if a determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a threshold deviation; receiving a further notification from the server, the further notification substantially matching or equal to the notification; - obtaining further response data indicative of a further response of the user to the further notification; A user device described in any of Examples X.17 to Z.17, further configured to: determine a further sentiment score based on the obtained further reaction data, the further sentiment score indicating a further sentiment of the user in reaction to the further notification.

[0441] AB.17: The control circuit -A user device as described in Example AA.17, further configured to compare a further sentiment score determined based on further reaction data indicating the user's further reaction to the further notification with the sentiment score or a current sentiment score determined based on reaction data indicating the user's reaction to the notification.

[0442] AC.17: The control circuit - The user device of Example AB.17, further configured to transmit at least one of the sentiment score, the current sentiment score, and the further sentiment score to the server only if the further sentiment score, determined based on further reaction data indicative of the user's further reaction to the further notification, substantially matches the sentiment score or the current sentiment score, determined based on the reaction data indicative of the user's reaction to the notification.

[0443] A.18: A user device configured for interactive communication with a server, the user device comprising: a communication circuit communicatively coupling the user device to a server and configured to receive one or more notifications from the server; a user interface configured to provide one or more notifications to a user of a user device; A control circuit including one or more processors, the control circuit comprising: obtaining reaction data indicative of one or more reactions of a user to one or more notifications; - determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; and determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores; The final sentiment score is usable by the server to train a reinforcement learning model implemented on the server, the user device.

[0444] B.18: A user device as described in Example A.18, wherein at least one of the multiple sentiment scores indicates a user's sentiment in the user's response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0445] C.18: The user device of embodiment A.18 or B.18, wherein at least two of the plurality of sentiment scores indicate a user sentiment in the user's reaction to at least two notifications received from the server.

[0446] D.18: A user device of any of embodiments A.18-C.18, wherein the control circuitry is further configured to determine a final sentiment score based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

[0447] E.18: A user device according to any of embodiments A.18 to D.18, wherein the communications circuitry is further configured to receive at least one of the plurality of sentiment scores from at least one user-side device communicatively coupled to the user device.

[0448] F.18: A user device described in any of Examples A.18 to E.18, wherein the communications circuitry is further configured to receive user-side sensor data from at least one user-side device, and the control circuitry is further configured to compute at least one of a plurality of sentiment scores based on deriving further reaction data from the received user-side sensor data.

[0449] G.18: A method according to the present invention, comprising: a communication circuit configured to receive a first notification and a second notification for a user from a server, the second notification substantially matching or equal to the first notification; and a control circuit configured to: obtaining first reaction data indicative of a first reaction of a user to a first notification; obtaining second reaction data indicative of a second reaction of the user to the second notification; -A user device of any of Examples A.18 to F.18, further configured to determine a first sentiment score based on the first response data and to determine a second sentiment score based on the second response data.

[0450] H.18: The control circuit determining a deviation between at least two of the plurality of sentiment scores; The user device of any of embodiments A.18 to G.18, further configured to: compare the determined deviation to a threshold for the deviation.

[0451] I.18: A user device as described in embodiment H.18, wherein the control circuitry is further configured to discard at least one of the at least two sentiment scores if the determined deviation reaches or exceeds a threshold for the deviation.

[0452] J.18: A user device as described in embodiment H.18 or I.18, wherein the control circuitry is further configured to prevent the final sentiment score from being transmitted to the server if the determined deviation between the at least two sentiment scores reaches or exceeds a threshold for deviation, and / or the control circuitry is further configured to transmit the final sentiment score to the server only if the at least two sentiment scores substantially match one another.

[0453] A.19. Use of a user device according to any one of embodiments A.17-J.18 to interactively communicate with a server.

[0454] A.20: A server configured for interactive communication with a user device, the server comprising: - a communication arrangement communicatively coupling the server to a user device and configured to transmit a notification to the user device, the communication arrangement further configured to receive a sentiment score from the user device, the sentiment score indicative of a sentiment of the user in response to the notification; and a control arrangement including a reinforcement learning model, The sentiment score is correlated with a reinforcement learning reward for training a reinforcement learning model, The server, wherein the control arrangement is configured to train a reinforcement learning model based on the received sentiment scores.

[0455] B.20: The server of embodiment A.20, wherein the server includes a knowledge base and is configured to select further notifications for transmission to the user device based on the received sentiment score.

[0456] C.20: A server as described in embodiment A.20 or B.20, wherein the control arrangement further includes a reinforcement learning model, the sentiment score correlates with a reinforcement learning reward for training the reinforcement learning model, and the control arrangement is configured to train the reinforcement learning model based on the received sentiment score.

[0457] D.20: The server of any of embodiments A.20-C.20, wherein the control arrangement is configured to train the reinforcement learning model based on maximizing a reward function of the reinforcement learning model.

[0458] E.20: A server described in any of embodiments A.20 to C.20, wherein the communication arrangement is further configured to receive a query from the user device, and the control arrangement is configured to transmit a notification from the server to the user device upon receiving the query from the user device.

[0459] F.20: A server described in any of embodiments A.20 to E.20, wherein the communication arrangement is configured to receive a further query from the user device, and the control arrangement is configured to select a further notification based on the received sentiment score and based on the received further query.

[0460] G.20: The server of embodiment F.20, wherein the control arrangement is further configured to transmit, via the communication arrangement, a further notification to the user device in response to the further query.

[0461] H.20: The server of any of embodiments A.20-G.20, wherein the control arrangement further includes a natural language processing engine configured to process the query and / or the further query using the natural language processing engine.

[0462] I.20: A server described in any of embodiments A.20 to H.20, wherein the communication arrangement is further configured to receive at least one further sentiment score, and the control arrangement is further configured to select from the knowledge base a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score.

[0463] J.20: The server of embodiment I.20, wherein at least one further sentiment score is received by the server from the user device or from a user-side device via a communication arrangement.

[0464] K.20: The server of embodiment I.20 or J.20, wherein the control arrangement is further configured to compare the sentiment score to at least one additional sentiment score.

[0465] L.20: A server as described in any of embodiments I.20 to K.20, wherein the control arrangement is further configured to determine a deviation between the sentiment score and at least one further sentiment score, and compare the determined deviation to a threshold for the deviation.

[0466] M.20: The server of embodiment L.20, further configured to discard at least one of the sentiment score and the at least one further sentiment score if a determined deviation between the sentiment score and the at least one further sentiment score reaches or exceeds a threshold value for the deviation.

[0467] N.20: A server as described in embodiment L.20 or M.20, wherein the control arrangement includes a reinforcement learning model, and the control arrangement is configured to train the reinforcement learning model based on the sentiment score and / or the at least one further sentiment score only if the sentiment score and the at least one further sentiment score substantially match each other.

[0468] A.21: A server configured for interactive communication with a user device, the server comprising: a communication arrangement communicatively coupling the server to the user device and configured to transmit one or more notifications from the server to the user device; a control arrangement including a reinforcement learning model; The communication arrangement is further configured to receive a plurality of sentiment scores, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; The server, wherein the control arrangement is configured to train a reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores.

[0469] B.21: The server of embodiment A.21, wherein the communication arrangement is configured to receive, by the server, at least one of the plurality of sentiment scores from the user device or a user-side device.

[0470] C.21: A server as described in embodiment A.21 or B.21, wherein at least one of the plurality of sentiment scores is related to and / or indicative of a user's environment affecting the user's sentiment, indicates a user's sentiment in the user's reaction to at least one environmental parameter, and / or at least two of the plurality of sentiment scores indicate a user's sentiment in the user's reaction to at least two notifications received from the server.

[0471] D.21: A server described in any of embodiments A.21 to C.21, wherein the communication arrangement is configured to transmit a first notification and a second notification from the server to the user device, the second notification substantially matching or equal to the first notification, and the server is configured to receive a first sentiment score in response to transmitting the first notification to the user device, and to receive a second sentiment score in response to transmitting the second notification to the user device.

[0472] E.21: A server described in any of embodiments A.21 to D.21, wherein the control arrangement is configured to determine a deviation between at least two of the plurality of sentiment scores, and the control arrangement is configured to compare the determined deviation to a threshold value for the deviation.

[0473] F.21: The server of embodiment E.21, wherein the control arrangement is configured to discard at least one of the plurality of sentiment scores if the determined deviation between the at least two of the plurality of sentiment scores reaches or exceeds a threshold for the deviation.

[0474] G.21: A server as described in embodiment E.21 or F.21, wherein the control arrangement is configured to train a reinforcement learning model implemented on the server based on at least one of the received plurality of sentiment scores only if the at least two sentiment scores substantially match each other.

[0475] H.21: The control arrangement is determining a final sentiment score based on at least one of the plurality of sentiment scores; Deriving a reinforcement learning reward from the determined final sentiment score for training a reinforcement learning model; and The server of any of embodiments A.21 to G.21, further configured to: - feed the reinforcement learning reward to the reinforcement learning model.

[0476] A.22: Use of the server of any of embodiments A.20 to H.21 for interactive communication with a user device.

[0477] A.23: A system for interactive communication between a server and a user device, the system comprising: A user device according to any of embodiments A.17 to J.18, A system comprising: a server according to any one of embodiments A.20 to H.21.

[0478] The embodiments will now be further described with reference to the figures. [Brief description of the drawings]

[0479] [Figure 1] FIG. 1 illustrates a system for interactive communication between a server and at least one user device. [Diagram 2] FIG. 2 shows a user device for interactive communication with a server. [Diagram 3] FIG. 3 illustrates a server for interactive communication with a user device. [Figure 4] FIG. 4 shows a flow chart illustrating a method for interactive communication. [Diagram 5] FIG. 5 shows a flow chart illustrating a method for interactive communication of a user device with a server. [Figure 6] FIG. 6 shows a flow chart illustrating a method for interactive communication of a user device with a server. [Figure 7] FIG. 7 shows a flow chart illustrating a method for interactive communication of a server with a user device. [Figure 8] FIG. 8 shows a flow chart illustrating a method for interactive communication of a server with a user device. [Figure 9] FIG. 9 shows a flow chart illustrating a method between a user device and a server. [Figure 10]FIG. 10 shows a flow chart illustrating a method between a user device and a server.

[0480] The figures are merely schematic and not to scale. As a rule, identical or similar parts, elements and / or steps are provided with identical or similar reference numbers in the figures.

[0481] Fig. 1 shows a system 500 for interactive communication between a server 100 and at least one user device 10. Fig. 2 shows a user device 10 for interactive communication with the server 100. Fig. 3 shows a server 100 for interactive communication with the user device 10. Unless otherwise stated, reference is made to any or all of Figs. 1-3 below.

[0482] The system 500 comprises at least one user device 10 and at least one server 100 .

[0483] In the example shown in Figures 1 and 2, the user device 10 is a handheld user device 10. Alternatively, the user device 10 may be a stand-alone or fixedly installed device. By way of example, the user device 10 may refer to a handheld, a smartphone, a personal computer ("PC"), a tablet PC, a laptop, or a computer, etc.

[0484] The user device 10 comprises a user interface 12, e.g., a display and / or a touch display, configured to receive user input, e.g., one or more queries, and / or to provide, e.g., display, one or more notifications 14, 16 to a user of the user device 10. In the example shown in Figure 1, two notifications are displayed on the user interface 12.

[0485] The user device 10 further comprises a microphone 13 for providing one or more notifications to the user. It should be noted that multiple notifications 14, 16 may be provided to the user simultaneously or sequentially.

[0486] Additionally, the user device 10 comprises a communication circuit 18 configured to communicatively couple the user device 10 to the server 100 and to receive one or more notifications 14, 16 from the server 100. By way of example, the user device 10 may be coupled to and / or configured to communicate with the server 100 (or vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol. The communication link or connection between the server 100 and the user device 10 is indicated by reference numeral 101 in FIG. 1 .

[0487] The user device 10 further comprises control circuitry 20 including one or more processors 21. The control circuitry 20 is configured to obtain reaction data indicative of a user's reaction to the one or more notifications 14, 16. The control circuitry 20 is further configured to determine, based on the obtained reaction data, one or more sentiment scores for transmission to the server 100, where the at least one sentiment score is indicative of a user's sentiment in reaction to the one or more of the notifications 14, 16.

[0488] The user device 10 further comprises a data storage device 22 or memory 22 for storing, for example, one or more notifications 14, 16, reaction data, any other data, and / or software instructions.

[0489] The user device 10 further comprises a number of sensors 24, 26, 28 for capturing and / or acquiring sensor data. For example, the sensor 24 may refer to the camera 24 of the user device 10 configured to capture image data including one or more images as sensor data. Furthermore, the sensor 26 may be a motion sensor and the sensor 28 may be a GPS sensor. However, any of the sensors 24, 26, 28 may be a different type of sensor, such as, for example, a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitive sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, a position sensor, and a global positioning system ("GPS") sensor.

[0490] The server 100 comprises a communication arrangement 102 communicatively coupling the server 100 to the user device 10 and configured to transmit notifications to the user device 10, the communication arrangement 102 being further configured to receive one or more sentiment scores from the user device 10.

[0491] The server 100 further comprises a control arrangement 104 and an artificial intelligence module 108. The server 100 further comprises a reinforcement learning engine or model 110 configured to select one or more notifications 14, 16 for transmission to the user device 10.

[0492] The server 100 further comprises a data storage device 106 having a knowledge base 107 configured to store, for example, one or more notifications 14, 16, one or more queries from the user device 10, one or more sentiment scores, and / or software instructions.

[0493] As described herein above, the sentiment score is an anonymized numerical measure indicative of a user's reaction to one or more notifications 14, 16. Additionally, the sentiment score correlates with and / or is indicative of a reinforcement learning reward configured to be used by the server 100 to train the reinforcement learning model 110 implemented on the server 100.

[0494] The system 500 further comprises a user-side sensor 50 and two user-side devices 52, 54, each having a user-side sensor 51, 53. Each of the user-side sensors 50, 51, 53 is configured to capture user-side sensor data. Furthermore, each of the user-side sensors 50, 51, 53 and / or each of the user-side devices 52, 54 is configured to transmit the sensor data to the user device 10 via a communication link, such as an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0495] Further, each of the user-side sensors 50, 51, 53 and / or each of the user-side devices 52, 54 is configured to transmit one or more sentiment scores to the server 100 via a communication link, such as an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0496] The user side devices 52, 54 may be, for example, a smart television (television), a smart speaker, a smart watch, a health monitor, an IoT (Internet of Things) device, and an aerosol generating device.

[0497] Further, the user-side sensors 50, 51, 53 may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a close-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0498] The control circuitry 20 of the user device 10 may be configured to determine an intermediate sentiment score based on the acquired reaction data, and to anonymize the intermediate sentiment score to generate a sentiment score, for example, based on normalizing the intermediate sentiment score with a reference sentiment score, so that privacy may be effectively protected.

[0499] The determined sentiment score may then be transmitted to the server 100 via the communications circuitry 18. Upon transmission, the sentiment score and / or the intermediate sentiment score may be removed from the user device 10. By determining the sentiment score on the user device, the computing load for training the reinforcement learning model 110 on the server 100 may be distributed to one or more user devices 10.

[0500] To determine the sentiment score (also referred to as a final sentiment score), sentiment scores from multiple sources may be used, such as sensors 24, 26, 28, user-side sensors 50, 51, 53, and / or user-side devices 52, 54. For example, sensor data from any of sensors 24, 26, 28 of user device 10 and / or user-side sensor data from any of user-side sensors 50, 51, 53 may be used to determine and / or derive reaction data, and based on the reaction data, a sentiment score may be determined by user device 10.

[0501] For example, one or more queries may be transmitted from the user device 10 to the server 100, and the server 10 may transmit one or more notifications 14, 16 to the user device 10. The camera 24 of the user device 10 may be used to record an image(s) of the user's facial reaction to the one or more notifications 14, 16 received from the server. The images of the user's facial reaction are not transmitted to the server and do not leave the user device 10. Instead, the user device 10 identifies the user's facial reaction using, for example, a machine learning classifier 23 and / or classifier circuitry 23 of the user device 10.

[0502] The control circuitry 20 of the user device 10 may then determine a reaction pattern, which may be indicative of an emotional expression, such as, for example, "happy" or "irritated," of the user of the user device 10. Based thereon, the user device 10 may determine a sentiment score and transmit the sentiment score to the server 100 for training the reinforcement learning model 110 of the server 100. Thus, it may be possible to personalize and / or improve the overall interactive communication between the server 100 and the user device 10.

[0503] Alternatively, or additionally, the one or more user-side sentiment scores may be determined by one or more of the user-side sensors 50, the user device 52, and the user device 54. The one or more user-side sentiment scores may then be transmitted to the user device 10 and / or the server 100.

[0504] Using multiple sentiment scores from multiple sources, such as sensors 24, 26, 28, 50, 51, 53, the sentiment scores transmitted to and used by server 100 may be verified, as exemplarily described below.

[0505] For example, one or more user-side sentiment scores may be determined by the user device 10 based on sensor data of any of the user-side sensors 50, 51, 53. Alternatively or additionally, one or more user-side sentiment scores may be determined by the user-side sensor 50 and / or the user-side device 52, 54 and transmitted to the user device 10. Alternatively or additionally, one or more sentiment scores, e.g., current sentiment scores, may be determined by the user device based on sensor data of one or more of the sensors 24, 26, 28. One or more user-side sentiment scores and / or one or more (current) sentiment scores may then be compared with each other, and a deviation between the scores may be determined, for example, for each pair of sentiment scores, and compared with a threshold for the deviation. If the threshold is exceeded or reached, one or more of the determined (current and / or user-side) sentiment scores may be discarded. If the threshold is not exceeded, one or more of the determined (current and / or user-side) sentiment scores may be used to determine a final sentiment score and / or sentiment score to be transmitted to the server 100. The final sentiment score may be determined based on selecting one of the determined (current and / or user-side) sentiment scores as the final sentiment score. Alternatively, multiple sentiment scores may be combined to generate the final sentiment score.

[0506] It should be noted that the validation described herein above may alternatively or additionally be performed on the server 100. To this end, a number of determined (current and / or user-side) sentiment scores may be transmitted to the server 100 via the user device 10 and / or via the user-side devices 52, 54 and compared with each other by the server 100.

[0507] The validation is exemplarily summarized below. User sentiment may be implicitly calculated and / or determined from multiple sources, such as, for example, the smart TV 52, the thermostat 50, the personal assistant 54, etc. Thus, it may be possible to train the reinforcement learning model 110 only with reactions that are the result of the model's 110 response or determination, as opposed to reactions based on external factors and / or one or more environmental parameters. For example, the server 100 may receive a negative reaction and / or sentiment from the user and provide notifications 14, 16 that result in a negative sentiment score. However, this negative sentiment score, sentiment and / or reaction may be the result of the temperature in the user's vicinity, not from the notifications 14, 16. By determining user sentiment scores from multiple sources, it is possible, for example, to use only user sentiment scores that are consistent with each other for training the reinforcement learning model 110, and to ignore those that are anomalous.

[0508] Another possibility is not to immediately classify a particular response or sentiment as correlated with a negative reaction or sentiment if at least two sentiment scores are inconsistent with each other (e.g., one positive and one negative). Instead, the same type or a similar notification 16 may be provided again to the user device 10, e.g., at a different time. The user's reaction or sentiment to the first notification 14 may then be confirmed based on the second notification 16 if there is a discrepancy and / or deviation between the at least two sentiment scores.

[0509] By using multiple sources and / or delaying training until responses, sentiments and / or sentiment scores are ascertained, the reinforcement learning model 110 may be trained more accurately, for example, when compared to traditional systems that are trained one at a time.

[0510] In the following, various examples and / or advantages of the present disclosure are described. The system 500 may involve an interactive app, which may be able to converse with the user, providing the user with interesting and relevant information and / or notifications 14, 16, for example, in the form of timely and relevant alerts (e.g., using a "chatbot" software application), in response to a query, or proactively (on demand). In general, the system 500 allows for dialogue personalization. In conventional systems, sentiment is usually derived from user queries and / or responses, e.g., user feedback explicitly provided by the user. This leads to a dependency on the user to provide this feedback, which may imply additional efforts for the user. Also, the feedback is very individual and is specifically provided by the user without considering the surroundings, the location (home, office, market, etc.), other family members present in the vicinity, or any other environmental parameters or factors.

[0511] However, the system 500 according to the present disclosure can provide a highly personalized interactive experience based on implicit user feedback that adapts to the user feedback and the user's environmental context, which may be considered as "implicit feedback", in which a sentiment score may be computed based on explicit feedback provided by the user, e.g., user responses, queries, NPS scores, ratings, etc., and based on implicit feedback, e.g., facial expression changes upon receiving one or more notifications 14, 16 from the server 100.

[0512] Additionally, environmental aspects, parameters, and / or factors of the user may be taken into account, such as location (home, office, market, etc.), temperature, lighting, time of day, weather, other family members present in the vicinity, etc. Thus, the sentiment score may be used to train the reinforcement learning model 110 to adapt the conversation and / or interactive communication according to the user and / or the user's environment.

[0513] Alternatively or additionally, the conversation history may be taken into account. For example, the user feedback may be considered in the context of the user's reactions to previous conversations. Thus, it may be possible to ignore sudden changes in the user feedback and / or the determined sentiment score that are due to external factors and / or environmental parameters and that may be, for example, unrelated to the notifications 14, 16.

[0514] Additionally, privacy-preserving interactions may be provided by the system 500. This may be achieved, for example, by adding a privacy layer to the server 100 and / or the user device 10, for example, in that no private data is transmitted from the user device 10 to the server 100.

[0515] From the perspective of the server 100, the conversation history may be adapted in a privacy-preserving manner as a sentiment score to adapt and / or personalize the user conversation, where the sentiment score does not contain private or personal data of the user. With respect to the reinforcement learning model 110, the reinforcement learning reward and / or sentiment score determination may be distributed, thereby preserving user privacy while providing an accurate representation of the reward function.

[0516] Further examples of the present disclosure are described below. The one or more user side devices 52, 54 can be, for example, a (standalone) camera, a microphone, a thermostat, a smartwatch, etc. The one or more sensors 13, 24, 26, 28 of the user device 10 can be, for example, a camera, a microphone, an accelerometer embedded in a mobile device, for example, hosting an app for interactive communication on the user device 10.

[0517] The user-side sentiment score may be computed, for example, based on audio, visual, and / or textual feedback captured by one or more user-side devices 52, 54 and / or one or more user-side sensors 50, 51, 53. The privacy sensitivity level and / or privacy level, e.g., low, medium, or high, may be defined by the user on the user device. The privacy level may correspond to different aspects, e.g., places, people, health, activities, and / or may indicate one or more features in one or more of the reaction data, sensor data, and user-side sensor data, which one or more features are to be operated on and / or removed from one or more of the reaction data, sensor data, and user-side sensor data for and / or before determining the (final) sentiment score.

[0518] A t , V t , and T t Let t denote the captured audio, visual, and text feedback during time t.

[0519] For example, V t may correspond to a video frame of a user with family members in the background. In this case, a "low" privacy level for "people" would correspond to keeping the frame or image intact. A "medium" privacy level would correspond to blurring the person's face. A "high" privacy level would correspond to cropping the image to completely remove the person from the frame.

[0520] Similarly, T tLet correspond to a text response provided by a user, e.g., "I'm going on a business trip to Krakow tomorrow." In this case, a "low" privacy level for "location" might correspond to keeping the response as is. A "medium" privacy level would correspond to abstracting "Krakow" to "somewhere in Europe" in the response. A "high" privacy level would correspond to removing the destination completely from the response: "I'm going on a business trip tomorrow."

[0521] Thus, based on the defined privacy level for one or more of the feature groups, one or more processing operations for processing one or more of the reaction data, sensor data, and user-side sensor data may be selected by the user device 10 for one or more features defined by the one or more feature groups.

[0522] Furthermore, P(A t ), P(V t ), and P(T t Let P(A) denote the respective captured sensor data and / or user-side sensor data with privacy protection applied by the user-side sensors 50, 51, 53 and / or the user-side devices 52, 54 according to the user-specified privacy level setting. Then, the user-side sensors 50, 51, 53 and / or the user-side devices 52, 54 calculate P(A) t ), P(V t ), and P(T t ) with the user device 10, for example, with an app on the user device 10.

[0523] Referring to the user device 10, At =f s (P(A t ))、s Vt =f s (P(V t ))、s Tt =f s (P(T tLet (a) denote the user-side sentiment score computed independently based on the respective sensory data or feedback. The (final) sentiment score calculation can be considered as a classifier that outputs a value between a minimum and a maximum, e.g., 1-10.

[0524] The user device 10 then aggregates the above (independently computed) sentiment scores to produce a consolidated, or final, sentiment S t The aggregation function can be, for example, a weighted average: t = 1 / 3 × [(w A ×s At )+(w V ×s Vt )+(w T ×s Tt )], w i indicates weighting.

[0525] Two or more sentiment scores At , s Vt , and s Tt If there is a significant discrepancy between, e.g., s Vt = 9, but s Tt = 3, different strategies can be applied to integrate them.

[0526] For example, a feedback cycle can be ignored when there is excessive discrepancy between different sentiment scores. This is because the aggregate user sentiment S t This can be indicated by assigning a value of 0 to

[0527] Alternatively, a higher weighting or weighting may be applied to, for example, explicit versus implicit feedback, e.g., s Tt is computed based on user type responses or user inputs, while s Vt If s is computed based on the background frame using the sensor side device 52, 54, a higher weighting is assigned to s TtFor example, a user may be laughing with a child in a snapshot, but from the text / voice response it appears that the user is stressed. Hence, the "stressed" sentiment score may be prioritized by assigning a higher weight to it.

[0528] The output of the user device 10, or an app running on it, is the (aggregated) final sentiment S t Alternatively, the server 100 may determine the sentiment score. In addition, the server 100 may provide the sentiment score based on, for example, an explicit user response or query provided by the user device, {P(A t ), P(V t ), P(T t )} is set in the user device, the user-side sensors 50, 51, 53, and / or the user-side devices 52, 54. For example, in the case of a chat conversation, according to the "location"-"medium" privacy level, the user device 10 may receive the S along with the user response "I'm going on a business trip to Europe tomorrow." t may be sent to the server 100.

[0529] Additionally, the server 100 may be configured to compute, calculate, and / or derive a reinforcement learning reward based on one or more sentiment scores, e.g., a final sentiment score, as exemplarily described below.

[0530] To train the reinforcement learning model 110, two functions of the reinforcement learning (RL) engine or model 110 may be considered: a reward function and a reinforcement learning (RL) agent policy, which together may regulate the personalization of the content of the RL engine or model 110. The inventory of categorized content may be considered, for example, in a knowledge base, and the server 100 or app may provide recommendations, notifications 14, 16, and / or (chat) responses, which may be grouped into categories related to the user's interests, e.g., travel, entertainment, health, places, etc.

[0531] Determined user sentiment S t Based on the last RL engine (measure a t ) RL rewards corresponding to recommendations, notifications 14, 16, and / or responses r t The reward function f r can be given as:

[0532] r t =f r (S t ), where function f r The logic can be given as follows: (S t =0), this learning loop and / or sentiment score may be ignored due to inconsistencies in the sentiment scores, as described herein above.

[0533] According to history normalization:S t For = [1-10], S t The value is the reward value r t Before assigning it to , it may be further normalized based on the historical background. The historical background is determined as follows:

[0534] For continuous conversations, the current sentiment score S t To normalize the effect, the SC t-1 For example, if the conversation sentiment score has already dropped, the low current sentiment S t can be assumed not to be the failure of the last measure alone, so that the reward r t can be calibrated accordingly. On the other hand, the declining sentiment curve SC t-1 Considering this, high sentiment S t would imply that the last action had a very positive impact on the user, and thus, its corresponding reward may be further boosted and / or increased.

[0535] In the case of special recommendations, a similar normalization logic can be applied, where the current sentiment S t can be calibrated according to previously received feedback (sentiment scores) for recommendations of the same category as that of the last measure.

[0536] For delayed rewards, for both continuous conversations and special recommendations, the RL reward for the penultimate m-th action is [a t , a t-1 , …, a t-m Another strategy may also be applied, which may be combined and applied (retroactively) to [ ]. For example, given the current sentiment S for a recommendation in the category “travel” to which a user is known to have responded very positively in the past (to other “travel” recommendations), t When is low, the delayed reward strategy simply increases the current (a t , S t ) and try another recommendation in the same category, t and a t+1 The goal is to provide instructions to the RL agent (policy) to “verify” the user sentiment score before updating the rewards for both the

[0537] Thus, the server 100 may be configured to determine and / or compute a reinforcement learning reward based on determining a trend of a plurality and / or series of previous sentiment scores, e.g., a plurality and / or series of previously determined final sentiment scores determined at least in part based on one or more notifications provided to a user. Further, a weighting may be determined based on the determined trend of the plurality and / or series of previous sentiment scores, and a reinforcement learning reward may be determined based on the determined weighting and the final sentiment score. Alternatively, or additionally, the final sentiment score may be compared to a trend of the plurality and / or series of previous sentiment scores, and a reinforcement learning reward may be determined based on the comparison. Alternatively, or additionally, a reinforcement learning reward may be determined based on a plurality of previously determined reinforcement learning rewards.

[0538] Regarding the RL agent policy function, a policy (1-p) can be considered along the lines of an epsilon-leading strategy, where the agent policy is to attempt to "search" with a (configurable) probability p. Two adaptations can be envisaged to accommodate reward function strategies (discussed above), e.g., "delayed reward".

[0539] In the delayed reward case, the RL agent does not apply policy (1-p) to select the next best action, but rather chooses the last action a t+1 ~a t and may provide "similar" recommendations.

[0540] For hierarchical policies, given a categorized content catalog, a hierarchical (1-p) policy may be applied, where the policy may first be used to select a “category”: {travel, entertainment, health, etc.}, and then recommendations within the selected category C may be assigned a usage probability q C :(1-q C ) can be applied using p and q C The policy utilization probability values ​​of can be dynamically adapted based on the coverage of recommendations in each category with known rewards.

[0541] 4 shows a flowchart illustrating a method for interactive communication. The interactive communication may be implemented by, for example, the system 500 described with reference to FIGS. 1 to 3 and / or according to any of the first to twenty-fifth aspects of the present disclosure.

[0542] The left column of Fig. 4 shows steps performed by one or more user-side sensors 50, 51, 53 and / or one or more user-side devices 52, 54. The center column of Fig. 4 shows steps performed by the user device 10. The right column of Fig. 4 shows steps performed by the server 100.

[0543] It should be noted that at least some of the steps of the method of FIG. 4 may be performed simultaneously, and some steps may be performed sequentially.

[0544] In step S1', a user input is received at the user device 10, for example via the user interface 12. The user input may refer to and / or indicate a query, for example a question. The query is further transmitted to the server 100 in step S1' and received by the server 100 in step S1''.

[0545] Furthermore, in step S1″, the server 100 determines at least one notification 14, 16, for example in the form of a response to a query or question. Thus, a notification provided in response to a query or question provided by the user device 10 may be considered a response notification.

[0546] In step S2'', at least one notification 14, 16 is transmitted to the user device 10 and provided on the user device 10, for example displayed on the user interface 12, in step S2'.

[0547] Further, in step S3', reaction data is acquired by the user device 10, for example using sensor data from one or more sensors 24, 26, 28, and one or more (current) sentiment scores are computed based on the reaction data.

[0548] Further, in step S1, user-side sensor data is captured by one or more user-side sensors 50, 51, 53 and / or on one or more user-side devices 52, .

[0549] In step S2, one or more user-side sentiment scores are determined and transmitted to the user device 10 in step S3.

[0550] In step S4', one or more user-side sentiment scores are received at the user device 10 and compared with one or more (current) sentiment scores determined by the user device, wherein a final sentiment score may be determined in S4'.

[0551] In step S5', at least one of the final sentiment score, the one or more (current) sentiment scores, and the one or more user-side sentiment scores are transmitted to the server 100 which receives one or more of these sentiment scores in step S3''.

[0552] In step S4'', the reinforcement learning ("RL") model 110 of the server 100 is trained based on at least one of the final sentiment score, the one or more (current) sentiment scores, and the one or more user-side sentiment scores.

[0553] Further, in step S5″, a further notification 14, 16 may be sent to the user device 10 or the server 100 may wait for a further query from the user device 10. The further notification may be a response notification provided in response to the further query from the user device 10.

[0554] Various modifications to the method of Figure 4 are possible: for example, the user-side sentiment score may be transmitted to the server 100 and / or the (current) sentiment score may be transmitted to the server 100, which may then determine a final sentiment score.

[0555] Furthermore, one or more of the final sentiment score, the (current) sentiment score, and the user-side sentiment score may be verified on the user device 10 and / or the server 100 based on comparing at least two of these sentiment scores, as described herein above and in more detail herein below.

[0556] Fig. 5 shows a flow chart illustrating a method for interactive communication of a user device 10 with a server 100. In particular, the method illustrated in Fig. 5 may correspond to a method according to the first aspect of the present disclosure. The method in Fig. 5 may refer to a method of operating a user device 10, for example, a user device 10 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0557] Step S1 involves providing, on the user device 10, a notification 14, 16 to a user of the user device 10.

[0558] Step S2 involves obtaining reaction data indicative of the user's reaction to the notification 14,16.

[0559] Step S3 includes determining a sentiment score for transmission to the server 100 based on the obtained reaction data, the sentiment score indicating the user's sentiment in reaction to the notification 14,16.

[0560] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0561] Fig. 6 shows a flow chart illustrating a method for interactive communication of a user device 10 with a server 100. In particular, the method illustrated in Fig. 6 may correspond to a method according to the fourth aspect of the present disclosure. The method in Fig. 6 may refer to a method of operating a user device 10, for example, a user device 10 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0562] In step S1, one or more notifications 14, 16 are provided to a user on the user device 10.

[0563] Step S2 involves obtaining reaction data indicative of one or more reactions of the user to the one or more notifications 14,16.

[0564] Step S3 includes determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores being indicative of a user's sentiment in reaction to the one or more notifications 14,16.

[0565] In step S4, a final sentiment score is determined for transmission to the server 100 based on comparing at least two of the plurality of sentiment scores to each other, the final sentiment score usable by the server 100 to train an artificial intelligence module 110 implemented on the server 100.

[0566] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0567] Fig. 7 shows a flow chart illustrating a method for interactive communication of the server 100 with the user device 10. In particular, the method illustrated in Fig. 7 may correspond to a method according to a seventh aspect of the present disclosure. The method in Fig. 7 may refer to a method of operating the server 100, for example, the server 100 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0568] Step S1 involves transmitting a notification 14, 16 from the server 100 to the user device 10.

[0569] Step S2 involves receiving, by the server 100, the sentiment score, which is correlated with a reinforcement learning reward for training a reinforcement learning model 110 implemented on the server 100.

[0570] Step S3 involves training a reinforcement learning ("RL") model 110 implemented on the server 100 based on the received sentiment scores.

[0571] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0572] Fig. 8 shows a flowchart illustrating a method for interactive communication of the server 100 with the user device 10. In particular, the method illustrated in Fig. 8 may correspond to a method according to a tenth aspect of the present disclosure. The method in Fig. 8 may refer to a method of operating the server 100, for example, the server 100 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0573] Step S1 involves transmitting one or more notifications 14, 16 from the server 100 to the user device 10.

[0574] Step S2 includes receiving, by the server 100, a plurality of sentiment scores, each of the plurality of sentiment scores correlated with a reinforcement learning reward for training a reinforcement learning model 110 implemented on the server 100.

[0575] Step S3 includes training a reinforcement learning ("RL") model 110 implemented on the server 100 based on comparing at least two of the plurality of sentiment scores with each other.

[0576] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0577] Fig. 9 shows a flowchart illustrating a method between the user device 10 and the server 100. In particular, the method illustrated in Fig. 9 may correspond to a method according to a thirteenth aspect of the present disclosure. The method in Fig. 9 may refer to a method of operating a system 500, for example, the server 500 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0578] Step S1 involves transmitting a notification 14, 16 from the server 100 to the user device 10.

[0579] Step S2 involves providing, on the user device 10, a notification 14, 16 to a user of the user device 10.

[0580] Step S3 involves obtaining, using the user device 10 and / or using at least one user side device 52,54, reaction data indicative of a user's reaction to the notification 14,16.

[0581] Step S4 includes determining, using the user device 10 and / or using at least one user-side device 52, 54, a sentiment score based on the acquired reaction data, the sentiment score indicating the user's sentiment in reaction to the notification 14, 16, and the sentiment score correlating with a reinforcement learning reward for training the reinforcement learning model 110 implemented on the server 100.

[0582] Step S5 involves transmitting the determined sentiment score from the user device 10 and / or from at least one user side device 52, 54 to the server 100.

[0583] Step S6 involves receiving, by the server 100, the sentiment score.

[0584] Step S7 involves training a reinforcement learning (“RL”) model 110 implemented on the server 100 based on the sentiment scores received by the server 100.

[0585] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0586] Fig. 10 shows a flowchart illustrating a method between the user device 10 and the server 100. In particular, the method illustrated in Fig. 9 may correspond to a method according to the fourteenth aspect of the present disclosure. The method in Fig. 10 may refer to a method of operating a system 500, for example, the server 500 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0587] Step S1 involves transmitting one or more notifications 14, 16 from the server 100 to the user device 10.

[0588] Step S2 involves providing, on the user device 10, one or more notifications 14, 16 to a user of the user device 10.

[0589] Step S3 involves obtaining, using the user device 10 and / or using at least one user side device 52,54, reaction data indicative of one or more reactions of the user to the one or more notifications 14,16.

[0590] Step S4 includes determining, using the user device 10 and / or using at least one user-side device 52, 54, a plurality of sentiment scores based on the acquired reaction data, each of the plurality of sentiment scores being correlated with a reinforcement learning reward for training the reinforcement learning model 110 implemented on the server 100.

[0591] Step S5 includes transmitting at least one of the plurality of sentiment scores from the user device 10 and / or from at least one user side device 52, 54 to the server 100.

[0592] Step S6 includes receiving, by the server 100, at least one of the plurality of sentiment scores.

[0593] Step S7 includes training a reinforcement learning (“RL”) model 110 implemented on the server 100 based on at least one of the sentiment scores received by the server 100.

[0594] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0595] For the purposes of this specification and the appended claims, unless otherwise indicated, all numbers expressing amounts, quantities, percentages, and the like should be understood in all instances as being modified by the term "about." Also, all ranges include the maximum and minimum points disclosed, and include any intermediate ranges therein, which may or may not be specifically recited herein. Thus, in this context, the number A is understood as A±20%. Within this context, the number A may be considered to include values ​​that are within the general standard error for the measurement of the property that the number A modifies. The number A may deviate by the percentages recited above, provided that in some cases, as used in the appended claims, the amount by which A deviates does not substantially affect the basic and novel properties of the claimed invention. Also, all ranges include the maximum and minimum points disclosed, and include any intermediate ranges therein, which may or may not be specifically recited herein.

[0596] While the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description is illustrative or representative and not restrictive, and the present invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art, from a study of the drawings, the disclosure, and the appended claims, and can be practiced in the practice of the claims.

[0597] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims are not to be construed as limiting their scope.

Claims

1. 1. A computer-implemented method for interactive communication of a user device with a server, the method comprising: providing, on the user device, one or more notifications to a user of the user device; obtaining reaction data indicative of one or more reactions of the user to the one or more notifications; determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; determining a final sentiment score for transmission to the server based on comparing at least two of the plurality of sentiment scores to one another, wherein the final sentiment score is usable by the server to train a reinforcement learning model implemented on the server.

2. The method of claim 1 , wherein at least one of the plurality of sentiment scores indicates the user's sentiment in response to at least one environmental parameter related to the user's environment that affects the user's sentiment.

3. The method according to claim 1 , wherein at least two of the plurality of sentiment scores indicate the sentiment of the user in response to at least two notifications received from the server.

4. The method of claim 1 , wherein the final sentiment score is determined based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

5. 5. The method of claim 1, further comprising: receiving, using the user device, at least one of the plurality of sentiment scores from at least one user-side device communicatively coupled to the user device.

6. further receiving, using the user device, user-side sensor data from at least one user-side device; and computing, with the user device, at least one of the plurality of sentiment scores based on deriving at least a portion of the reaction data or further reaction data from the received user-side sensor data.

7. The reaction data obtained is receiving, with the user device, user-side sensor data from at least one user-side sensor communicatively coupleable to the user device; The method of any preceding claim, comprising deriving the response data from received user-side sensor data of at least one user-side sensor using the user device.

8. 8. The method of claim 7, wherein the at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, an IoT device, and an electric aerosol generating device.

9. The reaction data obtained is capturing sensor data using at least one sensor of the user device; and deriving, using the user device, the response data from the captured sensor data of the at least one sensor of the user device.

10. providing one or more notifications includes receiving at the user device a first notification and a second notification, the second notification substantially matching or coinciding with the first notification; acquiring reaction data includes acquiring first reaction data indicating a first reaction of the user to the first notification, and acquiring second reaction data indicating a second reaction of the user to the second notification; determining the plurality of sentiments includes determining a first sentiment score based on the first response data and determining a second sentiment score based on the second response data. The method according to any one of claims 1 to 9.

11. Determining the final sentiment score includes: determining a deviation between at least two of the plurality of sentiment scores; comparing the determined deviation to a threshold value for the deviation; Optionally, the method further comprises discarding at least one of the at least two sentiment scores when the determined deviation reaches or exceeds the threshold value for the deviation.

12. and / or further preventing the final sentiment score from being transmitted to the server when the determined deviation between the at least two sentiment scores reaches or exceeds the threshold value for the deviation. and transmitting the final sentiment score to the server only if the at least two sentiment scores substantially agree with one another.

13. A computer program comprising instructions which, when executed on one or more processors of said user device, cause said user device to perform the steps of the method of any of claims 1 to 12.

14. A non-transitory computer readable medium storing the computer program of claim 13.

15. A user device for interactive communication with a server, said user device being configured to perform the steps of the method according to any one of claims 1 to 12.

Images