Method and apparatus for interactive and privacy-preserving communication between a server and a user device

Abstract

A method and apparatus for interactive communication between a user device and a server are described, wherein a notification is provided to a user on the user device and reaction data indicative of the user's reaction to the notification is acquired. Further, a sentiment score is determined for transmission to the server based on the acquired reaction data, the sentiment score indicative of the user's sentiment in reaction to the notification. Further, the sentiment score is usable by the server to improve interactive communication between the server and the user device.

Description

[Technical Field]

[0001] The present disclosure generally relates to the field of interactive communication between a server and a user device. In particular, the present disclosure relates to a computer-implemented method for interactive communication of a user device with a server, a computer-implemented method for interactive communication of a server with a user device, and a computer-implemented method for interactive communication between a server and a user device. The present disclosure further relates to a user device for interactive communication with a server, the use of such a user device, a server for interactive communication with a user device, and a system including such a user device and such a server. Furthermore, the present disclosure relates to one or more computer programs and one or more computer-readable media.

[0002] In exemplary interactive communications, a query received from a user device may be processed by one or more servers, for example, using a natural language processing (NLP) engine or any other engine configured to process queries. The query may be sent or transmitted from the user device to the one or more servers via a communication link or data connection between the user device and the one or more servers. The query may then be processed by the one or more servers, and one or more notifications may be provided to the user device in response. As an example, such a query may include or indicate one or more of a user's text input via a user interface of the user device, a verbal input via a microphone of the user device, and a video-based input via a camera of the user device. Furthermore, the one or more notifications provided by the one or more servers in response to the query may include, for example, at least one of text, voice, an audio message, a message, an alert, an image, and a video.

[0003] Therein, identifying or selecting a notification that is an appropriate response to a query received from a user device can be a challenging task. To this end, for example, a deep reinforcement learning model or a reinforcement learning model can be implemented on one or more servers, and feedback from users regarding notifications provided in response to queries can be used as a reward to improve the reinforcement learning model with respect to providing notifications to queries with more appropriate, accurate, and / or correct answers or content.

[0004] However, the process of improving the reinforcement learning model may require a significant amount of computing power on one or more servers. Furthermore, feedback from users may contain personal information about the users, which may raise data protection concerns on the user side and require additional security measures on the server side.

[0005] It may therefore be desirable to provide an improved method and apparatus for interactive communication between a server and a user device. This is achieved by the subject matter of the independent claims, with further embodiments incorporated in the dependent claims and the description below.

[0006] Various aspects and embodiments and examples of these aspects of the present disclosure are described below. It should be noted that various aspects of the present disclosure relate, at least in part, primarily, or entirely, to one or more of at least one server, at least one user device, at least one user-side device, at least one user-side sensor, and at least one system including one or more of such components or devices. In other words, the present invention can be considered to be distributed among one or more of at least one server, at least one user device, at least one user-side device, at least one user-side sensor, and at least one system including one or more of such components or devices.

[0007] However, it is emphasized that any feature, step, function, element, technical effect and / or advantage described herein above and below with reference to one aspect of the present disclosure applies to any other aspect of the present disclosure as well, as described herein above and below.

[0008] According to a first aspect of the present disclosure, a computer-implemented method for interactive communication of a user device with a server is provided. The method according to the first aspect may alternatively or additionally relate to a computer-implemented method for operating a user device, e.g., for interactive communication with a server, and optionally for operating one or more user-side devices. The method includes: on the user device, providing a notification to a user of the user device; - obtaining reaction data indicative of a user's reaction to the notification; - determining a sentiment score for transmission to the server based on the obtained reaction data, the sentiment score indicating the user's sentiment in reaction to the notification.

[0009] As described and discussed in more detail below, determining a sentiment score for a transmission to a server on a user device may significantly improve interactive communication between the server and the user device, or between the server and a user of the user device. In particular, determining the sentiment score on the user device may significantly reduce the computing resources used by the server for interactive communication. Alternatively, or in addition, a process or method for selecting, identifying, and / or generating notifications on the server (or an engine or module configured to do so) may be improved in terms of efficiency, quality, and accuracy. Alternatively, or in addition, determining the sentiment score on the user device may protect the privacy or anonymity of the user of the user device.

[0010] In the context of the present disclosure, interactive communication between a server and a user device may include, for example, one or more notifications sent or transmitted from the server to the user device. Alternatively, or additionally, interactive communication between a server and a user device may include, for example, one or more queries sent or transmitted from the user device to the server in response to one or more user inputs at the user device. Therein, one or more notifications may be sent or transmitted from the server to the user device in response to the one or more queries. Alternatively, or additionally, one or more notifications may be sent or transmitted from the server to the user device proactively or automatically. For example, the one or more notifications or at least some of the notifications may be time-triggered and / or event-triggered.

[0011] Furthermore, a notification provided on a user device may refer to a notification having or including content that may, for example, attract a user's attention via the user device. For example, a notification may include one or more of at least one letter, at least one symbol, at least one number, text, voice, an audio message, a question, an answer to a question, a statement, a request, a message, a text message, at least one icon, an alarm, an audio alarm, an optical alarm, at least one image, and a video. Generally, a notification may trigger and / or be configured to trigger a user response.

[0012] Similarly, in the context of the present disclosure, a query may include one or more of at least one letter, at least one symbol, at least one number, text, a rating, a question, an answer to a question, a statement, a request, a voice, an audio message, a message, a text message, at least one icon, at least one image, and a video. For example, a query may include text input received at a user device via a user interface, verbal input received at a user device via a microphone, and / or video-based input received via a camera of the user device.

[0013] In a non-limiting example, the interactive communication between the user device and the server may be based on Frequently Asked Questions ("FAQ"), where a query may relate to or include a user question transmitted to the server via the user device, and the question or query may be answered using one or more notifications of the server.

[0014] Furthermore, a user's reaction to a notification may be or refer to, for example, user movements, user gestures, user movements, user mimicry, user facial expressions, head movements, emotions, emotional expressions, head shaking, nodding, smiling, mouth movements, user skin color changes, skin color changes, verbal responses, acoustic responses, and physical responses such as goose skin, goose bumps, shaking body parts, waving hands, etc. Those skilled in the art will understand that these reactions can be sensed using readily available systems. For example, various software packages are provided for detecting emotions and emotional expressions from image data. One example of this software is the face detection package provided by Microsoft Azure™.

[0015] In the context of the present disclosure, the term "response data" should be understood or interpreted broadly. For example, response data may include or refer to data that contains information about, indicates, represents, and / or is descriptive of one or more reactions of a user to one or more notifications. Alternatively, or additionally, response data may include or refer to data that contains information about, indicates, represents, and / or is descriptive of one or more reactions of a user, where the reactions may not be related to one or more notifications and / or may not be caused or triggered by one or more notifications.

[0016] As an example, the response data may include or refer to sensor data of a sensor configured to sense one or more responses of a user. Alternatively, or additionally, the response data may refer to or include combined sensor data, such as sensor data from multiple sensors or independent sensors. Alternatively, or additionally, the response data may refer to or include data derived from sensor data of one or more sensors.

[0017] As discussed further below, in at least some embodiments, the reaction data may be obtained based on or using one or more sensors of the user device. Alternatively, or additionally, the reaction data may be obtained based on or using one or more user-side sensors of one or more user-side devices.

[0018] Generally, a user's reaction may be associated with, indicative of, and / or an expression of a user's sentiment in reaction and / or response to a notification provided on a user device. Therein, a user's sentiment may refer to or be indicative of a user's feelings, moods, and / or emotions, for example. Thus, by obtaining reaction data that may indicate, describe, and / or represent a user's reaction to a notification, the user's sentiment may be determined and / or derived from the reaction data.

[0019] In the context of the present disclosure, a sentiment score may refer to or be a scale or indicator, e.g., a numerical scale, indicator, and / or score, indicating a user's sentiment in response to a notification provided on a user device. Thus, a user's sentiment or actual sentiment may be encoded and / or converted into a sentiment score. In other words, the sentiment score may be determined, for example, based on the reaction data, based on converting and / or encoding the user's sentiment into a sentiment score. Alternatively, or additionally, the sentiment score may be derived from the reaction data, for example, based on processing at least some or a portion of the reaction data. Alternatively, or additionally, determining the sentiment score may include, for example, calculating and / or computing the sentiment score based on the reaction data.

[0020] Furthermore, the term sentiment score “for transmission to a server” may mean that the sentiment score may be configured for transmission and / or may be configured to be transmitted from a user device to a server.

[0021] Because the sentiment score indicates the user's sentiment in response to the notification, the sentiment score may function as and / or indicate the user's feedback on the notification. Thus, the sentiment score and / or the feedback indicated therewith may reflect the accuracy, level of appropriateness, and / or quality of the notification. Because the notification may be transmitted to the user device, for example, in response to one or more queries transmitted from the user device to a server, the sentiment score and / or the feedback indicated therewith may alternatively or additionally reflect the accuracy, level of appropriateness, and / or quality of the notification in response to the one or more queries.

[0022] Additionally, the determined sentiment score may be used by the server, may be usable by the server, and / or may be configured to be used by the server to improve, personalize, adapt, and / or modify interactive communications between the server and the user device. For example, the sentiment score may be usable by the server to modify, personalize, and / or adapt an engine or module of the server configured to select, identify, determine, and / or generate notifications, e.g., notifications responsive to one or more queries from the user device.

[0023] In the context of this disclosure, a server may comprise at least one computing device, including, for example, one or more processors. However, it should be noted that a server may refer to a server system, a computer system, a server network, a computing network, a cloud computing network, etc. Thus, any reference to "a server" or "the server" should be construed as including multiple servers.

[0024] Generally, the server may comprise or include an engine or module for identifying, selecting, and / or generating notifications to be sent or transmitted to the user device, e.g., in response to one or more queries from the user device. The server may comprise, e.g., a natural language processing (“NLP”) module or engine configured to process, analyze, and / or resolve one or more queries received from the user device. The server may be further configured to determine one or more notifications to be transmitted to the user device, e.g., in a proactive, automated, time-triggered, and / or event-triggered manner, in response to the one or more queries.

[0025] Additionally, the server may comprise artificial intelligence (also referred to as a recommendation engine) that enables it to determine, identify, select, and / or generate a notification that may represent an appropriate or most appropriate notification, for example, from among or based on a plurality of potential notifications that may be transmitted to the user device. By way of example, the server may comprise a knowledge base that includes a plurality of notifications that may be transmitted to the user device. The artificial intelligence module of the server may be configured to determine, select, identify, and / or generate an appropriate or most appropriate notification from among or based on the plurality of notifications in the knowledge base. For example, one or more queries received from the user device may be classified using the artificial intelligence module of the server to determine, select, identify, and / or generate an appropriate or most appropriate notification from among or based on the plurality of notifications in the knowledge base.

[0026] For example, the sentiment score may be advantageously used by the server to train an artificial intelligence module to improve the quality, accuracy, level of appropriateness, and / or relevance of notifications selected by the server's artificial intelligence module in response to one or more queries from the user device. Thus, by using one or more sentiment scores to train the artificial intelligence module, it may be possible to continuously improve the artificial intelligence module over time.

[0027] Generally, an artificial intelligence module (and / or recommendation engine) may refer to a classifier arrangement or classifier circuitry of a server configured to classify inputs, e.g., one or more queries, and determine outputs, e.g., one or more notifications to be transmitted to a user device based on the classification. For example, the artificial intelligence module may include a neural network, may refer to a machine learning module, and / or may relate to a deep learning module. However, any other type of artificial intelligence module may be used.

[0028] By way of example, the server, the artificial intelligence module, and / or the recommendation engine may include a reinforcement learning model, and the sentiment score may be used by the server to train the server's reinforcement learning model. Generally, the reinforcement learning model may be configured to be trained based on maximizing and / or optimizing a reward function over time, for example, continuously and / or iteratively in multiple training steps. At each training step, a reinforcement learning reward (also referred to as a "reward") reflecting and / or indicative of the level of correctness or quality of the decisions of the reinforcement learning module performed may be provided or dispensed to the reinforcement learning model to penalize it when it makes an incorrect decision and reward it when it makes a correct decision.

[0029] Thus, the sentiment score may be advantageously used as a reinforcement learning reward. Alternatively, or additionally, a reinforcement learning reward may be derived from and / or determined based on the sentiment score, and the reinforcement learning reward may be used by or assigned on the server to train the server's reinforcement learning model. For example, the sentiment score may correlate with and / or be indicative of a reinforcement learning reward for training the server's reinforcement learning model.

[0030] Thus, based on the sentiment score, the server's reinforcement learning model and / or artificial intelligence module can be efficiently and accurately trained, thereby enabling efficient improvement of interactive communication between the server and the user device, for example, in terms of improving the level of accuracy, the level of appropriateness, the quality, and / or the relevance of one or more notifications selected, identified, and / or generated by the server in response to one or more queries from the user device.

[0031] Furthermore, interactions between the server and the user device may be personalized over time based on the sentiment score. Therefore, it may be possible to meet user demands while requiring minimal user effort to ensure a sufficient level of accuracy in providing correct notifications in responses from the user device. Such efforts may involve, for example, requesting explicit feedback from the user based on evaluating the correctness of notifications provided in response to queries. Thus, based on the sentiment score, interactions between the server and the user device may be improved in an automated manner while minimizing or eliminating user effort, such as explicit user feedback.

[0032] In the context of the present disclosure, a user device may refer to any device or apparatus configured to communicate with a server, for example, based on the exchange of data or information. Generally, a user device may be a handheld or portable device. Alternatively, a user device may be a standalone or fixedly installed device. By way of example, a user device may refer to a handheld, smartphone, personal computer ("PC"), tablet PC, laptop, or computer. A user device may be configured to receive input from a user, such as, for example, one or more queries, via a user interface of the user device.

[0033] The user device may include one or more processors for data processing. Additionally, the user device may include a data storage device for storing data, such as reaction data. The user device may have software instructions or computer programs stored in the data storage device that, when executed by the one or more processors, instruct the user device to perform one or more steps of the method according to the first aspect and / or any other aspect of the present disclosure. Alternatively, or additionally, an app or computer program may be stored on the user device configured to interactively communicate with the server, for example, based on transmitting one or more queries to the server and receiving one or more notifications in response thereto.

[0034] The user device may be configured to communicate with the server via a communication link or connection between the server and the user device, for example, using a communication protocol, standard, or technology. To this end, the user device may include communication circuitry coupleable with a corresponding communication arrangement of the server to enable data exchange between the user device and the server. By way of example, the user device may be coupleable with and / or configured to communicate with the server (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a radio connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0035] The method may further include transmitting the determined sentiment score to a server. In other words, the sentiment score may be transmitted from the user device to the server. In that, transmitting the sentiment score to the server may include providing and / or sending the sentiment score to the server. Optionally, transmitting the sentiment score to the server may include establishing a communication link or connection between the user device and the server. Optionally, transmitting the sentiment score may include confirming receipt of the sentiment score at the server, for example, based on a signal or confirmation transmitted by the server to the user device upon receipt of the sentiment score. By transmitting the sentiment score to the server, the sentiment score is provided to the server, thereby enabling the server to use the sentiment score, for example, to improve, modify, personalize, and / or adapt interactive communication between the server and the user device.

[0036] The sentiment score may be transmitted by the user device to the server in an automated manner. This may mean that user interaction with and / or user input on the user device is not required to transmit the sentiment score. Thus, determination of the sentiment score may trigger transmission of the sentiment score to the user device. Alternatively, or additionally, the sentiment score may be transmitted upon or in response to user input at the user device.

[0037] The sentiment score may be determined by the user device, for example, based on processing at least a portion of the reaction data using control circuitry of the user device. Alternatively or additionally, the sentiment score may be determined at and / or on at least one user-side device communicatively coupled or couplable with the user device. For example, the user device may instruct the user-side device to determine the sentiment score based on processing at least a portion of the reaction data. Alternatively or additionally, determining the sentiment score may include determining the sentiment score on the user device and providing the sentiment score to the user-side device, for example, for transmission to a server or further processing of the sentiment score. Alternatively or additionally, determining the sentiment score may include determining the sentiment score on the user-side device and providing the sentiment score to the user device, for example, for transmission to a server or further processing of the sentiment score.

[0038] Determining the sentiment score on the user device and / or user-side device may advantageously allow, for example, to reduce the computing resources used on the server, since the computing load of determining the sentiment score may be distributed to the user device and / or user-side device. The overall interactive communication between the server and the user device may then be significantly improved, for example, in terms of a more efficient and faster reaction of the server in response to queries from the user device. Apart from the effort and cost, for example, hardware costs and / or maintenance costs for the server may be advantageously reduced.

[0039] In the context of the present disclosure, a user-side device may refer to a device or apparatus located near, around, in the environment, and / or remotely from a user device. Alternatively or additionally, a user-side device may refer to a device or apparatus that is under at least partial control of a user of the user device. For example, a user device may be configured to transmit one or more control signals to a user-side device, and / or a user-side device may be configured to receive one or more control signals from a user device. Alternatively or additionally, a user-side device may refer to a device or apparatus that is connected to a user device via a common communication network, for example, the same local area network (LAN) as the user device.

[0040] The user-side device may be configured to communicate with the user device via a communication link or connection between the user-side device and the user device, for example, using a communication protocol, standard, or technology. To this end, the user-side device may include communication circuitry coupleable with corresponding communication circuitry of the user device to enable data exchange between the user-side device and the user-side device. As an example, the user-side device may be coupleable with and / or configured to communicate with the user device (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0041] Furthermore, the user-side device may be configured to communicate with the server via a communication link or connection between the user-side device and the server, for example, using a communication protocol, standard, or technology. To this end, the user-side device may include a communication circuit that can be coupled to a corresponding communication arrangement of the server to enable data exchange between the server and the user-side device. As an example, the user-side device may be coupled to and / or configured to communicate with the server (or vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol. The user-side device may be configured to communicate with the server via the user device.

[0042] The user-side device may be configured to sense, monitor, and / or detect its vicinity, environment, and / or surroundings. To this end, the user-side device may include one or more user-side sensors. The user-side device may be configured, for example, to capture and / or acquire at least a portion of the reaction data. Alternatively, or additionally, the user-side device may include a controller or control circuit configured to process data, for example, the reaction data and / or user-side sensor data acquired by one or more user-side sensors of the user-side device.

[0043] By way of example, the user-side device may refer to or be one or more of a smart TV, a smart speaker, a smart watch, a smart device health monitor, and an electric aerosol generating device. However, it should be noted that the present disclosure is not limited to the above-mentioned exemplary user-side devices.

[0044] The determined sentiment score may be transmitted from the user device to the server and / or from at least one user-side device communicatively coupled to the user device. As an example, the user device may instruct the user-side device, for example, upon determination of the sentiment score at the user device or the user-side device, to transmit the sentiment score to the server. Alternatively or additionally, transmitting the sentiment score to the server may include transmitting the sentiment score from the user device to the user-side device and transmitting the sentiment score from the user-side device to the server. Conversely, transmitting the sentiment score to the server may include transmitting the sentiment score from the user-side device to the user device and transmitting the sentiment score from the user device to the server.

[0045] The reaction data, or at least a portion thereof, may be acquired at and / or by the user device. Alternatively or additionally, the reaction data, or at least a portion thereof, may be acquired at and / or by at least one user-side device that is communicatively coupled or couplable with the user device. Alternatively or additionally, a portion of the reaction data may be acquired at the user device, and a further portion of the reaction data may be acquired at the at least one user-side device. Various portions of the reaction data may be further merged and / or combined, for example, on the user device and / or at least one user-side device. It should be noted that, alternatively or additionally, the reaction data may be acquired entirely at the user device and / or at the user-side device. In other words, the complete set of reaction data may be acquired at the user device and / or at the user-side device. Thus, for example, it may be possible to perform a plausibility check of the reaction data based on comparing reaction data acquired using the user device with reaction data acquired at the user-side device.

[0046] The method may further include receiving a query at the user device, wherein the notification is provided by the server in response to the query. In which the query may be received at the user device based on one or more user inputs, for example, via a user interface, a microphone, a camera, and / or a sensor of the user device. Alternatively, or additionally, the notification may be provided on the user device by the server in response to the query.

[0047] The method may further include transmitting the query received at the user device to a server, wherein transmitting the query may include transmitting the query to the server, for example, in the form of one or more data packages. Optionally, transmitting the query to the server may include establishing a communication link or connection between the user device and the server.

[0048] The query may include text input received at the user device via a user interface, verbal input received at the user device via a microphone, and / or video-based input received via a camera on the user device. As noted above, the query may include one or more of at least one letter, at least one symbol, at least one number, text, a question, an answer to a question, a statement, a request, voice, an audio message, a message, a text message, at least one icon, at least one image, and a video.

[0049] The notification may include one or more of at least one letter, at least one symbol, at least one number, text, voice, an audio message, a question, an answer to a question, a statement, a request, a message, a text message, at least one icon, an alarm, an audio alarm, an optical alarm, at least one image, and a video. However, it should be noted that the notification may be any other type of notification. The notification may include any suitable content.

[0050] The sentiment score may be an anonymized numerical measure (or score) indicative of a user's reaction to the notification. Alternatively, or additionally, the sentiment score may correlate with and / or be indicative of a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0051] Therein, the term "anonymized" may mean that the sentiment score does not contain, is devoid of, and / or lacks personal and / or private information about the user of the user device. Generally, the sentiment score may be considered or refer to a numerical measure of a user's reaction to a notification provided on the user device.

[0052] Additionally, the sentiment score may be measured on a scale that may be an absolute or relative value scale between a minimum sentiment score value and a maximum sentiment score value, and the actual value of the sentiment score may indicate and / or reflect the user's sentiment, current sentiment, and / or actual sentiment in response to the notification.

[0053] By way of example, the sentiment score may be measured on a scale between "-10" and "+10," where a sentiment score of "-10" may indicate a user's most negative reaction to the notification, a sentiment score of "0" may indicate a user's neutral reaction to the notification, and a sentiment score of "+10" may indicate a user's most positive reaction to the notification. However, it is emphasized that any other scale may be used for the sentiment score, including a scale with non-integer values ​​for the sentiment score.

[0054] Determining the sentiment score involves: determining an intermediate sentiment score based on the obtained response data; and anonymizing the intermediate sentiment score, thereby generating the sentiment score.

[0055]

[0013] In particular, the intermediate sentiment score may include information or data related to the user's personal information and / or personal data, while the sentiment score may not contain or be free of the user's personal information and / or personal data. Thus, the intermediate sentiment score may be considered an intermediate quantity or measure that may be determined, for example, temporarily determined, to determine the sentiment score. Generally, the intermediate sentiment score may be determined and anonymized on the user device and / or on at least one user-side device.

[0056] Generally, anonymizing the intermediate sentiment score to generate an anonymized score or measure advantageously protects the privacy or anonymity of the interactive communication between the server and the user device. This may mean that the user or any personal information or data related to the user is not transmitted from the user device to the server. On the other hand, the (anonymized) sentiment score, regardless of anonymization, may similarly be usable to improve and / or personalize the interactive communication between the user device and the server, for example, based on training an artificial intelligence module and / or a reinforcement learning model on the server based on the sentiment score, even if data such as personal information or reaction data is transmitted to the server. Thus, determining the sentiment score based on anonymizing the intermediate sentiment score may enable selectively providing the server with information or data that may be useful for improving the overall interactive communication, without providing the server with any personal information or data.

[0057] Additionally, users' privacy concerns may be respected in a comprehensive manner, which may reduce the likelihood of secure data relating to the user being compromised.

[0058] Furthermore, from the server's perspective, since none of the personal data or information may be processed by or stored on the server, the processing load and costs associated with security or safety-related measures may be effectively reduced. Therefore, no computing resources may be required for additional safety measures on the server, which may lead to more efficient and faster reactions of the server in response to queries from user devices. Also, the overall performance of the server may be improved, and the processing load and costs, e.g., hardware costs and / or maintenance costs, that may otherwise be required to provide comprehensive safety measures on the server may be effectively reduced.

[0059] Alternatively, by determining anonymized sentiment scores and transmitting the sentiment scores to a server to improve interactive communication, the amount or volume of data transmitted from the user device and / or user-side device to the server may be significantly reduced when compared to, for example, transmitting reaction data to the server and determining the sentiment scores on the server, which may further improve the overall performance of the server and improve the overall interactive communication between the server and the user device, for example, in terms of more efficient and faster reaction of the server in response to queries from the user device.

[0060] The sentiment score may be anonymized based on normalizing the intermediate sentiment score with a reference sentiment score (and / or reference sentiment score value). For example, the reference sentiment score may be selected depending on the scale on which the sentiment score is determined. For example, the reference sentiment score may refer to and / or indicate the user's most negative reaction to the notification, the user's neutral reaction to the notification, the user's most positive reaction to the notification, or any reaction of the user in between. Furthermore, normalizing the intermediate sentiment score may include converting the intermediate sentiment score to a sentiment score based on the reference sentiment score. For example, the intermediate sentiment score may be divided by or multiplied by the reference sentiment score. However, any other mathematical operation may be applied. Furthermore, it should be noted that the sentiment score may be given in absolute or relative values.

[0061] Optionally, further security measures and / or privacy protection schemes may be implemented on the user device and / or user-side device to protect the intermediate sentiment score and / or sentiment score, for example, secret sharing, encryption, and / or homomorphic encryption may be used on the user device and / or user-side device.

[0062] The method may further include removing the intermediate sentiment score from the user device upon anonymizing the intermediate sentiment score and / or upon generating the sentiment score. Alternatively, or additionally, the intermediate sentiment score may be removed from at least one user-side device. By removing the intermediate sentiment score from the user device and / or the at least one user device, the user's private or personal information may be further protected. In particular, removing the intermediate sentiment score may include deleting, overwriting, and / or destroying the intermediate sentiment score.

[0063] The method may further include removing the sentiment score from the user device when transmitting the sentiment score to the server. Alternatively, or additionally, the sentiment score may be removed from at least one user-side device when transmitting the sentiment score to the server. In other words, for example, transmission of the sentiment score from the user device and / or at least one user-side device to the server may trigger removal of the sentiment score from the user device and / or the user-side device. Thus, it can be ensured that no storage resources or memory are used or blocked by the sentiment score when it is being transmitted to the server.

[0064] Providing a notification on the user device may include displaying the notification on a user interface of the user device. Displaying the notification on the user device may ensure that the user's attention is directed to the notification. Alternatively, or additionally, providing a notification on the user device may include playing or outputting the notification via a microphone.

[0065] Obtaining the reaction data may include obtaining the reaction data from a data storage device of the user device. Alternatively, or additionally, obtaining the reaction data may include obtaining the reaction data from a data storage device or memory of one or more user-side devices. Alternatively, or additionally, obtaining the reaction data may include requesting, by the user device, the reaction data to be transmitted from one or more user-side devices to the user device. Alternatively, or additionally, obtaining the reaction data may include instructing, by the user device, one or more user-side devices to transmit the reaction data to the user device.

[0066] Obtaining the reaction data may include capturing sensor data using at least one sensor of the user device. In other words, the reaction data, or at least a portion thereof, may be obtained based on sensor data of one or more sensors of the user device.

[0067] The method may further include deriving, using the user device, response data from captured sensor data of at least one sensor of the user device. In this, deriving the response data may include processing the sensor data, for example, by control circuitry of the user device. Such processing may include one or more of filtering, transforming, and converting the captured sensor data, for example, to generate the response data. Alternatively or additionally, deriving the response data may include selecting at least a portion of the sensor data as the response data. Alternatively or additionally, deriving the response data may include using at least a portion of the sensor data as the response data. Alternatively or additionally, deriving the response data may include combining at least a portion of the sensor data with additional sensor data of additional sensors of the user device and / or with user-side sensor data of at least one user-side sensor.

[0068] The at least one sensor of the user device may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor. Such a sensor may advantageously enable detecting or sensing a user's reaction in response to the notification and, consequently, enabling reliable determination of reaction data indicative of the user's reaction. However, it should be noted that any other type of sensor of the user device may be used to capture sensor data and / or determine reaction data based thereon.

[0069] The response data may be acquired based on sensor data from multiple sensors of the user device, e.g., different types of sensors. Thus, acquiring the response data may include capturing sensor data using multiple sensors of the user device, wherein the sensor data from different sensors may be combined, merged, and / or fused to determine the response data. Thus, it may be possible to provide accurate and comprehensive response data that may accurately and comprehensively reflect a user's response to a notification and / or contain comprehensive and accurate information regarding the user's response to a notification. Alternatively, or additionally, using sensor data from multiple sensors to determine the response data may enable a plausibility check of the sensor data and / or the response data.

[0070] For example, first sensor data from a first sensor of a user device may be captured, and the first sensor data may be complemented, merged, combined, and / or fused with second sensor data from a second sensor of the user device to generate response data, where the first sensor may be different from the second sensor.

[0071] Obtaining the reaction data may include receiving user-side sensor data from at least one user-side sensor communicatively coupled to the user device using and / or in the user device. In other words, the reaction data, or at least a portion thereof, may be obtained based on user-side sensor data of one or more user-side sensors. Therein, "user-side sensor" may indicate or refer to a sensor of the user-side device. Furthermore, receiving the user-side sensor data may include obtaining the user-side sensor data from at least one user-side sensor. Alternatively or additionally, receiving the user-side sensor data may include instructing the at least one user-side sensor to transmit the user-side sensor data to the user device.

[0072] Generally, a user-side sensor may refer to a sensor located near, around, in the environment, and / or remotely from a user device. Alternatively or additionally, a user-side sensor may refer to a sensor coupled to, communicatively coupled to, couplable with, and / or configured to communicate with a user device. For example, a user-side sensor may be included within or part of a user-side device. Alternatively or additionally, a user-side sensor may be a standalone sensor. However, it should be noted that various or multiple different user-side sensors may be used to acquire response data, such as at least one user-side sensor included in at least one user-side device and at least one additional user-side sensor that may be a standalone user-side sensor. When a user-side sensor is included within a user-side device, communication between the user device and the user-side sensor (or user-side device) may be implemented via communication circuitry in the user-side device. If the user-side sensor is a standalone sensor, communication between the user-side sensor and the user device may be performed via corresponding communication circuitry of the user-side sensor, e.g., using a communication link, communication connection, and / or communication protocol, e.g., as described above with respect to the user-side device.

[0073] The method may further include deriving, using the user device, response data from the received user-side sensor data of the at least one user-side sensor. In this, deriving the response data may include, for example, processing the user-side sensor data by control circuitry of the user device. Such processing may include, for example, one or more of filtering, transforming, and converting the user-side sensor data to generate the response data. Alternatively or additionally, deriving the response data may include selecting at least a portion of the user-side sensor data as the response data. Alternatively or additionally, deriving the response data may include using at least a portion of the user-side sensor data as the response data. Alternatively or additionally, deriving the response data may include combining at least a portion of the user-side sensor data with additional user-side sensor data of additional user-side sensors and / or with sensor data of one or more sensors of the user device.

[0074] The at least one user-side sensor may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a short-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor. However, it should be noted that any other type of user-side sensor may be used.

[0075] The at least one user-side sensor may be included in at least one user-side device positioned in the vicinity of the user device, wherein the at least one user-side device is one or more of a smart television (television), a smart speaker, a smart watch, a health monitor, an IoT (Internet of Things) device, and an aerosol-generating device.

[0076] In the context of the present disclosure, an aerosol-generating device may refer to a device that interacts with an aerosol-forming substrate to generate an aerosol. The aerosol-forming substrate may be part of an aerosol-generating article, such as a smoking article. The aerosol-generating device may include one or more components used to supply energy from a power source to the aerosol-forming substrate to generate an aerosol. The aerosol-generating device and all its components may be portable and mobile. The heating elements therein may be provided in different shapes, sizes, and numbers. For example, the heating elements may be shaped as needles, pins, rods, or blades that can be inserted into the smoking article to contact the aerosol-forming substrate. The aerosol-generating device may include more than one heating element, and in the following description, reference to a heating element refers to one or more heating elements. The aerosol-generating device may also include an electronic circuit arranged to control the supply of current to the heating element to control its temperature. The aerosol-generating device may optionally include a means for sensing the temperature of the heating element.

[0077] The method may further include deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment that affects the user's sentiment. Therein, a sentiment score may be determined based on the at least one environmental parameter. Generally, by determining the at least one environmental parameter and determining a sentiment score based thereon, the user's one or more reactions to the at least one environmental parameter, to the user's environment, and / or to conditions in the user's environment may be distinguished and / or differentiated from the user's actual reaction in response to the notification. This may ensure that the sentiment score that may be used by the server to improve interactive communication preferably only reflects, or at least primarily indicates, the user's sentiment or reaction in direct response to the notification. Therefore, it may be possible to improve interactive communication in a more precise, specific, targeted, and goal-oriented manner. This may be particularly advantageous if the sentiment scores are used by the server to train the server's artificial intelligence modules and / or reinforcement learning models, as erroneous training may be effectively avoided or at least reduced to a minimum.

[0078] In the context of the present disclosure, the at least one environmental parameter indicative of the user's environment that influences the user's sentiment may refer to, for example, any factor or circumstance that overlaps or interferes with the user's reaction to the notification, influences the user's sentiment, and / or (potentially) triggers the user's reaction. Examples of such environmental parameters may be the temperature in the user's environment, the noise in the user's environment, family or friends present in the user's environment, the user's location, etc.

[0079] Illustrative examples for determining at least one environmental parameter and associated benefits are provided below. The sentiment score may be implicitly calculated from multiple sources, such as one or more sensors on the user device and / or one or more user-side sensors or devices, such as a smart TV, thermostat, or GPS sensor. Thus, it may be possible to validate the determined sentiment score to ensure that it at least primarily indicates the user's reaction to the notification. This may enable, for example, training a reinforcement learning model on the server based primarily or solely on reactions resulting from the server-provided notification and / or the reinforcement learning model's determination to select a particular notification, as opposed to reactions based on external factors. For example, the server may provide a notification to the user device, and a negative sentiment or reaction from the user may be determined based on one or more sensor data, such as sensor data or image data from the user device's camera. However, this negative sentiment or reaction from the user may be the result of low temperatures in the user's environment, noise in the user's environment, or any other environmental parameters that negatively affect the user's sentiment. Therefore, a user's negative sentiment or negative reaction may be caused not by the notification but by one or more environmental parameters. By determining the sentiment score based on the determined at least one environmental parameter, it can be determined whether the sentiment score primarily reflects the user's sentiment or reaction in direct consequence of the notification, or whether the user's reaction or sentiment is primarily caused by one or more environmental parameters. Therefore, the quality of the sentiment score can be improved by determining the sentiment score based on at least one environmental parameter.

[0080] The reaction data may be acquired by the user device based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor disposed in the environment of the user device. Using multiple sources to determine the sentiment score, such as one or more sensors of the user device and / or one or more user-side sensors, may enable checking the plausibility of the determined sentiment score. As a result, the quality of the sentiment score may be further improved.

[0081] Determining the sentiment score may include providing at least a portion of the reaction data to a classifier circuit of the user device. Alternatively, or additionally, the sentiment score may be determined based on classifying at least a portion of the reaction data with a classifier circuit of the user device. The classifier circuit may refer to, for example, an artificial intelligence module and / or a machine learning classifier, such as a neural network, of the user device, which may enable determining a user's reaction to determine the sentiment score.

[0082] The method is: capturing, with a camera of the user device, image data indicative of one or more images of a user of the user device; - determining response data based on the captured image data; determining a sentiment score based on providing at least a portion of the reaction data to a classifier circuit of the user device.

[0083] Thus, a camera on a user device may be used to acquire sensor data in the form of image data. However, the image data is not transmitted from the user device to a server; instead, determining a sentiment score may mean that an anonymized sentiment score may be transmitted to the server without the image data leaving the user device. Reaction data may be determined based on the sensor data, for example, by selecting one or more images from the image data. Furthermore, the sentiment score may then be determined based on processing the reaction data. In this regard, providing at least a portion of the reaction data to a classifier circuit may include providing at least a portion of the reaction data to the classifier circuit. The classifier circuit may reliably determine many different reactions of the user, allowing the sentiment score to be determined quickly and accurately.

[0084] Determining the sentiment score involves: determining, with classifier circuitry of the user device, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - deriving a sentiment score from the determined reaction pattern.

[0085] The emotional response patterns therein may reflect the user's reaction to the notification. Thus, by using the classifier circuit to determine the response patterns, the sentiment score may be determined with high accuracy and precision.

[0086] The sentiment score may be determined based on computing and / or calculating a current sentiment score with the user device and deriving the sentiment score from the computed current sentiment score.

[0087] In the context of this disclosure, a "current sentiment score" may refer to a sentiment score that is determined or computed locally on a user device. For example, the current sentiment score may be determined based on sensor data from one or more sensors of the user device. In particular, the current sentiment score may be determined based solely on sensor data from one or more sensors of the user device.

[0088] Determining the sentiment score may include receiving, using the user device, at least one user-side sentiment score from at least one user-side device communicatively coupleable to and / or communicatively coupled to the user device, and the sentiment score may be determined based on the user-side sentiment score received from the at least one user-side device.

[0089] In the context of this disclosure, a "user-side sentiment score" may refer to a sentiment score determined or computed based on user-side sensor data of one or more user-side sensors. The user-side sentiment score may be determined by a user device and / or by one or more user-side devices based on user-side sensor data of one or more user-side sensors. For example, the user-side sentiment score may be determined based solely on user-side sensor data of one or more user-side sensors.

[0090] The sentiment score may be determined based on (a) a current sentiment score computed using the user device and (b) at least one user-side sentiment score received by the user device from at least one user-side device communicatively coupleable to the user device, or (b) at least one user-side sentiment score computed by the user device based on user-side sensor data received from at least one user-side device. In other words, the sentiment score may be determined based on multiple sentiment scores, i.e., based on the current sentiment score and based on the at least one user-side sentiment score.

[0091] In other words, the sentiment score may be determined based on selecting either the current sentiment score or at least one user-side sentiment score as the sentiment score. Alternatively, the current sentiment score and the at least one user-side sentiment score may be combined to determine the sentiment score. For example, a mean value or average value of the current sentiment score and the at least one user-side sentiment score may be computed to determine the sentiment score.

[0092] Determining the sentiment score may include comparing the current sentiment score and at least one user-side sentiment score. By comparing the current sentiment score with one or more user-side sentiment scores, it may be determined whether the current sentiment score and the one or more user-side sentiment scores match and / or are consistent with each other. Thus, the sentiment score may be determined based on multiple sources. Thus, for example, to improve interactive communication, it may be possible to verify the sentiment score, which may be transmitted to the server and, in at least certain embodiments, may be used to train an artificial intelligence module of the server. Furthermore, it may be possible to perform a plausibility check of the current sentiment score and the user-side sentiment score.

[0093] Determining the sentiment score may include determining a deviation between the current sentiment score and at least one user-side sentiment score and comparing the determined deviation with a threshold value for deviation. By comparing the deviation with the threshold value, agreement, consistency, inconsistency, and / or contradiction between the current sentiment score and the at least one user-side sentiment score may be efficiently determined. Therein, the threshold value may be a predetermined threshold value. Optionally, the threshold value may be stored in a data storage device of the user device or may be obtained from another source, for example, from a server. Furthermore, it should be noted that the threshold value for deviation may refer to or indicate a range of deviation.

[0094] The method may further include discarding and / or ignoring at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if the determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a threshold deviation, wherein discarding the current sentiment score, the at least one user-side sentiment score, and the sentiment score may include, for example, removing and / or deleting the current sentiment score, the at least one user-side sentiment score, and the sentiment score from the user device and / or the at least one user-side device.

[0095] For this reason, it may be possible to ensure that only sentiment scores are used by the server to improve interactive communication, and the sentiment scores are based on the current sentiment score and one or more user-side sentiment scores that are consistent and / or agree with each other. On the other hand, sentiment scores determined based on the current sentiment score and one or more user-side sentiment scores that are different from another may be ignored and not used by the server. Overall, it may be possible to effectively improve interactive communication between the server and the user device, for example, by using the sentiment scores to train an artificial intelligence module and / or a reinforcement learning model of the server.

[0096] The method may further include preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a threshold deviation. Alternatively, or additionally, the method may further include transmitting the sentiment score to the server only if the current sentiment score and the at least one user-side sentiment score substantially match each other. Thus, it can be ensured that a sentiment score is only transmitted to the server and / or used by the server to improve interactive communication if the sentiment score is based on the current sentiment score and the at least one user-side sentiment score that match each other and / or are consistent with each other.

[0097] In the context of the present disclosure, "substantially" matching (or differing) may mean that the relative deviation between each quantity or measure, e.g., the current sentiment score and at least one user-side sentiment score, is within a certain range. For example, substantially matching or differing may mean that the respective measures differ from each other by less than 40%, particularly less than 30%, less than 25%, less than 20%, less than 15%, or less than 10%.

[0098] The method may further include aggregating the plurality of sentiment scores, e.g., one or more of the current sentiment score and the at least one user-side sentiment score, to determine a sentiment score for transmission to the server. Thus, the sentiment score for transmission to the server may refer to an integrated sentiment score determined based on the plurality of sentiment scores.

[0099] Aggregating multiple sentiment scores may include applying an aggregation function, such as, for example, a weighted average.

[0100] Additionally, the plurality of sentiment scores may be aggregated according to a deviation between at least two of the plurality of sentiment scores.

[0101] As an example, if the deviation between at least two sentiment scores reaches and / or exceeds a threshold for deviation, different weights may be applied to the different sentiment scores to determine the sentiment score for transmission to the server.

[0102] Alternatively, or additionally, weighting may be applied depending on the source of the aggregated sentiment score, e.g., the user device and / or the user-side device. For example, a current sentiment score determined based on sensor data of the user device may have a higher weighting than one or more user-side sentiment scores determined based on user-side sensor data of one or more user-side sensors.

[0103] The method is: - preventing and / or delaying transmission of the sentiment score to the server if the determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a deviation threshold; receiving a further notification from the server to the user, the further notification substantially matching and / or equal to the notification; obtaining further response data indicative of the user's further response to the further notification; The method may further include determining a further sentiment score based on the obtained further reaction data, the further sentiment score indicating a further sentiment of the user in reaction to the further notification.

[0104] In other words, transmission of the determined sentiment score to the server may be delayed, and the further sentiment score may be determined before transmitting the sentiment score to the server. Therefore, when a similar or identical user reaction that substantially matches, is equal to, and / or is identical to, each other, should be triggered by the notification and the further notification, it may be possible to use the further sentiment score to verify the sentiment score. As a result, it may be possible to detect whether the sentiment score or the further sentiment score actually indicates the user's reaction to the notification and does not indicate the user's reaction to environmental parameters or factors, such as, for example, low temperature or noise in the user's environment. Thus, the quality of the sentiment score may be improved and / or it may be ensured that the sentiment score at least primarily reflects and / or indicates the user's sentiment in response to the notification. Furthermore, for example, if the sentiment score and the further sentiment score are different from each other, such sentiment score may be effectively identified based on comparing the sentiment score and the further sentiment score, thereby effectively improving interactive communication because an incorrect sentiment score may not be used by the server.

[0105] The method may further include comparing a further sentiment score determined based on further response data indicative of the user's further response to the further notification with the sentiment score or the current sentiment score determined based on the response data indicative of the user's response to the notification, wherein the comparing may include determining a deviation between the sentiment score and the further sentiment score. Optionally, a deviation between the further sentiment score and the sentiment score or the current sentiment score may be determined.

[0106] The method may further include transmitting at least one of the sentiment score, the current sentiment score, and the further sentiment score to a server only if the further sentiment score determined based on further reaction data indicative of the user's further reaction to the further notification substantially matches the sentiment score determined based on the reaction data indicative of the user's reaction to the notification or the current sentiment score.

[0107] As an example, determining the sentiment score and the further sentiment score before transmitting the sentiment score to the server and comparing these sentiment scores may avoid misinterpretation of the user's reaction. For example, the determined sentiment score may not be considered to be correlated with a negative reaction if the sentiment score and the further sentiment score do not match each other, for example, if one is positive and the other is negative. Instead, a further notification that may be identical to or similar to the notification may be provided on the user device, for example, at a different time from the notification. Then, the user's further reaction to the further notification and / or the further sentiment score may confirm that the sentiment score at least primarily indicates the user's reaction to the notification (and not to any environmental parameters) if the sentiment score and the further sentiment score substantially match each other. Thus, determining the sentiment score and the further sentiment score and comparing these sentiment scores may make it possible to confirm the correctness of the sentiment score and / or the further sentiment score. As a result, interactive communication can be effectively improved since only the confirmed sentiment scores can be used by the server, for example, to train an artificial intelligence module and / or a reinforcement learning model.

[0108] The method may further include determining a privacy level for at least one of the reaction data, the sensor data of one or more sensors of the user device, and the user-side sensor data of the one or more user-side sensors, where the privacy level indicates one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data, which are to be manipulated in and / or removed from one or more of the reaction data, the sensor data, and the user-side sensor data to determine and / or before determining the sentiment score. Such manipulation may be performed on or by one or more of the user device, the one or more user-side sensors, and the one or more user-side devices.

[0109] The privacy level may be, for example, at least one of definable, user-configurable, and programmable. For example, a user may define one or more privacy levels for one or more feature groups, such as, for example, "location, person, health, activity." Therein, the privacy level may be set, for example, to "low, medium, high." Furthermore, the feature group may define one or more features in one or more of the response data, the sensor data, and the user-side sensor data.

[0110] Based on the defined privacy level for one or more of the feature groups, one or more processing operations for processing one or more of the reaction data, sensor data, and user-side sensor data may be selected by the user device for one or more features defined by the one or more feature groups.

[0111] The one or more features in one or more of the reaction data, sensor data, and user-side sensor data may include and / or refer to a user's reaction features and / or reaction patterns related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expression, changes in the user's skin color, etc. Alternatively, or additionally, the one or more features may include and / or refer to one or more environmental parameters that affect the user's sentiment, such as, for example, people in the user's vicinity, the location of the user and / or the user device, the temperature in the user device's environment, etc.

[0112] The one or more processing operations selected for processing one or more of the response data, the sensor data, and the user-side sensor data may depend on the data type of the response data, the sensor data, and / or the user-side sensor data.

[0113] For example, when the response data, sensor data, and / or user-side sensor data include one or more images or image data, the processing operation associated with the privacy level may leave the image intact for a "low" privacy level, blur a person's face for a "medium" privacy level, and crop the person from the image for a "high" privacy level.

[0114] Similarly, when the response data, sensor data, and / or user-side sensor data includes text input or audio received, for example, via a user interface or via a microphone, depending on the privacy level set for the particular feature, the text input or audio input may be trimmed and particular features or information, such as name, date of birth, city, address, etc., may be removed from the response data, sensor data, and / or user-side sensor data.

[0115] According to a second aspect of the present disclosure, there is provided a computer program comprising instructions or software instructions, which, when executed on one or more processors of a user device, cause the user device to perform a method according to the first aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the user device.

[0116] According to a third aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the second aspect of the present disclosure.

[0117] According to a fourth aspect of the present disclosure, there is provided a computer-implemented method for interactive communication of a user device with a server. The method according to the fourth aspect may alternatively or additionally relate to a computer-implemented method for operating a user device, for example for interactive communication with a server, and optionally for operating one or more user-side devices. The method includes: on the user device, providing one or more notifications to a user of the user device; obtaining reaction data indicative of one or more reactions of a user to one or more notifications; determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores to one another, wherein the final sentiment score is usable by the server to train an artificial intelligence module and / or a reinforcement learning model implemented on the server.

[0118] In other words, multiple sentiment scores may be determined for at least one notification provided on the user device, and a final sentiment score may be determined based on a comparison of at least two of the sentiment scores. Also, for example, by comparing at least two sentiment scores, as discussed above and below with reference to the first aspect of the present disclosure, deviations, agreements, consistency, inconsistencies, and / or contradictions between the at least two sentiment scores may be determined. Thus, it may be possible to verify the determined sentiment scores and / or plausibility checks. Thus, it may be possible to ensure that only the (final) sentiment score is used by the server to improve interactive communication, and the (final) sentiment score is based on at least two sentiment scores, e.g., the current sentiment score and one or more user-side sentiment scores, that are consistent and / or agree with each other.

[0119] It should be noted that the term "final sentiment score" may refer to a sentiment score that can be transmitted to a server and used by the server to train an artificial intelligence module and / or a reinforcement learning module.

[0120] In general, the final sentiment score may refer to, among other things, a "sentiment score," as described with reference to the first aspect of the present disclosure. Furthermore, each of the "at least two sentiment scores" and / or each of the "plurality of sentiment scores" may refer to, for example, among other things, at least one of a current sentiment score and a user-side sentiment score, as described with reference to the first aspect of the present disclosure.

[0121] At least one notification may be provided on the user device. Further, multiple sentiment scores are determined. The multiple sentiment scores may be determined based on the same source or different sources. For example, all or a subset of the multiple sentiment scores may be determined using the user device, for example, based on sensor data of one or more sensors of the user device and / or based on user-side sensor data of one or more user-side sensors. Alternatively or additionally, all or a subset of the multiple sentiment scores may be determined using one or more user-side devices, for example, based on user-side sensor data of the one or more user-side devices. Among them, at least one of the determined sentiment scores may indicate a user's reaction to the at least one notification. One or more of the multiple sentiment scores may indicate a user's reaction to at least one environmental parameter or factor. Alternatively, all of the multiple sentiment scores may indicate a user's reaction to the at least one notification. Alternatively, or additionally, at least a subset of the sentiment scores may indicate a user's reaction to different notifications, for example, notifications provided at different times.

[0122] For example, if the first sentiment score determined based on the user-side sensor data of at least one user-side device and the second sentiment score determined based on the sensor data of the user device do not match or do not substantially match each other, this may indicate that the user's reaction to the at least one notification may be triggered or caused by one or more environmental parameters or factors, such as a low room temperature or noise in the user's environment. Thus, by comparing these two sentiment scores, it can be reliably determined whether the final sentiment score primarily or correctly indicates or reflects the user's sentiment in reaction to the notification, or whether the user's reaction was caused by one or more environmental parameters or factors that affect the user's sentiment.

[0123] Alternatively, or additionally, for example, if a first sentiment score indicative of a user's first reaction to a first notification and a second sentiment score indicative of a user's second reaction to a second notification are determined and do not match or do not substantially match each other, this may indicate that at least one of the user's first and second reactions may be triggered or caused by one or more environmental parameters or factors, such as a cold room temperature or noise in the user's environment, wherein the first and second notifications may be provided on the user device at different times, e.g., before or after each other.

[0124] Thus, it may be possible to verify the sentiment score and / or final sentiment score based on comparing at least two different sentiment scores. The two different sentiment scores may be based on different sensor data and / or reaction data, such as, for example, user-side sensor data from at least one user-side sensor and / or sensor data from at least one sensor in the user device. Alternatively, or additionally, the at least two sentiment scores may indicate different or multiple reactions of the user in response to multiple notifications, for example, provided at different times. Comparing the at least two sentiment scores may enable reliable determination of whether the at least two sentiment scores are consistent with each other, which enables comprehensive verification of the determined sentiment score and / or plausibility check. Thus, it may be possible to provide a final sentiment score that can be used to train an artificial intelligence module and / or a reinforcement learning model on the server that accurately reflects or indicates the user's sentiment in response to the at least one notification.

[0125] It is emphasized that any feature, step, element, and / or example described herein above and below with reference to one aspect of the present disclosure applies to any other aspect of the present disclosure, and vice versa. In particular, any embodiment of the first aspect may be combined with any embodiment of the fourth aspect, and vice versa.

[0126] At least one of the plurality of sentiment scores may be indicative of the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that influences the user's sentiment.

[0127] At least two of the plurality of sentiment scores may indicate, for example, a user's sentiment in response to at least two notifications received from the server at different times.

[0128] The final sentiment score may be determined based on selecting at least one of the plurality of sentiment scores as the final sentiment score. Alternatively, or additionally, the plurality of sentiment scores, e.g., the at least two sentiment scores, may be combined to determine the final sentiment score. For example, the final sentiment score may be determined as a mean value or average value of the at least two sentiment scores.

[0129] The method may further include receiving, using the user device, at least one of the plurality of sentiment scores from at least one user-side device communicatively coupleable and / or communicatively coupled to the user device. In other words, at least one of the plurality of sentiment scores may be determined at one or more user-side devices and transmitted to the user device. In this, receiving the sentiment score may include obtaining the sentiment score from the at least one user-side device and / or instructing the at least one user-side device to transmit or send the sentiment score to the user device.

[0130] Obtaining the reaction data may include capturing sensor data using at least one sensor of the user device. In other words, the reaction data, or at least a portion thereof, may be obtained based on sensor data of one or more sensors of the user device.

[0131] The method may further include deriving, using the user device, response data from captured sensor data of at least one sensor of the user device. In this, deriving the response data may include processing the sensor data, for example, by control circuitry of the user device. Such processing may include one or more of filtering, transforming, and converting the captured sensor data, for example, to generate the response data. Alternatively or additionally, deriving the response data may include selecting at least a portion of the sensor data as the response data. Alternatively or additionally, deriving the response data may include using at least a portion of the sensor data as the response data. Alternatively or additionally, deriving the response data may include combining at least a portion of the sensor data with additional sensor data of additional sensors of the user device and / or with user-side sensor data of at least one user-side sensor.

[0132] The at least one sensor of the user device may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor. Such a sensor may advantageously enable reliable detection or sensing of a user's reaction in response to a notification, and consequently enable reliable determination of reaction data indicative of the user's reaction. However, it should be noted that any other type of sensor of the user device may be used to capture sensor data and / or determine reaction data based thereon.

[0133] The reaction data may be acquired based on sensor data from multiple sensors of the user device, e.g., different types of sensors. Thus, acquiring the reaction data may include capturing sensor data using multiple sensors of the user device, where the sensor data from different sensors may be combined, merged, and / or fused to determine the reaction data. Thus, it may be possible to provide accurate and comprehensive reaction data that may accurately and comprehensively reflect a user's reaction to one or more notifications or that may contain comprehensive and accurate information regarding a user's reaction to one or more notifications. Alternatively, or additionally, using sensor data from multiple sensors to determine the reaction data may enable a plausibility check of all sensor data and / or reaction data.

[0134] For example, first sensor data from a first sensor of a user device may be captured, and the first sensor data may be complemented, merged, combined, and / or fused with second sensor data from a second sensor of the user device to generate response data, where the first sensor may be different from the second sensor.

[0135] Obtaining the response data may include receiving user-side sensor data from at least one user-side sensor communicatively coupled to the user device using and / or in the user device. In other words, the response data, or at least a portion thereof, may be obtained based on user-side sensor data of one or more user-side sensors.

[0136] A "user-side sensor" may indicate or refer to a sensor of a user-side device. Furthermore, receiving user-side sensor data may include obtaining user-side sensor data from at least one user-side sensor. Alternatively, or additionally, receiving user-side sensor data may include instructing at least one user-side sensor to transmit the user-side sensor data to the user device.

[0137] As described hereinabove with reference to the first aspect of the present disclosure, a user-side sensor may refer to a sensor located near, around, in the environment, and / or remotely from a user device. Alternatively or additionally, a user-side sensor may refer to a sensor coupled to, communicatively coupled to, couplable with, and / or configured to communicate with a user device via a common communication network, e.g., the same local area network (LAN) to which the user device is connected. For example, a user-side sensor may be included within or part of a user-side device. Alternatively or additionally, a user-side sensor may be a standalone sensor. However, it should be noted that various or multiple different user-side sensors may be used to acquire response data, such as at least one user-side sensor included in at least one user-side device and at least one additional user-side sensor that may be a standalone user-side sensor. When a user-side sensor is included within a user-side device, communication between the user device and the user-side sensor (or user-side device) may be implemented via communication circuitry in the user-side device. If the user-side sensor is a standalone sensor, communication between the user-side sensor and the user device may be performed via corresponding communication circuitry of the user-side sensor, e.g., using a communication link, communication connection, and / or communication protocol, e.g., as described above with respect to the user-side device.

[0138] The method may further include deriving, using the user device, response data from the received user-side sensor data of the at least one user-side sensor. In this, deriving the response data may include, for example, processing the user-side sensor data by control circuitry of the user device. Such processing may include, for example, one or more of filtering, transforming, and converting the user-side sensor data to generate the response data. Alternatively or additionally, deriving the response data may include selecting at least a portion of the user-side sensor data as the response data. Alternatively or additionally, deriving the response data may include using at least a portion of the user-side sensor data as the response data. Alternatively or additionally, deriving the response data may include combining at least a portion of the user-side sensor data with additional user-side sensor data of additional user-side sensors and / or with sensor data of one or more sensors of the user device.

[0139] The at least one user-side sensor may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a short-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor. However, it should be noted that any other type of user-side sensor may be used.

[0140] The at least one user-side sensor may be included in at least one user-side device positioned in the vicinity of the user device, wherein the at least one user-side device is one or more of a smart television (television), a smart speaker, a smart watch, a health monitor, an IoT (Internet of Things) device, and an aerosol-generating device.

[0141] The method may further include computing at least one of the plurality of sentiment scores based on receiving, with the user device, user-side sensor data from the at least one user-side device and deriving, with the user device, at least a portion of the reaction data and / or further reaction data from the received user-side sensor data. In other words, the user-side sensor data may be obtained from at least one user-side sensor, and the further reaction data may be derived therefrom by the user device to determine one or more of the plurality of sentiment scores.

[0142] The method may further include receiving at least one query at the user device, wherein the at least one notification is provided by the server in response to the at least one query, wherein the at least one query may be received at the user device based on one or more user inputs, for example, via a user interface, a microphone, a camera, and / or a sensor of the user device. Alternatively, or additionally, the one or more notifications may be provided on the user device by the server in response to the at least one query.

[0143] The method may further include transmitting the at least one query received at the user device to a server, wherein transmitting the at least one query may include transmitting the at least one query to the server, for example, in the form of one or more data packages. Optionally, transmitting the at least one query to the server may include establishing a communication link or connection between the user device and the server.

[0144] The at least one query may include text input received at the user device via a user interface, verbal input received at the user device via a microphone, and / or video-based input received via a camera of the user device. As noted above, the at least one query may include one or more of at least one letter, at least one symbol, at least one number, text, a question, an answer to a question, a statement, a request, voice, an acoustic message, a message, a text message, at least one icon, an alert, an acoustic alert, an optical alert, at least one image, and a video.

[0145] Each of the one or more notifications may include one or more of at least one letter, at least one symbol, at least one number, text, voice, an audio message, a question, an answer to a question, a statement, a request, a message, a text message, at least one icon, an alarm, an audio alarm, an optical alarm, at least one image, and a video. However, it should be noted that any other type of notification or content of any notification is contemplated.

[0146] The determined final sentiment score may be transmitted to a server, wherein the final sentiment score may be transmitted to the server from the user device and / or from one or more user-side devices.

[0147] The method may further include displaying one or more notifications, for example, on a user interface of the user device. Alternatively, or additionally, one or more of the notifications may be played or output over a microphone of the user device.

[0148] Determining the final sentiment score involves: determining an intermediate sentiment score based on the obtained response data; and anonymizing the intermediate sentiment score, thereby generating a final sentiment score.

[0149] As described with reference to the first aspect of the present disclosure, the intermediate sentiment score may include information or data related to the user's personal information and / or personal data, while the final sentiment score may not contain or be free of the user's personal information and / or personal data. Thus, the intermediate sentiment score may be considered an intermediate quantity or measure that may be determined, e.g., temporarily determined, to determine the final sentiment score. Generally, the intermediate sentiment score may be determined and anonymized on the user device and / or on at least one user-side device. Generally, anonymizing the intermediate sentiment score to generate an anonymized score or measure advantageously enables protecting the privacy or anonymity of interactive communications between the server and the user device, as described herein above with respect to the first aspect of the present disclosure.

[0150] The final sentiment score may be anonymized based on normalizing the intermediate sentiment score with a reference sentiment score (and / or reference sentiment score value). For example, the reference sentiment score may be selected depending on the scale on which the sentiment score is determined. For example, the reference sentiment score may refer to and / or indicate the user's most negative reaction to the notification, the user's neutral reaction to the notification, the user's most positive reaction to the notification, or any reaction of the user in between. Furthermore, normalizing the intermediate sentiment score may include converting the intermediate sentiment score to a final sentiment score based on the reference sentiment score. For example, the intermediate sentiment score may be divided by or multiplied by the reference sentiment score. However, any other mathematical operation may be applied. Furthermore, it should be noted that the final sentiment score may be given in absolute or relative terms.

[0151] The method may further include removing the intermediate sentiment score from the user device upon anonymizing the intermediate sentiment score and / or upon generating the final sentiment score. Alternatively, or additionally, the intermediate sentiment score may be removed from at least one user-side device. By removing the intermediate sentiment score from the user device and / or the at least one user device, the user's private or personal information may be further protected. In particular, removing the intermediate sentiment score may include deleting, overwriting, and / or destroying the intermediate sentiment score.

[0152] The method may further include removing the final sentiment score from the user device when transmitting the final sentiment score to the server. Alternatively, or additionally, the final sentiment score may be removed from at least one user-side device when transmitting the final sentiment score to the server. In other words, for example, transmission of the final sentiment score from the user device and / or at least one user-side device to the server may trigger removal of the final sentiment score from the user device and / or user-side device. In this way, it can be ensured that no storage resources or memory are used or blocked by the final sentiment score when it is being transmitted to the server.

[0153] Providing one or more notifications may include receiving a first notification and a second notification at the user device, where the second notification substantially matches and / or is equal to the first notification. In this regard, obtaining response data may include obtaining first response data indicative of a first user response to the first notification and obtaining second response data indicative of a second user response to the second notification. Furthermore, determining a plurality of sentiment scores may include determining a first sentiment score based on the first response data and determining a second sentiment score based on the second response data. The first and second response data may be determined based on at least one of sensor data of one or more sensors of the user device and user-side sensor data of one or more user-side sensors. By determining independent or separate sentiment scores, e.g., first and second sentiment scores, based on independent, separate, and / or different notifications, e.g., first and second notifications that substantially match each other, it can be ensured that the final sentiment score primarily or correctly represents the user's sentiment in response to one of the notifications. As a result, interactive communications can be accurately improved and / or personalized, for example, based on training an artificial intelligence module and / or a reinforcement learning model on a server. For example, the final sentiment score can be determined based on the first and second sentiment scores.

[0154] Determining the final sentiment score may include determining a deviation between at least two of the plurality of sentiment scores, e.g., a first sentiment score and a second sentiment score, and comparing the determined deviation to a threshold value for the deviation. The threshold value may be predetermined and / or may be stored in a data storage or memory of the user device. Alternatively, or additionally, the threshold value may be obtained from another source, e.g., a server. Furthermore, the threshold value for the deviation may refer to a range of deviation, e.g., a percentage of relative deviation between the at least two sentiment scores.

[0155] The method may further include discarding and / or ignoring at least one of the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, if a determined deviation between the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold value for deviation.

[0156] The method may further include preventing the final sentiment score from being transmitted to the server if the determined deviation between the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold deviation. Alternatively, or additionally, the method may further include transmitting the final sentiment score to the server only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0157] The method may further include aggregating the plurality of sentiment scores to determine a final sentiment score for transmission to the server. Accordingly, the final sentiment score for transmission to the server may refer to a consolidated sentiment score determined based on the plurality of sentiment scores.

[0158] Aggregating multiple sentiment scores may include applying an aggregation function, such as, for example, a weighted average.

[0159] Additionally, the plurality of sentiment scores may be aggregated according to a deviation between at least two of the plurality of sentiment scores.

[0160] As an example, if the deviation between at least two sentiment scores reaches and / or exceeds a threshold for deviation, different weights may be applied to the different sentiment scores to determine the final sentiment score.

[0161] Alternatively, or additionally, weighting may be applied depending on the source of the aggregated sentiment score, e.g., the user device and / or the user-side device. For example, a sentiment score determined based on sensor data of a user device may have a higher weighting than one or more (user-side) sentiment scores determined based on user-side sensor data of one or more user-side sensors.

[0162] The method may further include determining a privacy level for at least one of the reaction data, the sensor data of one or more sensors of the user device, and the user-side sensor data of the one or more user-side sensors, where the privacy level indicates one or more features in one or more of the reaction data, the sensor data, and the user-side sensor data, which are to be manipulated in and / or removed from one or more of the reaction data, the sensor data, and the user-side sensor data to determine and / or before determining the sentiment score. Such manipulation may be performed on or by one or more of the user device, the one or more user-side sensors, and the one or more user-side devices.

[0163] The privacy level may be, for example, at least one of definable, user-configurable, and programmable. For example, a user may define one or more privacy levels for one or more feature groups, such as, for example, "location, person, health, activity." Therein, the privacy level may be set, for example, to "low, medium, high." Furthermore, the feature group may define one or more features in one or more of the response data, the sensor data, and the user-side sensor data.

[0164] Based on the defined privacy level for one or more of the feature groups, one or more processing operations for processing one or more of the reaction data, sensor data, and user-side sensor data may be selected by the user device for one or more features defined by the one or more feature groups.

[0165] The one or more features in one or more of the reaction data, sensor data, and user-side sensor data may include and / or refer to the user's reaction features and / or reaction patterns related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expression, etc. Alternatively, or additionally, the one or more features may include and / or refer to one or more environmental parameters that affect the user's sentiment, such as, for example, people in the user's vicinity, the location of the user and / or the user device, the temperature in the environment of the user device, etc.

[0166] The one or more processing operations selected for processing one or more of the response data, the sensor data, and the user-side sensor data may depend on the data type of the response data, the sensor data, and / or the user-side sensor data.

[0167] For example, when the response data, sensor data, and / or user-side sensor data include one or more images or image data, the processing operation associated with the privacy level may leave the image intact for a "low" privacy level, blur a person's face for a "medium" privacy level, and crop the person from the image for a "high" privacy level.

[0168] Similarly, when the response data, sensor data, and / or user-side sensor data includes text input or audio received, for example, via a user interface or via a microphone, depending on the privacy level set for the particular feature, the text input or audio input may be trimmed and particular features or information, such as name, date of birth, city, address, etc., may be removed from the response data, sensor data, and / or user-side sensor data.

[0169] According to a fifth aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which, when executed on one or more processors of a user device, cause the user device to perform a method according to the fourth aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the user device.

[0170] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the fifth aspect of the present disclosure.

[0171] According to a seventh aspect of the present disclosure, there is provided a computer-implemented method for interactive communication of a server with a user device. The method according to the seventh aspect may alternatively or additionally relate to a computer-implemented method for operating a server, e.g., for interactive communication with a user device, and optionally for operating one or more user-side devices. The method includes: transmitting a notification from the server to the user device; receiving, by a server, a sentiment score, the sentiment score correlating with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on the received sentiment scores.

[0172] In one embodiment, the sentiment score may be received from a user device and / or from one or more user-side devices. Further, receiving the sentiment score may include obtaining the sentiment score from the user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or at least one user-side device to transmit or send the sentiment score to a server.

[0173] In general, the method according to the seventh aspect may correspond to one or more of the methods of the first and fourth aspects of the present disclosure, but from the perspective of the server, or at least primarily from the perspective of the server. Any of the disclosures, embodiments, and examples described herein above and below with reference to any aspect of the present disclosure apply equally to and may be combined with the method according to the seventh aspect of the present disclosure, and vice versa.

[0174] The method may further include selecting, from the server's knowledge base, a further notification to be transmitted to the user device based on the received sentiment score. The further notification may be selected, for example, upon or in response to receipt at the server of one or more queries from the user device.

[0175] The server may comprise, for example, a natural language processing ("NLP") module or engine configured to process, analyze, and / or resolve one or more queries received from the user device. Additionally, the server's knowledge base may include a plurality of notifications transmittable to the user device, and the server may be configured to determine, select, identify, and / or generate further notifications from among or based on the plurality of notifications, for example, based on or using the server's artificial intelligence module, recommendation engine, and / or reinforcement learning model.

[0176] The reinforcement learning model may be trained based on maximizing and / or optimizing a reward function of the reinforcement learning model implemented on the server.

[0177] Thus, it may be possible to improve the interactive communication and / or reinforcement learning model over time, for example, continuously or iteratively over multiple training steps, in each of which reinforcement learning rewards reflecting and / or indicative of the level of correctness or quality of decisions of the implemented reinforcement learning model may be provided or dispensed to the reinforcement learning model to penalize it when it makes incorrect decisions and reward it when it makes correct decisions.

[0178] The method further includes receiving at least one query from the user device, and the one or more notifications are transmitted from the server to the user device upon receiving the at least one query from the user device.

[0179] The method further includes receiving a further query from the user device, wherein the further notification is selected based on the received sentiment score and based on the received further query. The sentiment score allows the reinforcement learning model to be accurately trained so that the further notification is more appropriate for the further query, thereby improving the interactive communication. Alternatively, or additionally, the interactive communication may be personalized based on or using the sentiment score to reflect or indicate the user's sentiment in response to the notification, where the sentiment score may be provided earlier than the further notification during the interactive communication, for example.

[0180] The method may further include transmitting a further notification to the user device in response to the further query.

[0181] The method may further include processing the query and / or the further query using a natural language processing engine implemented on the server.

[0182] The method may further include receiving at least one additional sentiment score and selecting, from the server's knowledge base, an additional notification to be transmitted to the user device based on the received sentiment score and based on the received additional sentiment score, wherein both the sentiment score and the additional sentiment may be used by the server to train a reinforcement learning model of the server. Alternatively, or additionally, the sentiment score and the additional sentiment score may be used to validate one or both of the sentiment score and the additional sentiment score, for example, as described with reference to the first and fourth aspects of the present disclosure.

[0183] The at least one further sentiment score may be received by the server from the user device and / or from a user-side device, wherein receiving the further sentiment score may include obtaining the further sentiment score from the user device and / or from the at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or the at least one user-side device to transmit or send the sentiment score to the server.

[0184] The method may further include comparing the sentiment score with at least one additional sentiment score. Based on such comparison, the sentiment score and / or the additional sentiment may be verified. Thus, it may be possible to effectively improve the reinforcement learning model or its training.

[0185] The method may further include determining a deviation of the sentiment score and the at least one additional sentiment score and comparing the determined deviation to a threshold value for the deviation. The threshold value may be predetermined and / or may be stored in a data storage or memory of the server. Alternatively, or additionally, the threshold value may be obtained from another source, for example, a user device. Furthermore, the threshold value for the deviation may refer to a range of deviation, for example, a percentage of relative deviation between the sentiment score and the at least one additional sentiment score.

[0186] The method may further include discarding at least one of the sentiment score and the at least one additional sentiment score if a determined deviation between the sentiment score and the at least one additional sentiment score meets and / or exceeds a threshold value for deviation.

[0187] The method may further include training a reinforcement learning model implemented on the server based on the sentiment score and / or the at least one additional sentiment score only if the sentiment score and the at least one additional sentiment score substantially match each other.

[0188] According to an eighth aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which, when executed on one or more processors of a server, cause the server to perform the method according to the seventh aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the server.

[0189] According to a ninth aspect of the present disclosure, there is provided a non-transitory computer readable medium having stored thereon a computer program according to the ninth aspect of the present disclosure.

[0190] According to a tenth aspect of the present disclosure, there is provided a computer-implemented method for interactive communication of a server with a user device. The method according to the tenth aspect may alternatively or additionally relate to a computer-implemented method for operating a server, e.g., for interactive communication with a user device, and optionally for operating one or more user-side devices. The method comprises: transmitting one or more notifications from the server to the user device; receiving, by a server, a plurality of sentiment scores, each of the plurality of sentiment scores correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores.

[0191]

[0013] In one embodiment, each of the sentiment scores may be received from a user device and / or from one or more user-side devices. Further, receiving the sentiment scores may include obtaining the sentiment scores from the user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment scores may include instructing the user device and / or at least one user-side device to transmit or send the sentiment scores to a server.

[0192] In general, the method according to the tenth aspect may correspond to one or more of the methods of the first, fourth, and seventh aspects of the present disclosure, but from the perspective of the server, or at least primarily from the perspective of the server. Any of the disclosures, embodiments, and examples described herein above and below with reference to any aspect of the present disclosure apply equally to and may be combined with the method according to the tenth aspect of the present disclosure, and vice versa.

[0193] Thus, at least two sentiment scores, e.g., a first sentiment score and a second sentiment score, may be determined, and each of the first sentiment score and the second sentiment score may be correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server. Further, the reinforcement learning model implemented on the server may be trained based on a comparison of the first sentiment score and the second sentiment score.

[0194] The multiple sentiment scores, for example, the first and / or second sentiment scores, may be received by the server, from the user device, or from a user-side device.

[0195] At least one of the plurality of sentiment scores, e.g., the first and / or second sentiment scores, may indicate a user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment. Alternatively, or additionally, at least one of the plurality of sentiment scores, e.g., the first and / or second sentiment scores, may indicate a user's sentiment in response to one or more notifications, e.g., at least one of the first and / or second notifications, received by the user device from the server.

[0196] Transmitting one or more notifications from the server to the user device may include transmitting a first notification and a second notification from the server to the user device, where the second notification substantially matches and / or is equal to the first notification. In particular, receiving a plurality of sentiment scores may include receiving a first sentiment score in response to transmitting the first notification to the user device and receiving a second sentiment score in response to transmitting the second notification to the user device. The first and second notifications may be transmitted, for example, before or after each other and / or at different times. For this reason, as described herein above, it may be possible to validate the first and / or second sentiment scores and, consequently, effectively train a reinforcement learning model.

[0197] The method may further include determining a deviation between at least two of the plurality of sentiment scores, e.g., between a first sentiment score and a second sentiment score, and comparing the determined deviation to a threshold value for the deviation. The threshold value may be predetermined and / or may be stored in a data storage or memory of the server. Alternatively, or additionally, the threshold value may be obtained from another source, e.g., a user device. Furthermore, the threshold value for the deviation may refer to a range of deviation, e.g., a percentage of relative deviation between the sentiment score and the at least one.

[0198] The method may further include discarding and / or ignoring at least one of the plurality of sentiment scores, e.g., the first sentiment score and / or the second sentiment score, if the determined deviation between at least two of the plurality of sentiment scores, e.g., between the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold value for deviation.

[0199] The method may further include training a reinforcement learning model implemented on the server based on at least one of the received plurality of sentiment scores, e.g., based on at least one of the first sentiment score and the second sentiment score, only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0200] Training a reinforcement learning model is determining a final sentiment score based on at least one of the plurality of sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score; deriving a reinforcement learning reward for training a reinforcement learning model from the determined final sentiment score and / or computing a reinforcement learning reward based on the final sentiment score; - feeding the reinforcement learning reward to the reinforcement learning model.

[0201] Therein, the final sentiment score may be determined based on selecting at least one of the multiple sentiment scores, for example, at least one of the first sentiment score and the second sentiment score, as the final sentiment score. Alternatively, or additionally, the multiple sentiment scores, for example, the first and second sentiment scores, may be combined to determine the final sentiment score. For example, the final sentiment score may be determined as a mean value or average value of the multiple sentiment scores, for example, the first and second sentiment scores.

[0202] The method may further include aggregating the plurality of sentiment scores to determine a final sentiment score for transmission to the server. Accordingly, the final sentiment score for transmission to the server may refer to a consolidated sentiment score determined based on the plurality of sentiment scores.

[0203] Aggregating multiple sentiment scores may include applying an aggregation function, such as, for example, a weighted average.

[0204] Additionally, the plurality of sentiment scores may be aggregated according to a deviation between at least two of the plurality of sentiment scores.

[0205] As an example, if the deviation between at least two sentiment scores reaches and / or exceeds a threshold for deviation, different weights may be applied to the different sentiment scores to determine the final sentiment score.

[0206] Alternatively, or additionally, weighting may be applied depending on the source of the aggregated sentiment score, e.g., the user device and / or the user-side device. For example, a sentiment score determined based on sensor data of the user device may have a higher weighting than one or more (user-side) sentiment scores determined based on user-side sensor data of one or more user-side sensors. A reinforcement learning reward may refer to a reinforcement learning reward value. The reinforcement learning reward may be determined, calculated, and / or computed, for example, based on at least the final sentiment score and a reward function of the reinforcement learning model.

[0207] Determining and / or computing the reinforcement learning reward may further include determining a trend of a plurality and / or series of previous sentiment scores, e.g., a plurality and / or series of previously determined final sentiment scores determined at least in part based on one or more notifications provided to the user. Thus, a plurality and / or series of previous sentiment scores may refer to a series of past sentiment scores, e.g., past final sentiment scores, determined at least in part based on one or more notifications provided to the user device.

[0208] The method may further include determining a weight based on the determined trend of the plurality and / or series of previous sentiment scores, and determining a reinforcement learning reward based on the determined weight and the final sentiment score.

[0209] By determining a trend and determining a reinforcement learning reward based thereon, it may be possible, for example, to put the user's current sentiment in reaction to the notification against the historical backdrop of ongoing or recurring interactive communication between the user device and the server, such that the reinforcement learning reward may accurately reflect the user's current sentiment.

[0210] The method may further include comparing the final sentiment score to trends of a plurality and / or series of previous sentiment scores and determining a reinforcement learning reward based on the comparison.

[0211] The method may further include determining the reinforcement learning reward based on one or more previous sentiment scores, e.g., one or more previous reinforcement learning rewards determined based on one or more previous final sentiment scores.

[0212] According to an eleventh aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which, when executed on one or more processors of a server, cause the server to perform the method according to the tenth aspect of the present disclosure. The computer program may, for example, be stored on a data storage device or memory of the server.

[0213] According to a twelfth aspect of the present disclosure, there is provided a non-transitory computer-readable medium having stored thereon a computer program according to the eleventh aspect of the present disclosure.

[0214] According to a thirteenth aspect of the present disclosure, there is provided a computer-implemented method for interactive communication between a user device and a server. The method according to the thirteenth aspect may alternatively or additionally relate to a computer-implemented method for operating a system for interactive communication, the system including at least one server and at least one user device. Optionally, the system may include one or more user-side devices and / or one or more user-side sensors. The method includes: transmitting a notification from the server to the user device; on the user device, providing a notification to a user of the user device; - obtaining, using the user device and / or using at least one user-side device, reaction data indicative of a user's reaction to the notification; - determining, using the user device and / or using at least one user-side device, a sentiment score based on the acquired reaction data, wherein the sentiment score indicates the user's sentiment in reaction to the notification, and the sentiment score correlates with a reinforcement learning reward for training a reinforcement learning model implemented on a server; transmitting the determined sentiment scores from the user device and / or from at least one user-side device to a server; receiving, by a server, a sentiment score; training an enrichment model implemented on the server based on the sentiment scores received by the server.

[0215] In one embodiment, receiving the sentiment score may include obtaining the sentiment score from a user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or the at least one user-side device to transmit or send the sentiment score to a server.

[0216] In general, the method according to the thirteenth aspect may correspond to one or more of the methods of the first, fourth, seventh and tenth aspects of the present disclosure, but from a systems perspective, or at least primarily from a systems perspective. Any of the disclosures, embodiments and examples described herein above and below with reference to any aspect of the present disclosure apply equally to and may be combined with the method according to the thirteenth aspect of the present disclosure, and vice versa.

[0217] According to a fourteenth aspect of the present disclosure, there is provided a computer-implemented method for interactive communication between a user device and a server. The method according to the fourteenth aspect may alternatively or additionally relate to a computer-implemented method for operating a system for interactive communication, the system including at least one server and at least one user device. Optionally, the system may include one or more user-side devices and / or one or more user-side sensors. The method may include: transmitting one or more notifications from the server to the user device; on the user device, providing one or more notifications to a user of the user device; - obtaining, using the user device and / or using at least one user-side device, reaction data indicative of one or more reactions of the user to the one or more notifications; determining, using the user device and / or using at least one user-side device, a plurality of sentiment scores based on the acquired reaction data, wherein each of the plurality of sentiment scores correlates with a reinforcement learning reward for training a reinforcement learning model implemented on a server; comparing at least two of the determined plurality of sentiment scores with each other; transmitting at least one of the plurality of sentiment scores from the user device and / or from at least one user-side device to a server; receiving, by a server, at least one of a plurality of sentiment scores; training an enrichment model implemented on the server based on at least one of the plurality of sentiment scores received by the server.

[0218] In which, receiving the at least one sentiment score may include obtaining the sentiment score from a user device and / or from at least one user-side device. Alternatively or additionally, receiving the sentiment score may include instructing the user device and / or the at least one user-side device to transmit or send the sentiment score to a server.

[0219] In general, the method according to the fourteenth aspect may correspond to one or more of the methods of the first, fourth, seventh, and tenth aspects of the present disclosure, but from a systems perspective, or at least primarily from a systems perspective. Any of the disclosures, embodiments, and examples described herein above and below with reference to any aspect of the present disclosure apply equally to and may be combined with the method according to the fourteenth aspect of the present disclosure, and vice versa.

[0220] According to a fifteenth aspect of the present disclosure, there is provided a computer program comprising instructions and / or software instructions, which when executed on one or more processors of a system for interactive communication, cause the system to perform a method according to one or both of the thirteenth and fourteenth aspects of the present disclosure. The computer program may be stored, for example, on a data storage device or memory of a server, a user device, and / or one or more user-side devices.

[0221] According to a sixteenth aspect of the present disclosure, there is provided a non-transitory computer-readable medium having stored thereon a computer program according to the fifteenth aspect of the present disclosure.

[0222] According to a seventeenth aspect of the present disclosure, there is provided a user device configured to interactively communicate with a server. The user device may be configured to perform one or more steps of a method according to, for example, any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The user device comprises a communication circuit communicatively coupling the user device to the server and configured to receive notifications from the server. The user device further comprises a user interface configured to provide notifications to a user of the user device, and control circuitry including one or more processors, the control circuitry: - obtaining reaction data indicative of a user's reaction to the notification; - determining a sentiment score for transmission to the server based on the obtained reaction data, the sentiment score indicating the user's sentiment in reaction to the notification.

[0223] The user's communication circuitry may be configured to couple with a corresponding communication arrangement of the server to enable data exchange between the user device and the server. By way of example, the user device may be coupled to and / or configured to communicate with the server (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0224] The control circuitry of a user device may be referred to as a controller, a control module, and / or a control arrangement.

[0225] Additionally, the user device may include a data storage device or memory for storing, for example, sentiment scores, notifications, and / or reaction data. Alternatively, or additionally, computer programs, instructions, apps, applications, and / or software instructions may be stored in the data storage device or memory.

[0226] The control circuitry may be further configured to transmit the determined sentiment score to a server via communication circuitry of the user device.

[0227] The sentiment score may be an anonymized numerical measure indicative of a user's reaction to the notification. Alternatively, or additionally, the sentiment score may correlate with and / or be indicative of a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0228] The control circuit is determining an intermediate sentiment score based on the obtained response data; and anonymizing the intermediate sentiment score to generate a sentiment score.

[0229] The control circuitry may be configured to anonymize the sentiment score based on normalizing the intermediate sentiment score with a reference sentiment score.

[0230] The control circuitry may be configured to remove the intermediate sentiment scores from the user device when anonymizing the intermediate sentiment scores and / or when generating the sentiment scores.

[0231] The control circuitry may be configured to remove the sentiment score from the user device upon transmitting the sentiment score to the server.

[0232] The user device may further include at least one sensor, and the control circuitry may be further configured to capture sensor data using the at least one sensor of the user device.

[0233] The control circuitry may be configured to derive the response data from the captured sensor data of the at least one sensor.

[0234] The at least one sensor of the user device may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor.

[0235] The communication circuitry may be further configured to receive user-side sensor data from at least one user-side sensor communicatively coupled with the user device via the communication circuitry.

[0236] The control circuitry may be configured to derive response data from the received user-side sensor data of the at least one user-side sensor.

[0237] The at least one user-side sensor may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a short-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0238] The at least one user-side sensor may be included in at least one user-side device positioned in the vicinity of the user device, wherein the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, an IoT device, a health monitor, and an aerosol-generating device.

[0239] The control circuit - deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment that influences the user's sentiment; - determining a sentiment score based on the at least one environmental parameter.

[0240] The control circuit may be configured to acquire reaction data based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor disposed within the environment of the user device.

[0241] The control circuitry may include a classifier circuit configured to classify at least a portion of the response data to determine a sentiment score.

[0242] The user device may further include at least one camera configured to capture image data indicative of one or more images of a user of the user device, and the control circuitry may be further configured to determine reaction data based on the captured image data and determine a sentiment score based on classifying at least a portion of the reaction data with the classifier circuitry.

[0243] The control circuit is determining, with a classifier circuit, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - deriving a sentiment score from the determined reaction pattern.

[0244] The control circuitry may be configured to determine the sentiment score based on computing a current sentiment score and / or based on deriving the sentiment score from the computed current sentiment score.

[0245] The communication circuitry may be configured to receive at least one user-side sentiment score from at least one user-side device communicatively coupled to the user device, and the control circuitry may be configured to determine the sentiment score based on the user-side sentiment score received from the at least one user-side device.

[0246] The control circuit - computing a current sentiment score; - may be configured to determine the sentiment score based on at least one user-side sentiment score received from at least one user-side device communicatively coupled to the user device or at least one user-side sentiment score computed by the control circuitry based on user-side sensor data received from the at least one user-side device.

[0247] The control circuitry can be configured to compare the current sentiment score with at least one user-side sentiment score.

[0248] The control circuitry can be configured to determine a deviation between the current sentiment score and at least one user-side sentiment score and compare the determined deviation to a threshold value for the deviation.

[0249] The control circuitry may be configured to discard and / or ignore at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if a determined deviation between the current sentiment score and the at least one user-side sentiment score reaches and / or exceeds a threshold value for deviation.

[0250] The control circuit is -preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches and / or exceeds a deviation threshold; and / or The sentiment score may be configured to transmit the sentiment score to the server only if the current sentiment score and at least one user-side sentiment score substantially match each other.

[0251] The control circuit is - preventing transmission of the sentiment score to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches and / or exceeds a deviation threshold; receiving a further notification from the server, the further notification substantially matching or equal to the notification; obtaining further response data indicative of the user's further response to the further notification; - determining a further sentiment score based on the obtained further reaction data, the further sentiment score indicating a further sentiment of the user in reaction to the further notification.

[0252] The control circuitry may be configured to compare a further sentiment score determined based on further response data indicative of the user's further response to the further notification with the sentiment score or the current sentiment score determined based on the response data indicative of the user's response to the notification.

[0253] The control circuitry may be configured to transmit at least one of the sentiment score, the current sentiment score, and the further sentiment score to the server only if the further sentiment score determined based on the further reaction data indicative of the user's further reaction to the further notification substantially matches the sentiment score determined based on the reaction data indicative of the user's reaction to the notification or the current sentiment score.

[0254] According to an eighteenth aspect of the present disclosure, there is provided a user device configured to interactively communicate with a server. The user device may be configured to perform, for example, one or more steps of a method according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The user device may also refer to the user device described with reference to the seventeenth aspect of the present disclosure. The user device comprises a communication circuit communicatively coupling the user device to the server and configured to receive one or more notifications from the server. The user device further comprises a user interface configured to provide one or more notifications to a user of the user device, and control circuitry including one or more processors, the control circuitry: obtaining reaction data indicative of one or more reactions of a user to one or more notifications; determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; and determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores, e.g., the first sentiment score and the second sentiment score, to each other, wherein the final sentiment score is usable by the server to train a reinforcement learning model implemented on the server.

[0255] At least one of the plurality of sentiment scores, e.g., at least one first and / or second sentiment score, may indicate the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0256] At least two of the plurality of sentiment scores may indicate, for example, a user's sentiment in response to at least two notifications received from the server at different times.

[0257] The control circuitry may be configured to determine a final sentiment score based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

[0258] The communication circuitry may be further configured to receive at least one of the plurality of sentiment scores, e.g., at least one first and second sentiment score, from at least one user-side device communicatively coupleable to the user device.

[0259] The communication circuitry may be configured to receive user-side sensor data from the at least one user-side device, and the control circuitry may be further configured to compute at least one of a plurality of sentiment scores, e.g., at least one first and second sentiment score, based on deriving the further reaction data from the received user-side sensor data.

[0260] The communications circuitry may be configured to receive a first notification and a second notification to the user from the server, the second notification substantially matching and / or equal to the first notification. obtaining first reaction data indicative of a first reaction of the user to the first notification; obtaining second reaction data indicative of a second reaction of the user to the second notification; The system may be further configured to: determine a first sentiment score based on the first response data; and determine a second sentiment score based on the second response data.

[0261] The control circuitry may be configured to determine a deviation between at least two of the plurality of sentiment scores, e.g., between a first sentiment score and a second sentiment score, and compare the determined deviation to a threshold value for the deviation.

[0262] The control circuitry may be configured to discard and / or ignore at least one of the at least two sentiment scores, e.g., one of the first sentiment score and the second sentiment score, if the determined deviation meets and / or exceeds a threshold for deviation.

[0263] The control circuitry may be configured to prevent the final sentiment score from being transmitted to the server if a determined deviation between the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, reaches and / or exceeds a threshold deviation. Alternatively, or additionally, the control circuitry may be further configured to transmit the final sentiment score to the server only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0264] A nineteenth aspect of the present disclosure relates to the use of a user device, as described herein above and herein below, for interactive communication with a server and / or for determining a sentiment score.

[0265] A twentieth aspect of the present disclosure relates to the use of a user-side device, as described herein above and herein below, for interactively communicating with a server and / or for determining a sentiment score.

[0266] According to a twenty-first aspect of the present disclosure, there is provided a server configured to interactively communicate with a user device and / or a user-side device. The server may be configured to perform one or more steps of a method according to, for example, any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The server comprises a communication arrangement communicatively coupling the server to the user device and configured to transmit a notification to the user device, the communication arrangement being further configured to receive a sentiment score from the user device, the sentiment score indicating the user's sentiment in response to the notification. The server further comprises a control arrangement including a reinforcement learning model, the sentiment score being correlated with a reinforcement learning reward for training the reinforcement learning model, and the control arrangement being configured to train the reinforcement learning model based on the received sentiment score.

[0267] A server may comprise at least one computing device, for example, including one or more processors. However, it should be noted that a server may refer to a server system, a computer system, a server network, a computing network, a cloud computing network, etc. Thus, when referring to "a server" or "the server," this includes multiple servers.

[0268] Generally, the server may comprise or include an engine or module for identifying, selecting, and / or generating notifications to be sent or transmitted to the user device, e.g., in response to one or more queries from the user device. The server may comprise, e.g., a natural language processing (“NLP”) module or engine configured to process, analyze, and / or resolve one or more queries received from the user device. The server may be further configured to determine, e.g., based on processing of the one or more queries, notifications to be sent to the user device in response to the one or more queries.

[0269] Additionally, the server may comprise an artificial intelligence module and / or recommendation engine that enables the server to determine, identify, select, and / or generate a notification that may represent an appropriate or most appropriate notification from among or based on a plurality of notifications that may be transmittable to the user device, for example, in response to one or more queries from the user device. By way of example, the server may include a knowledge base that includes a plurality of notifications that may be transmittable to the user device, and the server's artificial intelligence module or recommendation engine may be configured to determine, select, identify, and / or generate an appropriate or most appropriate notification from among or based on a plurality of potential notifications, for example, based on categorizing one or more queries received from the user device.

[0270] The user's communication arrangement may be configured to couple with corresponding communication circuitry of the user device to enable data exchange between the user device and the server. By way of example, the server may be coupled to and / or configured to communicate with the user device (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0271] The control arrangement of the server may refer to a controller, a control module, and / or a control arrangement.

[0272] Additionally, the server may include a data storage device or memory for storing, for example, sentiment scores, notifications, and / or queries. Alternatively, or additionally, computer programs, instructions, apps, applications, and / or software instructions may be stored in the data storage device or memory.

[0273] The control arrangement may include a knowledge base and may be configured to select further notifications for transmission to the user device based on the received sentiment score.

[0274] The control arrangement may include a reinforcement learning model, the sentiment score may be correlated with a reinforcement learning reward for training the reinforcement learning model, and the control arrangement may be configured to train the reinforcement learning model based on the received sentiment score.

[0275] The control arrangement may be configured to train the reinforcement learning model based on maximizing and / or optimizing a reward function of the reinforcement learning model.

[0276] The communication arrangement may be further configured to receive a query from the user device, and the control arrangement may be configured to transmit a notification from the server to the user device upon receiving the query from the user device.

[0277] The communication arrangement may be configured to receive a further query from the user device, and the control arrangement may be configured to select a further notification based on the received sentiment score and based on the received further query.

[0278] The control arrangement may be configured to transmit further notifications to the user device in response to further queries via the communications arrangement.

[0279] The control arrangement may further include a natural language processing engine configured to process the query and / or the further query using the natural language processing engine.

[0280] The communication arrangement may be further configured to receive at least one further sentiment score, and the control arrangement may be further configured to select, from the knowledge base, a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score.

[0281] At least one further sentiment score may be received by the server, from the user device, and / or from the user-side device via the communication arrangement.

[0282] A control arrangement may be configured to compare the sentiment score with at least one further sentiment score.

[0283] The control arrangement may be configured to determine a deviation between the sentiment score and at least one further sentiment score, and to compare the determined deviation to a threshold value for the deviation.

[0284] The control arrangement may be configured to discard and / or ignore at least one of the sentiment score and the at least one further sentiment score if a determined deviation between the sentiment score and the at least one further sentiment score reaches and / or exceeds a threshold value for deviation.

[0285] The control arrangement may include a reinforcement learning model, and the control arrangement may be configured to train the reinforcement learning model based on the sentiment score and / or the at least one further sentiment score only if the sentiment score and the at least one further sentiment score substantially match each other.

[0286] According to a twenty-second aspect of the present disclosure, there is provided a server configured to interactively communicate with a user device and / or a user-side device. The server may be configured to perform, for example, one or more steps of a method according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The server may also refer to the server of the twenty-first aspect of the present disclosure. The server comprises a communication arrangement communicatively coupling the server with the user device and configured to transmit one or more notifications from the server to the user device. The server comprises a control arrangement including a reinforcement learning model, the communication arrangement being further configured to receive a plurality of sentiment scores, e.g., a first sentiment score and a second sentiment score, each of the plurality of sentiment scores correlating with a reinforcement learning reward for training the reinforcement learning model implemented on the server. In this regard, the control arrangement is configured to train the reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores, e.g., the first sentiment score and the second sentiment score, with each other.

[0287] The communication arrangement may be further configured to receive at least one of the plurality of sentiment scores, for example at least one of the first and second sentiment scores, from the user device or from a user-side device.

[0288] At least one of the plurality of sentiment scores, e.g., at least one of the first and / or second sentiment scores, may be indicative of the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0289] At least two of the plurality of sentiment scores, for example, the first and second sentiment scores, may indicate a user's sentiment in response to the at least two notifications received by the user device from the server.

[0290] The communication arrangement may be configured to transmit a first notification and a second notification from a server to the user device, where the second notification may substantially match and / or be equal to the first notification, and the server may be configured to receive a first sentiment score in response to transmitting the first notification to the user device and to receive a second sentiment score in response to transmitting the second notification to the user device.

[0291] The control arrangement may be configured to determine a deviation between at least two of the plurality of sentiment scores, e.g., between a first sentiment score and a second sentiment score, and the control arrangement may be configured to compare the determined deviation to a threshold value for the deviation.

[0292] The control circuitry may be configured to discard and / or ignore at least one of the plurality of sentiment scores, e.g., one of the first sentiment score and the second sentiment score, if a determined deviation between the at least two of the plurality of sentiment scores meets and / or exceeds a threshold value for deviation.

[0293] The control arrangement may be configured to train a reinforcement learning model implemented on the server based on at least one of the received plurality of sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score, only if the at least two sentiment scores, e.g., the first sentiment score and the second sentiment score, substantially match one another.

[0294] The control arrangement is - determining a final sentiment score based on and / or based on at least one of the plurality of sentiment scores, e.g., at least one of the first sentiment score and the second sentiment score; Deriving a reinforcement learning reward from the determined final sentiment score for training a reinforcement learning model; and - feeding the reinforcement learning reward to the reinforcement learning model.

[0295] A twenty-third aspect of the present disclosure relates to the use of a server, as described hereinabove and hereinbelow, for interactively communicating with a user device, and optionally with one or more user-side devices. Alternatively, or additionally, the twenty-third aspect may refer to the use of a server, as described hereinabove and hereinbelow, for determining a sentiment score.

[0296] According to a twenty-fourth aspect of the present disclosure, there is provided a system for interactive communication between a server, a user device, and optionally at least one user-side device. The system may be configured to perform one or more steps of a method, for example, according to any of the first, fourth, seventh, tenth, thirteenth, and fourteenth aspects of the present disclosure. The system includes at least one user device, for example, a user device according to one or more of the seventeenth and eighteenth aspects of the present disclosure, as described hereinabove and hereinbelow. The system further includes a server, for example, a server according to one or more of the twenty-first and twenty-second aspects of the present disclosure, as described hereinabove and hereinbelow. Optionally, the system may include one or more user-side devices and / or one or more user-side sensors, as described hereinabove and hereinbelow.

[0297] A twenty-fifth aspect of the present disclosure relates to the use of a system, as described hereinabove and hereinbelow, for interactive communication between a server and a user device, and optionally one or more user-side devices. Alternatively, or additionally, the twenty-fifth aspect may refer to the use of a system, as described hereinabove and hereinbelow, for determining a sentiment score.

[0298] Below is provided a non-exhaustive list of non-limiting examples, any one or more of the features of these examples may be combined with any one or more features of another example, embodiment, or aspect described herein.

[0299] A.1: A computer-implemented method for interactive communication of a user device with a server, the method comprising: on the user device, providing a notification to a user of the user device; - obtaining reaction data indicative of a user's reaction to the notification; determining a sentiment score for transmission to a server based on the obtained reaction data, the sentiment score indicating a sentiment of the user in reaction to the notification.

[0300] B.1: The method of example A.1, further comprising transmitting the determined sentiment score to a server.

[0301] C.1: The method of example A.1 or B.1, wherein the sentiment score is transmitted from the user device to the server.

[0302] D.1: The method of any of Examples A.1-C.1, wherein the determined sentiment score is transmitted from the user device and / or a user-side device communicatively coupled to the user device.

[0303] E.1: The method of any of embodiments A.1-D.1, further comprising receiving a query at the user device, wherein the notification is provided by the server in response to the query.

[0304] F.1: The method of any of Examples A.1 to E.1, wherein the query includes text input received via a user interface at the user device, verbal input received via a microphone at the user device, and / or video-based input received via a camera at the user device.

[0305] G.1: The method of any of embodiments A.1 to F.1, wherein the notification includes text, audio, at least one image, and / or video.

[0306] H.1: The sentiment score is an anonymized numerical measure that indicates the user's reaction to the notification, and / or The method of any of Examples A.1-G.1, wherein the sentiment score correlates with and / or indicates a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0307] I.1: Determining a sentiment score determining an intermediate sentiment score based on the obtained response data; The method of any of Examples A.1-H.1.

[0308] J.1: The method of example I.1, wherein the sentiment score is anonymized based on normalizing the median sentiment score with the reference sentiment score.

[0309] K.1: The method of example I.1 or J.1, further comprising removing the intermediate sentiment score from the user device when anonymizing the intermediate sentiment score and / or when generating the sentiment score.

[0310] L.1: The method of any of embodiments A.1-K.1, further comprising removing the sentiment score from the user device upon transmitting the sentiment score to the server.

[0311] M.1: The method of any of embodiments A.1-L.1, wherein providing a notification on the user device includes displaying the notification on a user interface of the user device.

[0312] N.1: The method of any of embodiments A.1-M.1, wherein obtaining the response data includes capturing sensor data using at least one sensor of the user device.

[0313] O.1: The method of embodiment N.1, further comprising using the user device to derive response data from captured sensor data of at least one sensor of the user device.

[0314] P.1: The method described in embodiment N.1 or O.1, wherein at least one sensor of the user device is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor.

[0315] Q.1: A method according to any of embodiments A.1 to P.1, wherein acquiring the reaction data includes receiving, using the user device, user-side sensor data from at least one user-side sensor communicatively coupled to the user device.

[0316] R.1: The method of example Q.1, further comprising deriving, using the user device, response data from the received user-side sensor data of at least one user-side sensor.

[0317] S.1: The method of example Q.1 or R.1, wherein at least one user-side sensor is at least one of a camera, acoustic sensor, accelerometer, motion sensor, gyroscope, capacitance sensor, touch sensor, piezoelectric sensor, piezoresistive sensor, Hall sensor, contact blood pressure sensor, photoplethysmography sensor, oximeter, (non-invasive) laser sensor, heart rate sensor, respiration sensor, airflow sensor, air pressure sensor, temperature sensor, electrochemical gas sensor, ultrasonic sensor, acoustic resonance sensor, optical sensor, infrared sensor, near-field sensor, time-of-flight sensor, radar sensor, and bioimpedance sensor.

[0318] T.1: A method described in any of Examples Q.1 to S.1, wherein at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, an IoT device, and an aerosol-generating device.

[0319] U.1: deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment affecting the user's sentiment; The method of any of embodiments A.1-T.1, wherein the sentiment score is determined based on at least one environmental parameter.

[0320] V.1: A method described in any of Examples A.1 to U.1, in which the reaction data is acquired by the user device based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor positioned in the environment of the user device.

[0321] W.1: The method of any of Examples A.1 to V.1, wherein determining the sentiment score includes providing at least a portion of the reaction data to a classifier circuit of the user device, and / or the sentiment score is determined based on classifying at least a portion of the reaction data using a classifier circuit of the user device.

[0322] X.1: capturing, with a camera of the user device, image data indicative of one or more images of a user of the user device; - determining response data based on the captured image data; The method of any of Examples A.1-W.1, further including: determining a sentiment score based on providing at least a portion of the reaction data to a classifier circuit of the user device.

[0323] Y.1: Determining a sentiment score determining, with classifier circuitry of the user device, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - deriving a sentiment score from the determined reaction patterns.

[0324] Z.1: A method as described in any of examples A.1 to Y.1, wherein the sentiment score is determined based on computing a current sentiment score using a user device and deriving a sentiment score from the computed current sentiment score.

[0325] AA.1: A method according to any of Examples A.1 to Z.1, wherein determining the sentiment score includes receiving, using the user device, at least one user-side sentiment score from at least one user-side device communicatively coupled to the user device, and the sentiment score is determined based on the user-side sentiment score received from the at least one user-side device.

[0326] AB.1: A method according to any of Examples A.1 to AA.1, in which the sentiment score is determined based on a current sentiment score computed using the user device and at least one user-side sentiment score received by the user device from at least one user-side device communicatively coupleable to the user device, or at least one user-side sentiment score computed by the user device based on user-side sensor data received from at least one user-side device.

[0327] AC.1: The method of example AB.1, wherein determining the sentiment score includes comparing the current sentiment score and at least one user-side sentiment score.

[0328] AD.1: Determining a sentiment score - determining a deviation between the current sentiment score and at least one user-side sentiment score; The method of example AB.1 or AC.1, comprising: comparing the determined deviation to a threshold value for the deviation.

[0329] AF.1: The method of embodiment AD.1, further comprising discarding at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if the determined deviation between the current sentiment score and the at least one user-side sentiment score meets or exceeds a threshold for deviation.

[0330] AG.1: -preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a deviation threshold; and / or The method of any of embodiments AD.1-AF.1, further comprising transmitting the sentiment score to the server only if the current sentiment score and the at least one user-side sentiment score substantially match each other.

[0331] AH.1: - preventing transmission of the sentiment score to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a deviation threshold; receiving a further notification from the server to the user, the further notification substantially matching or equal to the notification; obtaining further response data indicative of the user's further response to the further notification; The method of any of embodiments AD.1 to AG.1, further comprising: determining a further sentiment score based on the obtained further reaction data, the further sentiment score indicating the user's further sentiment in response to the further notification.

[0332] AI.1: The method of example AH.1, further comprising comparing a further sentiment score determined based on further reaction data indicating the user's further reaction to the further notification with the sentiment score or the current sentiment score determined based on the reaction data indicating the user's reaction to the notification.

[0333] AJ.1: The method of embodiment AH.1 or AI.1, further comprising transmitting at least one of the sentiment score, the current sentiment score, and the further sentiment score to a server only if the further sentiment score, determined based on further reaction data indicative of the user's further reaction to the further notification, substantially matches the sentiment score or the current sentiment score, determined based on reaction data indicative of the user's reaction to the notification.

[0334] AK.1: The method of any of A.1-AJ.1, further comprising aggregating multiple sentiment scores to determine a sentiment score for transmission to a server.

[0335] AL.1: The method of any of examples A.1-AK.1, wherein aggregating the plurality of sentiment scores includes applying an aggregation function.

[0336] AM.1: The method of any of Examples A.1 to AL.1, further comprising determining a privacy level for at least one of the reaction data, sensor data of one or more sensors of the user device, and user-side sensor data of one or more user-side sensors, wherein the privacy level is indicative of one or more features in one or more of the reaction data, sensor data, and user-side sensor data, and the one or more features are to be manipulated in and / or removed from one or more of the reaction data, sensor data, and user-side sensor data to determine and / or before determining the sentiment score.

[0337] AN.1: The method of any of embodiments A.1-AM.1, wherein the privacy level is at least one of definable, user-configurable, and programmable.

[0338] AO.4: A method as described in any of Examples A.1 to AN.1, further comprising selecting one or more processing operations based on a defined privacy level for one or more of the feature groups to process one or more of the reaction data, sensor data, and user-side sensor data that may be selected by the user device for one or more features defined by the one or more feature groups.

[0339] AP.1: The method of any of Examples A.1 to AO.1, wherein the one or more features in one or more of the reaction data, sensor data, and user-side sensor data may include and / or refer to a user's reaction features and / or reaction patterns related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expression, changes in the user's skin color, and one or more environmental parameters that affect the user's sentiment.

[0340] AQ.1: A method as described in any of embodiments A.1 to AP.1, wherein the selected one or more processing operations for processing one or more of the reaction data, sensor data, and user-side sensor data depend on the data type of the reaction data, sensor data, and / or user-side sensor data.

[0341] A.2: A computer program comprising instructions that, when executed on one or more processors of a user device, cause the user device to perform the method described in any of Examples A.1 to AQ.1.

[0342] A.3: A non-transitory computer-readable medium storing the computer program of Example A.2.

[0343] A.4: A computer-implemented method for interactive communication of a user device with a server, the method comprising: on the user device, providing one or more notifications to a user of the user device; obtaining reaction data indicative of one or more reactions of a user to one or more notifications; determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores to one another, wherein the final sentiment score is usable by the server to train a reinforcement learning model implemented on the server.

[0344] B.4: The method described in Example A.4, wherein at least one of the plurality of sentiment scores indicates the user's sentiment in response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment, and / or at least two of the plurality of sentiment scores indicate the user's sentiment in response to at least two notifications received from the server.

[0345] C.4. The method of example A.4 or B.4, wherein the final sentiment score is determined based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

[0346] D.4. The method of any of embodiments A.4-C.4, further comprising receiving, using the user device, at least one of the plurality of sentiment scores from at least one user-side device communicatively coupled to the user device.

[0347] E.4: The method of any of Examples A.4-D.4, ​​further including: receiving, using a user device, user-side sensor data from at least one user-side device; and computing at least one of a plurality of sentiment scores based on deriving, using the user device, at least a portion of the reaction data and / or further reaction data from the received user-side sensor data.

[0348] F.4: Obtaining reaction data receiving, with the user device, user-side sensor data from at least one user-side sensor communicatively coupleable to the user device; The method of any of embodiments A.4-E.4, including: - deriving, with the user device, the response data from the received user-side sensor data of the at least one user-side sensor.

[0349] G.4: The method of example F.4, wherein the at least one user-side sensor is at least one of a camera, acoustic sensor, accelerometer, motion sensor, gyroscope, capacitance sensor, touch sensor, piezoelectric sensor, piezoresistive sensor, Hall sensor, contact blood pressure sensor, photoplethysmography sensor, oximeter, (non-invasive) laser sensor, heart rate sensor, respiration sensor, airflow sensor, air pressure sensor, temperature sensor, electrochemical gas sensor, ultrasonic sensor, acoustic resonance sensor, optical sensor, infrared sensor, near-field sensor, time-of-flight sensor, radar sensor, and bioimpedance sensor.

[0350] H.4: The method described in example F.4 or G.4, wherein at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, an IoT device, and an aerosol-generating device.

[0351] I.4: A method as described in any of Examples A.4 to H.4, wherein obtaining the response data includes capturing sensor data using at least one sensor of the user device, and deriving the response data from the captured sensor data of the at least one sensor of the user device using the user device.

[0352] J.4: The method of Example I.4, wherein at least one sensor of the user device is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor.

[0353] K.4: The method of any of embodiments A.4-J.4, further comprising receiving at least one query at the user device, wherein at least one notification is provided by the server in response to the at least one query.

[0354] L.4: The method described in example K.4, wherein at least the query includes text input received via a user interface at the user device, verbal input received via a microphone at the user device, and / or video-based input received via a camera at the user device.

[0355] M.4: The method of any of embodiments A.4-L.4, wherein each of the one or more notifications includes at least one of text, voice, an audio message, a message, an alert, an image, and a video.

[0356] N.4: The method of any of embodiments A.4-M.4, further comprising transmitting the determined final sentiment score to a server.

[0357] O.4: A method according to any of embodiments A.4 to N.4, wherein the determined final sentiment score is transmitted from the user device and / or the determined final sentiment score is transmitted from a user-side device communicatively coupled to the user device.

[0358] P.4: The method of any of embodiments A.4-O.4, wherein the at least one notification includes at least one of text, voice, an audio message, a message, an alert, an image, and a video.

[0359] Q.4: The method of any of examples A.4 to P.4, wherein providing one or more notifications on the user device includes displaying one or more notifications on a user interface of the user device.

[0360] R.4: The method of any of examples A.4 to Q.4, wherein determining a final sentiment score includes determining an intermediate sentiment score based on the obtained response data and anonymizing the intermediate sentiment score, thereby generating a final sentiment score.

[0361] S.4: The method of example R.4, wherein the final sentiment score is anonymized based on normalizing the intermediate sentiment score with the reference sentiment score.

[0362] T.4: The method of any of examples R.4-S.4, further comprising, upon anonymizing the intermediate sentiment scores, removing the intermediate sentiment scores from the user device.

[0363] U.4: The method of any of examples A.4-T.4, further comprising removing the final sentiment score from the user device upon transmitting the final sentiment score to the server.

[0364] V.4: The method of any of examples A.4 to U.4, wherein providing one or more notifications includes receiving a first notification and a second notification at the user device, wherein the second notification substantially matches and / or is equal to the first notification; obtaining response data includes obtaining first response data indicative of the user's first response to the first notification and obtaining second response data indicative of the user's second response to the second notification; and determining a plurality of sentiment scores includes determining a first sentiment score based on the first response data and determining a second sentiment score based on the second response data.

[0365] W.4: The method of any of examples A.4 to V.4, wherein determining the final sentiment score includes determining a deviation between at least two of the plurality of sentiment scores and comparing the determined deviation to a threshold for the deviation.

[0366] X.4: The method of example W.4, further comprising discarding at least one of the at least two sentiment scores if the determined deviation meets or exceeds a threshold for deviation.

[0367] Y.4: -preventing the final sentiment score from being transmitted to the server if the determined deviation between the at least two sentiment scores reaches or exceeds a deviation threshold; and / or The method of example W.4 or X.4, further comprising transmitting the final sentiment score to the server only if the at least two sentiment scores substantially match one another.

[0368] Z.4: The method of any of A.4 to A.4, further comprising aggregating multiple sentiment scores to determine a final sentiment score for transmission to a server.

[0369] AA.4: The method of any of examples A.4-Z.4, wherein aggregating the plurality of sentiment scores includes applying an aggregation function.

[0370] AB.4: The method of any of Examples A.4 to AA.4, further comprising determining a privacy level for at least one of the reaction data, sensor data of one or more sensors of the user device, and user-side sensor data of one or more user-side sensors, wherein the privacy level is indicative of one or more features in one or more of the reaction data, sensor data, and user-side sensor data, and the one or more features are to be manipulated in and / or removed from one or more of the reaction data, sensor data, and user-side sensor data to determine and / or before determining the sentiment score.

[0371] AC.4: The method of any of examples A.4-AB.4, wherein the privacy level is at least one of definable, user-configurable, and programmable.

[0372] AD.4: The method of any of Examples A.4 to AC.4, further comprising selecting one or more processing operations based on a defined privacy level for one or more of the feature groups to process one or more of the reaction data, sensor data, and user-side sensor data that may be selected by the user device for one or more features defined by the one or more feature groups.

[0373] AE.4: The method of any of Examples A.4 to AD.1, wherein the one or more features in one or more of the reaction data, sensor data, and user-side sensor data may include and / or refer to a user's reaction features and / or reaction patterns related to one or more of the user's movement, physical movement, activity, health status, heart rate, facial expression, changes in the user's skin color, and one or more environmental parameters that affect the user's sentiment.

[0374] AF.4: The method of any of Examples A.4 to AE.1, wherein the selected one or more processing operations for processing one or more of the response data, sensor data, and user-side sensor data depend on the data type of the response data, sensor data, and / or user-side sensor data.

[0375] A.5: A computer program comprising instructions that, when executed on one or more processors of a user device, cause the user device to perform the method described in any of Examples A.4 to AF.4.

[0376] A.6: A non-transitory computer-readable medium storing the computer program of Example A.5.

[0377] A.7: A computer-implemented method for interactive communication of a server with a user device, the method comprising: transmitting a notification from the server to the user device; receiving, by a server, a sentiment score, the sentiment score correlating with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on the received sentiment scores.

[0378] B.7: The method of example A.7, wherein the reinforcement learning model is trained based on maximizing a reward function of the reinforcement learning model implemented on the server.

[0379] C.7: The method of example A.7 or B.7, further comprising selecting, from the server's knowledge base, a further notification to be transmitted to the user device based on the received sentiment score.

[0380] D.7: The method of any of embodiments A.7-C.7, further comprising receiving a query from the user device, wherein a notification is transmitted from the server to the user device upon receiving the query from the user device.

[0381] E.7: The method of any of embodiments A.7 to D.7, further comprising receiving an additional query from the user device, wherein the additional notification is selected based on the received sentiment score and based on the received additional query.

[0382] F.7: The method of any of embodiments E.7, further comprising transmitting a further notification to the user device in response to the further query.

[0383] G.7: The method of any of examples A.7-F.7, further comprising processing the query and / or the further query using a natural language processing engine implemented on the server.

[0384] H.7: receiving at least one further sentiment score; The method of any of Examples A.7 to G.7, further comprising: selecting, from the server's knowledge base, a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score.

[0385] I.7: The method of example H.7, wherein at least one additional sentiment score is received by the server, from the user device, and / or from at least one user-side device.

[0386] J.7: The method of example H.7 or I.7, further comprising comparing the sentiment score to at least one additional sentiment score.

[0387] K.7: - determining a deviation between the sentiment score and at least one further sentiment score; The method of any one of embodiments H.7-J.7, further comprising: comparing the determined deviation to a threshold value for the deviation.

[0388] L.7: The method of example K.7, further comprising discarding at least one of the sentiment score and the at least one additional sentiment score if the determined deviation between the sentiment score and the at least one additional sentiment score meets or exceeds a threshold for deviation.

[0389] M.7: The method of any of Examples H.7-L.7, further comprising training a reinforcement learning model implemented on the server based on the sentiment score and / or the at least one additional sentiment score only if the sentiment score and the at least one additional sentiment score substantially match each other.

[0390] A.8: A computer program comprising instructions, which when executed on one or more processors of a server, cause the server to perform the method described in any of embodiments A.7 to M.7.

[0391] A.9: A non-transitory computer-readable medium storing the computer program of Example A.8.

[0392] A.10: A computer-implemented method for interactive communication of a server with a user device, the method comprising: transmitting one or more notifications from the server to the user device; receiving, by a server, a plurality of sentiment scores, each of the plurality of sentiment scores correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; training a reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores.

[0393] B.10: The method of example A.10, wherein at least one of the plurality of sentiment scores is received by the server, from the user device, or from a user-side device.

[0394] C.10: The method of any of Examples A.10-B.10, wherein at least one of the plurality of sentiment scores indicates the user's sentiment in the user's response to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0395] D.10: A method as described in any of embodiments A.10 to C.10, wherein at least one of the plurality of sentiment scores indicates a user's sentiment in response to at least one of the one or more notifications received by the user device from the server.

[0396] E.10: The method of any of Examples A.10 to DC.10, wherein transmitting one or more notifications from the server to the user device includes transmitting a first notification and a second notification from the server to the user device, wherein the second notification substantially matches or is equal to the first notification, and receiving a plurality of sentiment scores includes receiving a first sentiment score in response to transmitting the first notification to the user device, and receiving a second sentiment score in response to transmitting the second notification to the user device.

[0397] F.10: The method of any of examples A.10-E.10, including determining a deviation between at least two of the plurality of sentiment scores and comparing the determined deviation to a threshold for the deviation.

[0398] G.10: The method of example F.10, further comprising discarding at least one of the plurality of sentiment scores if the determined deviation between the at least two of the plurality of sentiment scores meets or exceeds a threshold for deviation.

[0399] H.10: The method of example F.10 or G.10, further comprising training a reinforcement learning model implemented on a server based on at least one of the received plurality of sentiment scores only if the at least two sentiment scores substantially match one another.

[0400] I.10: Training a reinforcement learning model determining a final sentiment score based on at least one of the plurality of sentiment scores; training a reinforcement learning model from the determined final sentiment score and / or deriving a reinforcement learning reward for computing the determined sentiment score based on the final sentiment score; - feeding the reinforcement learning reward to the reinforcement learning model.

[0401] J.10: The method of any of A.10 to I.10, further comprising aggregating a plurality of sentiment scores to determine a sentiment score for transmission to a server.

[0402] K.10: The method of any of examples A.10-J.10, wherein aggregating the plurality of sentiment scores includes applying an aggregation function.

[0403] L.10: The method of any of embodiments A.10-K.10, wherein the reinforcement learning reward is determined, calculated, and / or computed based on at least the final sentiment score and a reward function of the reinforcement learning model.

[0404] M.10: The method of any of examples A.10-L.10, wherein determining and / or computing the reinforcement learning reward includes determining a trend of a plurality and / or series of previous sentiment scores.

[0405] N.10: The method of any of Examples A.10-M.10, further including: determining weights based on determined trends of a plurality and / or series of previous sentiment scores; and determining a reinforcement learning reward based on the determined weights and the final sentiment score.

[0406] O.10: The method of any of embodiments A.10-N.10, further comprising comparing the final sentiment score to a trend of a plurality and / or series of previous sentiment scores and determining a reinforcement learning reward based on the comparison.

[0407] P.10: The method of any of embodiments A.10-O.10, wherein determining the reinforcement learning reward is determined based on one or more previous reinforcement learning rewards, which are determined based on one or more previous sentiment scores.

[0408] A.11: A computer program comprising instructions that, when executed on one or more processors of a server, cause the server to perform the method described in any of embodiments A.10 to P.10.

[0409] A.12: A non-transitory computer-readable medium storing the computer program of example A.11.

[0410] A.13: A computer-implemented method for interactive communication between a user device and a server, the method comprising: transmitting a notification from the server to the user device; on the user device, providing a notification to a user of the user device; - obtaining, using the user device and / or using at least one user-side device, reaction data indicative of a user's reaction to the notification; - determining, using the user device and / or using at least one user-side device, a sentiment score based on the acquired reaction data, wherein the sentiment score indicates the user's sentiment in reaction to the notification, and the sentiment score correlates with a reinforcement learning reward for training a reinforcement learning model implemented on a server; transmitting the determined sentiment score from the user device and / or at least one user-side device to a server; receiving, by a server, a sentiment score; training an enrichment model implemented on the server based on the sentiment scores received by the server.

[0411] A.14: A computer-implemented method for interactive communication between a user device and a server, the method comprising: transmitting one or more notifications from the server to the user device; on the user device, providing one or more notifications to a user of the user device; - obtaining, using the user device and / or using at least one user-side device, reaction data indicative of one or more reactions of the user to the one or more notifications; determining, using the user device and / or using at least one user-side device, a plurality of sentiment scores based on the acquired reaction data, wherein each of the plurality of sentiment scores correlates with a reinforcement learning reward for training a reinforcement learning model implemented on a server; comparing at least two of the determined sentiment scores with each other; transmitting at least one of the plurality of sentiment scores from the user device and / or at least one user-side device to a server; receiving, by a server, at least one of a plurality of sentiment scores; training an enrichment model implemented on the server based on at least one of the plurality of sentiment scores received by the server.

[0412] A.15: A computer program comprising instructions that, when executed on one or more processors of a server, cause the server to perform the method of example A.13 or A.14.

[0413] A.16: A non-transitory computer-readable medium storing the computer program of example A.15.

[0414] A.17: A user device configured to interactively communicate with a server, the user device comprising: a communication circuit communicatively coupling the user device to a server and configured to receive notifications from the server; a user interface configured to provide a notification to a user of the user device; - a control circuit including one or more processors, the control circuit comprising: - obtaining reaction data indicative of a user's reaction to the notification; and a control circuit configured to: determine, based on the obtained reaction data, a sentiment score for transmission to the server, the sentiment score indicating a sentiment of the user in reaction to the notification.

[0415] B.17. The user device of embodiment A.17, wherein the control circuitry is further configured to transmit the determined sentiment score to a server via the communications circuitry of the user device.

[0416] C.17: A user device as described in Example A.17 or B.17, wherein the sentiment score is an anonymized numerical measure indicating the user's reaction to the notification and / or the sentiment score correlates with or indicates a reinforcement learning reward configured to be used by the server to train a reinforcement learning model implemented on the server.

[0417] D.17: The control circuit determining an intermediate sentiment score based on the obtained response data; and The user device of example A.17 or B.17, further configured to: anonymize the intermediate sentiment score to generate a sentiment score.

[0418] E.17: The user device of example D.17, wherein the control circuitry is configured to anonymize the sentiment score based on normalizing the intermediate sentiment score with a reference sentiment score.

[0419] F.17: The user device of example E.17, wherein the control circuitry is configured to remove the intermediate sentiment score from the user device when anonymizing the intermediate sentiment score and / or when generating the sentiment score.

[0420] G.17: The user device of any of embodiments A.17-F.17, wherein the control circuitry is configured to remove the sentiment score from the user device upon transmitting the sentiment score to the server.

[0421] H.17: A user device described in any of Examples A.17 to G.17, wherein the user device further comprises at least one sensor and the control circuitry is further configured to capture sensor data using the at least one sensor of the user device.

[0422] I.17: The user device of embodiment H.17, wherein the control circuitry is further configured to derive response data from captured sensor data of the at least one sensor.

[0423] J.17: A user device as described in embodiment H.17 or I.17, wherein at least one sensor of the user device is at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, and a position sensor.

[0424] K.17: A user device described in any of Examples A.17 to J.17, wherein the communication circuitry is further configured to receive user-side sensor data from at least one user-side sensor communicatively coupled to the user device via the communication circuitry.

[0425] L.17: A user device as described in embodiment K.17, wherein the control circuitry is further configured to derive response data from received user-side sensor data of at least one user-side sensor.

[0426] M.17: A user device as described in Example K.17 or L.17, wherein at least one user-side sensor is at least one of a camera, acoustic sensor, accelerometer, motion sensor, gyroscope, capacitance sensor, touch sensor, piezoelectric sensor, piezoresistive sensor, Hall sensor, contact blood pressure sensor, photoplethysmography sensor, oximeter, (non-invasive) laser sensor, heart rate sensor, respiration sensor, airflow sensor, air pressure sensor, temperature sensor, electrochemical gas sensor, ultrasonic sensor, acoustic resonance sensor, optical sensor, infrared sensor, near-field sensor, time-of-flight sensor, radar sensor, and bioimpedance sensor.

[0427] N.17: A user device as described in Examples K.17 to M.17, wherein at least one user-side sensor is included in at least one user-side device positioned in the vicinity of the user device, and the at least one user-side device is one or more of a smart TV, a smart speaker, a smart watch, a health monitor, and an aerosol-generating device.

[0428] O.17: The control circuit - deriving at least one environmental parameter from the reaction data, the at least one environmental parameter being related to and / or indicative of the user's environment that influences the user's sentiment; The user device of any of Examples A.17 to N.17, further configured to: determine a sentiment score based on at least one environmental parameter.

[0429] P.17: A user device described in any of Examples A.17 to O.17, wherein the control circuit is configured to acquire reaction data based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor positioned within the environment of the user device.

[0430] Q.17: A user device of any of examples A.17 to P.17, wherein the control circuitry includes classifier circuitry configured to classify at least a portion of the response data to determine a sentiment score.

[0431] R.17: A user device described in any of Examples A.17 to Q.17, wherein the user device further comprises at least one camera configured to capture image data indicating one or more images of a user of the user device, and the control circuitry is further configured to determine reaction data based on the captured image data and determine a sentiment score based on classifying at least a portion of the reaction data using a classifier circuitry.

[0432] S.17: The control circuit is determining, with a classifier circuit, a response pattern based on processing at least a portion of the response data, the response pattern being indicative of an emotional expression of the user in response to the notification; - The user device of Example R.17, further configured to: - derive a sentiment score from the determined reaction pattern.

[0433] T.17: A user device as described in any of Examples A.17 to S.17, wherein the control circuitry is configured to determine the sentiment score based on computing a current sentiment score and / or based on deriving the sentiment score from the computed current sentiment score.

[0434] U.17: A user device described in any of Examples A.17 to T.17, wherein the communication circuitry is configured to receive at least one user-side sentiment score from at least one user-side device communicatively coupled to the user device, and the control circuitry is configured to determine a sentiment score based on the user-side sentiment score received from the at least one user-side device.

[0435] V.17: The control circuit - computing a current sentiment score; -A user device described in any of Examples A.17 to U.17, configured to determine the sentiment score based on at least one user-side sentiment score received from at least one user-side device communicatively coupled to the user device, or at least one user-side sentiment score computed by the control circuitry based on user-side sensor data received from at least one user-side device.

[0436] W.17: The user device of example embodiment V.17, wherein the control circuitry is configured to compare the current sentiment score with at least one user-side sentiment score.

[0437] X.17: The control circuit - determining a deviation between the current sentiment score and at least one user-side sentiment score; The user device of embodiment W.17, further configured to: - compare the determined deviation with a threshold value for the deviation.

[0438] Y.17: The user device of example X.17, wherein the control circuitry is further configured to discard at least one of the current sentiment score, the at least one user-side sentiment score, and the sentiment score if a determined deviation between the current sentiment score and the at least one user-side sentiment score reaches or exceeds a threshold for deviation.

[0439] Z.17: The control circuit -preventing the sentiment score from being transmitted to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a deviation threshold; and / or The user device of example X.17 or Y.17, further configured to transmit the sentiment score to the server only if the current sentiment score and the at least one user-side sentiment score substantially match each other.

[0440] AA.17. The control circuit - preventing transmission of the sentiment score to the server if the determined deviation between the current sentiment score and at least one user-side sentiment score reaches or exceeds a deviation threshold; receiving a further notification from the server, the further notification substantially matching or equal to the notification; obtaining further response data indicative of the user's further response to the further notification; A user device described in any of Examples X.17 to Z.17, further configured to: - determine a further sentiment score based on the acquired further reaction data, wherein the further sentiment score indicates a further sentiment of the user in reaction to the further notification.

[0441] AB.17: The control circuit -The user device of Example AA.17, further configured to compare a further sentiment score determined based on further reaction data indicating the user's further reaction to the further notification with the sentiment score or the current sentiment score determined based on the reaction data indicating the user's reaction to the notification.

[0442] AC.17: The control circuit The user device of Example AB.17, further configured to transmit at least one of the sentiment score, the current sentiment score, and the further sentiment score to the server only if the further sentiment score, determined based on further reaction data indicative of the user's further reaction to the further notification, substantially matches the sentiment score or the current sentiment score, determined based on reaction data indicative of the user's reaction to the notification.

[0443] A.18: A user device configured to interactively communicate with a server, the user device comprising: a communication circuit communicatively coupling the user device to the server and configured to receive one or more notifications from the server; a user interface configured to provide one or more notifications to a user of the user device; - a control circuit including one or more processors, the control circuit comprising: obtaining reaction data indicative of one or more reactions of a user to one or more notifications; determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating a sentiment of the user in reaction to the one or more notifications; and determining a final sentiment score for transmission to a server based on comparing at least two of the plurality of sentiment scores; The final sentiment score is usable by the server to train a reinforcement learning model implemented on the server, the user device.

[0444] B.18: A user device as described in Example A.18, wherein at least one of the plurality of sentiment scores indicates a user's sentiment in the user's reaction to at least one environmental parameter related to and / or indicative of the user's environment that affects the user's sentiment.

[0445] C.18: The user device of example A.18 or B.18, wherein at least two of the plurality of sentiment scores indicate a user sentiment in the user's reaction to at least two notifications received from the server.

[0446] D.18: The user device of any of Examples A.18-C.18, wherein the control circuitry is further configured to determine a final sentiment score based on selecting at least one of the plurality of sentiment scores as the final sentiment score.

[0447] E.18: A user device according to any of embodiments A.18-D.18, wherein the communications circuitry is further configured to receive at least one of the plurality of sentiment scores from at least one user-side device communicatively coupleable to the user device.

[0448] F.18: A user device described in any of Examples A.18 to E.18, wherein the communications circuitry is further configured to receive user-side sensor data from at least one user-side device, and the control circuitry is further configured to compute at least one of a plurality of sentiment scores based on deriving further reaction data from the received user-side sensor data.

[0449] G.18: A method according to claim 1, wherein the communication circuitry is configured to receive a first notification and a second notification for a user from a server, the second notification substantially matching or equal to the first notification, and the control circuitry: obtaining first reaction data indicative of a first reaction of the user to the first notification; obtaining second reaction data indicative of a second reaction of the user to the second notification; The user device of any of Examples A.18 to F.18, further configured to: determine a first sentiment score based on the first reaction data; and determine a second sentiment score based on the second reaction data.

[0450] H.18: The control circuit determining a deviation between at least two of the plurality of sentiment scores; The user device of any of examples A.18 to G.18, further configured to: compare the determined deviation to a threshold value for the deviation.

[0451] I.18: A user device as described in embodiment H.18, wherein the control circuitry is further configured to discard at least one of the at least two sentiment scores if the determined deviation meets or exceeds a threshold for the deviation.

[0452] J.18: A user device as described in embodiment H.18 or I.18, wherein the control circuitry is further configured to prevent the final sentiment score from being transmitted to the server if the determined deviation between the at least two sentiment scores reaches or exceeds a threshold for deviation, and / or the control circuitry is further configured to transmit the final sentiment score to the server only if the at least two sentiment scores substantially match each other.

[0453] A.19. Use of a user device of any one of embodiments A.17-J.18 to interactively communicate with a server.

[0454] A.20: A server configured to interactively communicate with a user device, the server comprising: a communication arrangement communicatively coupling a server to a user device and configured to transmit a notification to the user device, the communication arrangement further configured to receive a sentiment score from the user device, the sentiment score indicating a sentiment of the user in response to the notification; and a control arrangement including a reinforcement learning model, The sentiment score is correlated with a reinforcement learning reward for training a reinforcement learning model, The server, wherein the control arrangement is configured to train a reinforcement learning model based on the received sentiment scores.

[0455] B.20: The server of embodiment A.20, wherein the server includes a knowledge base and is configured to select further notifications for transmission to the user device based on the received sentiment score.

[0456] C.20: A server as described in embodiment A.20 or B.20, wherein the control arrangement further includes a reinforcement learning model, the sentiment score is correlated with a reinforcement learning reward for training the reinforcement learning model, and the control arrangement is configured to train the reinforcement learning model based on the received sentiment score.

[0457] D.20: The server of any of embodiments A.20-C.20, wherein the control arrangement is configured to train the reinforcement learning model based on maximizing a reward function of the reinforcement learning model.

[0458] E.20: A server described in any of embodiments A.20 to C.20, wherein the communication arrangement is further configured to receive a query from the user device, and the control arrangement is configured to transmit a notification from the server to the user device upon receiving the query from the user device.

[0459] F.20: A server described in any of embodiments A.20 to E.20, wherein the communication arrangement is configured to receive a further query from the user device, and the control arrangement is configured to select a further notification based on the received sentiment score and based on the received further query.

[0460] G.20: The server of embodiment F.20, wherein the control arrangement is further configured to transmit, via the communication arrangement, a further notification to the user device in response to the further query.

[0461] H.20: The server of any of embodiments A.20-G.20, wherein the control arrangement further includes a natural language processing engine configured to process the query and / or the further query using the natural language processing engine.

[0462] I.20: A server described in any of embodiments A.20 to H.20, wherein the communication arrangement is further configured to receive at least one further sentiment score, and the control arrangement is further configured to select from the knowledge base a further notification to be transmitted to the user device based on the received sentiment score and based on the received further sentiment score.

[0463] J.20: The server of embodiment I.20, wherein at least one additional sentiment score is received by the server from the user device or from a user-side device via a communication arrangement.

[0464] K.20: The server of example I.20 or J.20, wherein the control arrangement is further configured to compare the sentiment score to at least one additional sentiment score.

[0465] L.20: The server of any of embodiments I.20 to K.20, wherein the control arrangement is further configured to determine a deviation between the sentiment score and at least one additional sentiment score and compare the determined deviation to a threshold value for the deviation.

[0466] M.20: The server of embodiment L.20, further configured to discard at least one of the sentiment score and the at least one additional sentiment score if the determined deviation between the sentiment score and the at least one additional sentiment score reaches or exceeds a threshold value for the deviation.

[0467] N.20: A server described in embodiment L.20 or M.20, wherein the control arrangement includes a reinforcement learning model, and the control arrangement is configured to train the reinforcement learning model based on the sentiment score and / or the at least one additional sentiment score only if the sentiment score and the at least one additional sentiment score substantially match each other.

[0468] A.21: A server configured to interactively communicate with a user device, the server comprising: a communications arrangement communicatively coupling the server to the user device and configured to transmit one or more notifications from the server to the user device; a control arrangement including a reinforcement learning model; the communication arrangement is further configured to receive a plurality of sentiment scores, each of the plurality of sentiment scores correlated with a reinforcement learning reward for training a reinforcement learning model implemented on the server; The server, wherein the control arrangement is configured to train a reinforcement learning model implemented on the server based on comparing at least two of the plurality of sentiment scores.

[0469] B.21: The server of embodiment A.21, wherein the communication arrangement is configured to receive, by the server, at least one of the plurality of sentiment scores from the user device or a user-side device.

[0470] C.21: A server as described in embodiment A.21 or B.21, wherein at least one of the plurality of sentiment scores indicates a user's sentiment in response to at least one environmental parameter related to and / or indicative of a user's environment that affects the user's sentiment, and / or at least two of the plurality of sentiment scores indicate a user's sentiment in response to at least two notifications received from the server.

[0471] D.21: A server described in any of Examples A.21 to C.21, wherein the communication arrangement is configured to transmit a first notification and a second notification from the server to the user device, wherein the second notification substantially matches or is equal to the first notification, and wherein the server is configured to receive a first sentiment score in response to transmitting the first notification to the user device and to receive a second sentiment score in response to transmitting the second notification to the user device.

[0472] E.21: A server described in any of embodiments A.21 to D.21, wherein the control arrangement is configured to determine a deviation between at least two of the plurality of sentiment scores, and the control arrangement is configured to compare the determined deviation with a threshold value for the deviation.

[0473] F.21: The server of embodiment E.21, wherein the control arrangement is configured to discard at least one of the plurality of sentiment scores if the determined deviation between the at least two of the plurality of sentiment scores reaches or exceeds a threshold for deviation.

[0474] G.21: A server described in embodiment E.21 or F.21, wherein the control arrangement is configured to train a reinforcement learning model implemented on the server based on at least one of the received plurality of sentiment scores only if the at least two sentiment scores substantially match each other.

[0475] H.21: The control arrangement is determining a final sentiment score based on at least one of the plurality of sentiment scores; Deriving a reinforcement learning reward from the determined final sentiment score for training a reinforcement learning model; and The server of any of embodiments A.21-G.21, further configured to: - feed the reinforcement learning reward to the reinforcement learning model.

[0476] A.22: Use of the server of any of Examples A.20-H.21 to interactively communicate with a user device.

[0477] A.23: A system for interactive communication between a server and a user device, the system comprising: A user device according to any of embodiments A.17 to J.18; and A system comprising: a server according to any one of embodiments A.20 to H.21.

[0478] The embodiments will now be further described with reference to the figures. [Brief explanation of the drawings]

[0479] [Figure 1] FIG. 1 illustrates a system for interactive communication between a server and at least one user device. [Figure 2] FIG. 2 shows a user device for interactive communication with a server. [Figure 3] FIG. 3 shows a server for interactive communication with a user device. [Figure 4] FIG. 4 shows a flow chart illustrating a method for interactive communication. [Figure 5] FIG. 5 shows a flowchart illustrating a method for interactive communication of a user device with a server. [Figure 6] FIG. 6 shows a flowchart illustrating a method for interactive communication of a user device with a server. [Figure 7] FIG. 7 shows a flowchart illustrating a method for interactive communication of a server with a user device. [Figure 8] FIG. 8 shows a flowchart illustrating a method for interactive communication of a server with a user device. [Figure 9] FIG. 9 shows a flowchart illustrating a method between a user device and a server. [Figure 10]FIG. 10 shows a flowchart illustrating a method between a user device and a server.

[0480] The figures are merely schematic and not to scale. As a rule, identical or similar parts, elements and / or steps are provided with identical or similar reference numbers in the figures.

[0481] Figure 1 shows a system 500 for interactive communication between a server 100 and at least one user device 10. Figure 2 shows a user device 10 for interactive communication with the server 100. Figure 3 shows a server 100 for interactive communication with the user device 10. Unless otherwise stated, reference is made to any or all of Figures 1-3 below.

[0482] The system 500 comprises at least one user device 10 and at least one server 100 .

[0483] 1 and 2, the user device 10 is a handheld user device 10. Alternatively, the user device 10 may be a standalone or fixedly installed device. By way of example, the user device 10 may refer to a handheld, a smartphone, a personal computer ("PC"), a tablet PC, a laptop, a computer, or the like.

[0484] The user device 10 comprises a user interface 12, e.g., a display and / or a touch display, configured to receive user input, e.g., one or more queries, and / or to provide, e.g., display, one or more notifications 14, 16 to a user of the user device 10. In the example shown in Figure 1, two notifications are displayed on the user interface 12.

[0485] The user device 10 further comprises a microphone 13 for providing one or more notifications to the user. It should be noted that multiple notifications 14, 16 may be provided to the user simultaneously or sequentially.

[0486] Additionally, the user device 10 comprises communications circuitry 18 communicatively coupling the user device 10 to the server 100 and configured to receive one or more notifications 14, 16 from the server 100. By way of example, the user device 10 may be coupled to and / or configured to communicate with the server 100 (and vice versa) via an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communications protocol. The communications link or connection between the server 100 and the user device 10 is indicated by reference numeral 101 in FIG. 1 .

[0487] The user device 10 further comprises control circuitry 20 including one or more processors 21. The control circuitry 20 is configured to obtain reaction data indicative of a user's reaction to the one or more notifications 14, 16. The control circuitry 20 is further configured to determine one or more sentiment scores for transmission to the server 100 based on the obtained reaction data, the at least one sentiment score indicative of the user's sentiment in reaction to one or more of the notifications 14, 16.

[0488] The user device 10 further comprises a data storage device 22 or memory 22 for storing, for example, one or more notifications 14, 16, reaction data, any other data, and / or software instructions.

[0489] The user device 10 further includes multiple sensors 24, 26, 28 for capturing and / or acquiring sensor data. For example, the sensor 24 may refer to the camera 24 of the user device 10, which is configured to capture image data, including one or more images, as sensor data. Furthermore, the sensor 26 may be a motion sensor, and the sensor 28 may be a GPS sensor. However, any of the sensors 24, 26, 28 may be a different type of sensor, such as, for example, a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, an optical sensor, an infrared sensor, a near-field sensor, a position sensor, and a global positioning system ("GPS") sensor.

[0490] The server 100 comprises a communication arrangement 102 communicatively coupling the server 100 to the user device 10 and configured to transmit notifications to the user device 10, the communication arrangement 102 being further configured to receive one or more sentiment scores from the user device 10.

[0491] The server 100 further comprises a control arrangement 104 and an artificial intelligence module 108. The server 100 further comprises a reinforcement learning engine or model 110 configured to select one or more notifications 14, 16 for transmission to the user device 10.

[0492] The server 100 further comprises a data storage device 106 having a knowledge base 107 configured to store, for example, one or more notifications 14, 16, one or more queries from the user device 10, one or more sentiment scores, and / or software instructions.

[0493] As described herein above, the sentiment score is an anonymized numerical measure that indicates a user's reaction to one or more notifications 14, 16. Furthermore, the sentiment score correlates with and / or is indicative of a reinforcement learning reward that is configured to be used by the server 100 to train the reinforcement learning model 110 implemented on the server 100.

[0494] The system 500 further includes a user-side sensor 50 and two user-side devices 52 and 54, each having a user-side sensor 51 and 53. Each of the user-side sensors 50, 51, and 53 is configured to capture user-side sensor data. Furthermore, each of the user-side sensors 50, 51, and 53 and / or each of the user-side devices 52 and 54 is configured to transmit the sensor data to the user device 10 via a communication link, such as an Internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0495] Further, each of the user-side sensors 50, 51, 53 and / or each of the user-side devices 52, 54 is configured to transmit one or more sentiment scores to the server 100 via a communication link, such as an internet connection, a WiFi connection, a Bluetooth connection, a cellular network, a 3G connection, an edge connection, an LTE connection, a BUS connection, a wireless connection, a wired connection, a wireless connection, a short-range connection, an IoT connection, or any other connection using any suitable communication protocol.

[0496] The user-side devices 52, 54 may be, for example, a smart television (television), a smart speaker, a smart watch, a health monitor, an IoT (Internet of Things) device, an aerosol-generating device, and the like.

[0497] Furthermore, the user-side sensors 50, 51, 53 may be at least one of a camera, an acoustic sensor, an accelerometer, a motion sensor, a gyroscope, a capacitance sensor, a touch sensor, a piezoelectric sensor, a piezoresistive sensor, a Hall sensor, a contact blood pressure sensor, a photoplethysmography sensor, an oximeter, a (non-invasive) laser sensor, a heart rate sensor, a respiration sensor, an airflow sensor, an air pressure sensor, a temperature sensor, an electrochemical gas sensor, an ultrasonic sensor, an acoustic resonance sensor, an optical sensor, an infrared sensor, a short-range sensor, a time-of-flight sensor, a radar sensor, and a bioimpedance sensor.

[0498] The control circuitry 20 of the user device 10 may be configured to determine an intermediate sentiment score based on the acquired reaction data, and to anonymize the intermediate sentiment score to generate the sentiment score, for example, based on normalizing the intermediate sentiment score with a reference sentiment score, thereby efficiently protecting privacy.

[0499] The determined sentiment score may then be transmitted to the server 100 via the communications circuitry 18. Upon transmission, the sentiment score and / or intermediate sentiment score may be removed from the user device 10. By determining the sentiment score on the user device, the computing load for training the reinforcement learning model 110 on the server 100 may be distributed to one or more user devices 10.

[0500] To determine the sentiment score (also referred to as a final sentiment score), sentiment scores from multiple sources may be used, such as sensors 24, 26, 28, user-side sensors 50, 51, 53, and / or user-side devices 52, 54. For example, sensor data from any of sensors 24, 26, 28 of user device 10 and / or user-side sensor data from any of user-side sensors 50, 51, 53 may be used to determine and / or derive reaction data, and based on the reaction data, a sentiment score may be determined by user device 10.

[0501] For example, one or more queries may be transmitted from the user device 10 to the server 100, and the server 10 may transmit one or more notifications 14, 16 to the user device 10. The camera 24 of the user device 10 may be used to record image(s) of the user's facial reaction to the one or more notifications 14, 16 received from the server. The images of the user's facial reaction are not transmitted to the server and do not leave the user device 10. Instead, the user device 10 identifies the user's facial reaction using, for example, a machine learning classifier 23 and / or classifier circuitry 23 of the user device 10.

[0502] The control circuitry 20 of the user device 10 may then determine a reaction pattern, which may indicate, for example, an emotional expression such as "satisfied" or "irritated" of the user of the user device 10. Based thereon, the user device 10 may determine a sentiment score and transmit the sentiment score to the server 100 for training the reinforcement learning model 110 of the server 100. Thus, it may be possible to personalize and / or improve the overall interactive communication between the server 100 and the user device 10.

[0503] Alternatively, or additionally, the one or more user-side sentiment scores may be determined by one or more of the user-side sensors 50, the user device 52, and the user device 54. The one or more user-side sentiment scores may then be transmitted to the user device 10 and / or the server 100.

[0504] Using multiple sentiment scores from multiple sources, such as sensors 24, 26, 28, 50, 51, 53, the sentiment scores transmitted to and used by server 100 may be verified, as exemplarily described below.

[0505] For example, one or more user-side sentiment scores may be determined by the user device 10 based on sensor data from any of the user-side sensors 50, 51, 53. Alternatively or additionally, one or more user-side sentiment scores may be determined by the user-side sensor 50 and / or the user-side devices 52, 54 and transmitted to the user device 10. Alternatively or additionally, one or more sentiment scores, e.g., current sentiment scores, may be determined by the user device based on sensor data from one or more of the sensors 24, 26, 28. The one or more user-side sentiment scores and / or one or more (current) sentiment scores may then be compared with each other, and a deviation between the scores may be determined, for example, for each pair of sentiment scores, and compared with a threshold for the deviation. If the threshold is exceeded or reached, one or more of the determined (current and / or user-side) sentiment scores may be discarded. If the threshold is not exceeded, one or more of the determined (current and / or user-side) sentiment scores may be used to determine a final sentiment score and / or sentiment score to be transmitted to server 100. The final sentiment score may be determined based on selecting one of the determined (current and / or user-side) sentiment scores as the final sentiment score. Alternatively, multiple sentiment scores may be combined to generate the final sentiment score.

[0506] It should be noted that the verification described herein above may alternatively or additionally be performed on the server 100. To this end, multiple determined (current and / or user-side) sentiment scores may be transmitted to the server 100 via the user device 10 and / or via the user-side devices 52, 54 and compared with each other by the server 100.

[0507] Validation is exemplarily summarized below. User sentiment may be implicitly calculated and / or determined from multiple sources, such as, for example, the smart TV 52, the thermostat 50, and the personal assistant 54. Thus, it may be possible to train the reinforcement learning model 110 only with responses that are the result of the model's 110 response or determination, as opposed to responses based on external factors and / or one or more environmental parameters. For example, the server 100 may receive a negative response and / or sentiment from a user and provide a notification 14, 16 that results in a negative sentiment score. However, this negative sentiment score, sentiment, and / or response may be the result of the temperature in the user's vicinity, rather than from the notification 14, 16. By determining user sentiment scores from multiple sources, it is possible, for example, to use only user sentiment scores that are consistent with each other for training the reinforcement learning model 110 and to ignore those that are anomalous.

[0508] Another possibility is not to immediately classify a particular response or sentiment as correlated with a negative reaction or sentiment if at least two sentiment scores disagree with each other (e.g., one positive and one negative). Instead, the same type or a similar notification 16 may be provided again to the user device 10, e.g., at a different time. The user's reaction or sentiment to the first notification 14 may then be confirmed based on the second notification 16 if there is a discrepancy and / or deviation between the at least two sentiment scores.

[0509] By using multiple sources and / or delaying training until responses, sentiments, and / or sentiment scores are confirmed, the reinforcement learning model 110 may be trained more accurately, for example, when compared to traditional systems that are trained one at a time.

[0510] Various examples and / or advantages of the present disclosure are described below. The system 500 may involve an interactive app, which may be able to converse with the user and provide the user with interesting and relevant information and / or notifications 14, 16, for example, in the form of timely and relevant alerts (e.g., using a “chatbot” software application), in response to a query, or proactively (on-demand). Generally, the system 500 enables interaction personalization. In conventional systems, sentiment is typically derived from user queries and / or responses, e.g., user feedback explicitly provided by the user. This can lead to a dependency on the user to provide this feedback and imply additional effort on the user's part. Also, the feedback is highly individualized and provided specifically by the user without considering the surroundings, location (home, office, market, etc.), other family members present nearby, or any other environmental parameters or factors.

[0511] However, the system 500 according to the present disclosure can provide a highly personalized interactive experience based on implicit user feedback that adapts to the user's feedback and the user's environmental context, which may be considered “implicit feedback.” In this, a sentiment score may be computed based on explicit feedback provided by the user, such as user responses, queries, NPS scores, ratings, etc., and based on implicit feedback, such as facial expression changes upon receiving one or more notifications 14, 16 from the server 100.

[0512] Additionally, environmental aspects, parameters, and / or factors of the user may be taken into account, such as location (home, office, market, etc.), temperature, lighting, time of day, weather, other family members present in the vicinity, etc. Thus, the sentiment score may be used to train the reinforcement learning model 110 to adapt the conversation and / or interactive communication according to the user and / or the user's environment.

[0513] Alternatively or additionally, conversation history may be taken into account. For example, user feedback may be considered in light of user reactions to previous conversations. Thus, it may be possible to ignore sudden changes in user feedback and / or determined sentiment scores that are due to external factors and / or environmental parameters and that may be unrelated to, for example, notifications 14, 16.

[0514] Additionally, privacy-preserving interactions may be provided by the system 500. This may be achieved, for example, by adding a privacy layer to the server 100 and / or the user device 10, for example, in that no private data is transmitted from the user device 10 to the server 100.

[0515] From the perspective of the server 100, the conversation history can be adapted as a sentiment score to adapt and / or personalize the user conversation in a privacy-preserving manner, where the sentiment score does not contain the user's private or personal data. With respect to the reinforcement learning model 110, the reinforcement learning reward and / or sentiment score determination can be distributed, thereby preserving user privacy while providing an accurate representation of the reward function.

[0516] Further examples of the present disclosure are described below: The one or more user-side devices 52, 54 may be, for example, a (standalone) camera, a microphone, a thermostat, a smartwatch, etc. The one or more sensors 13, 24, 26, 28 of the user device 10 may be, for example, a camera, a microphone, an accelerometer embedded in a mobile device, for example, hosting an app for interactive communication on the user device 10.

[0517] The user-side sentiment score may be computed, for example, based on audio, visual, and / or text feedback captured by one or more user-side devices 52, 54 and / or one or more user-side sensors 50, 51, 53. A privacy sensitivity level and / or privacy level, e.g., low, medium, or high, may be defined by the user on the user device. The privacy level may correspond to different aspects, e.g., location, people, health, activity, and / or may indicate one or more features in one or more of the reaction data, sensor data, and user-side sensor data, which are to be manipulated in and / or removed from one or more of the reaction data, sensor data, and user-side sensor data for and / or before determining the (final) sentiment score.

[0518] A t , V t , and T t Let t denote the captured audio, visual, and text feedback during time t.

[0519] For example, V t would correspond to a video frame of a user with family members in the background. In this case, a "low" privacy level for "people" would correspond to keeping the frame or image intact. A "medium" privacy level would correspond to blurring the person's face. A "high" privacy level would correspond to cropping the image to remove the person from the frame entirely.

[0520] Similarly, T twould correspond to a text response provided by a user, e.g., "I'm going on a business trip to Krakow tomorrow." In this case, a "low" privacy level for "location" would correspond to keeping the response as is. A "medium" privacy level would correspond to abstracting "Krakow" to "somewhere in Europe" in the response. A "high" privacy level would correspond to removing the destination completely from the response: "I'm going on a business trip tomorrow."

[0521] Thus, based on the defined privacy level for one or more of the feature groups, one or more processing operations for processing one or more of the reaction data, sensor data, and user-side sensor data may be selected by the user device 10 for one or more features defined by one or more feature groups.

[0522] Furthermore, P(A t ), P(V t ), and P(T t Let P(A ) denote the respective captured sensor data and / or user-side sensor data with privacy protection applied by the user-side sensors 50, 51, 53 and / or user-side devices 52, 54 according to the user-specified privacy level setting. Then, the user-side sensors 50, 51, 53 and / or user-side devices 52, 54 calculate P(A ) t ), P(V t ), and P(T t ) may be shared with the user device 10, for example, with an app on the user device 10.

[0523] Referring to the user device 10, s At =f s (P(A t )), s Vt =f s (P(V t )), s Tt =f s (P(T t)) denotes the user-side sentiment score independently computed based on the respective sensory data or feedback. The (final) sentiment score calculation can be considered as a classifier that outputs a value between a minimum and a maximum, e.g., 1 to 10.

[0524] The user device 10 then aggregates the (independently computed) sentiment scores and generates a consolidated or final sentiment S t The aggregation function can be, for example, a weighted average: S t =1 / 3×[(w A ×s At )+(w V ×s Vt )+(w T ×s Tt )], w i indicates weighting.

[0525] Two or more sentiment scores At , s Vt , and s Tt If there is a significant discrepancy between, e.g., s Vt = 9, but s Tt = 3, different strategies can be applied to integrate them.

[0526] For example, a feedback cycle can be ignored when there is excessive discrepancy between different sentiment scores. This is because the aggregated user sentiment S t This can be indicated by assigning a value of 0 to

[0527] Alternatively, a higher weighting or weighting may be applied to, for example, explicit versus implicit feedback, e.g., s Tt is computed based on user type responses or user inputs, while s Vt is computed based on the background frame using the sensor side device 52, 54, a higher weighting is given to s TtFor example, a user may be laughing with a child in a snapshot, but from the text / voice response it appears that the user is stressed. Therefore, the "stressed" sentiment score may be prioritized by assigning a higher weight.

[0528] The output of the user device 10, or an app running it, produces a (consolidated) final sentiment S t Alternatively, the server 100 may determine this sentiment score. In addition, the server 100 may provide an explicit user response or query {P(A t ), P(V t ), P(T t )} is set in the user device, the user-side sensors 50, 51, 53, and / or the user-side devices 52, 54. For example, in the case of a chat conversation, according to the "Location"-"Medium" privacy level, the user device 10 may record the S along with the user response "I'm going on a business trip to Europe tomorrow." t may be sent to the server 100.

[0529] Additionally, server 100 may be configured to compute, calculate, and / or derive a reinforcement learning reward based on one or more sentiment scores, e.g., a final sentiment score, as exemplarily described below.

[0530] To train the reinforcement learning model 110, two functions of the reinforcement learning (RL) engine or model 110 may be considered: a reward function and a reinforcement learning (RL) agent policy, which together may regulate the content personalization of the RL engine or model 110. The catalog of categorized content may be considered, for example, in a knowledge base, and the server 100 or app may provide recommendations, notifications 14, 16, and / or (chat) responses, which may be grouped into categories related to the user's interests, e.g., travel, entertainment, health, places, etc.

[0531] Determined user sentiment S t Based on the last RL engine (measure a t ) RL rewards corresponding to recommendations, notifications 14, 16, and / or responses r t The reward function f r can be given as:

[0532] r t =f r (S t ), where function f r The logic can be given as follows: (S t =0), this learning loop and / or sentiment score may be ignored due to inconsistencies in the sentiment scores, as explained herein above.

[0533] According to history normalization:S t For =[1-10], S t The value is the reward value r t Before assigning it to , it may be further normalized based on historical background. The historical background is determined as follows:

[0534] For continuous (continuous) conversations, the current sentiment score S t To normalize the effect, t-1 For example, if the conversation sentiment score has already dropped, the low current sentiment S t can be assumed not to be a failure of the last measure alone, so the reward r t can be calibrated accordingly. Meanwhile, the declining sentiment curve SC t-1 Considering this, high sentiment S t would imply that the last action had a very positive impact on the user, and therefore its corresponding reward may be further boosted and / or increased.

[0535] In the case of special recommendations, a similar normalization logic can be applied, where the current sentiment S t can be calibrated according to previously received feedback (sentiment scores) for recommendations of the same category as that of the last measure.

[0536] Regarding delayed rewards, for both continuous conversations and special recommendations, the RL reward for the mth measure from the end is [a t , a t-1 , …, a t-m ] can also be combined and applied (retroactively) to other strategies. For example, given the current sentiment S for a recommendation in the category "travel" to which a user is known to have responded very positively in the past (to other "travel" recommendations), t When is low, the delayed reward strategy simply t , S t ) buffer and try another recommendation in the same category, a t and a t+1 This provides instructions to the RL agent (policy) to "verify" the user sentiment score before updating the rewards for both the

[0537] Thus, the server 100 may be configured to determine and / or compute a reinforcement learning reward based on determining a trend of a plurality and / or series of previous sentiment scores, e.g., a plurality and / or series of previously determined final sentiment scores determined at least in part based on one or more notifications provided to a user. Further, a weight may be determined based on the determined trend of the plurality and / or series of previous sentiment scores, and the reinforcement learning reward may be determined based on the determined weight and the final sentiment score. Alternatively, or additionally, the final sentiment score may be compared to the trend of the plurality and / or series of previous sentiment scores, and the reinforcement learning reward may be determined based on the comparison. Alternatively, or additionally, the reinforcement learning reward may be determined based on a plurality of previously determined reinforcement learning rewards.

[0538] Regarding the RL agent policy function, a policy (1-p) can be considered along the lines of an epsilon-ready strategy, where the agent policy is to attempt "exploration" with a (configurable) probability p. Two adaptations can be envisaged to accommodate reward function strategies (discussed above), e.g., "delayed reward".

[0539] In the delayed reward case, the RL agent does not apply policy (1-p) to select the next best action, but rather chooses the last action a t+1 ~a t and may provide "similar" recommendations.

[0540] For hierarchical policies, given a categorized content catalog, a hierarchical (1-p) policy may be applied, where the policy may first be used to select a "category": {travel, entertainment, health, etc.}, and then recommendations within the selected category C are assigned a usage probability q C :(1-q C ) can be applied using p and q C The policy utilization probability values ​​of can be dynamically adapted based on the coverage of recommendations within each category with known rewards.

[0541] 4 shows a flowchart illustrating a method for interactive communication. The interactive communication may be implemented, for example, by the system 500 described with reference to FIGS. 1-3 and / or according to any of the first to twenty-fifth aspects of the present disclosure.

[0542] The left column of Figure 4 shows steps performed by one or more user-side sensors 50, 51, 53 and / or one or more user-side devices 52, 54. The center column of Figure 4 shows steps performed by the user device 10. The right column of Figure 4 shows steps performed by the server 100.

[0543] It should be noted that at least some of the steps of the method of FIG. 4 may be performed simultaneously, and some steps may be performed sequentially.

[0544] In step S1′, user input is received at the user device 10, for example, via the user interface 12. The user input may refer to and / or indicate a query, for example, a question. The query is further transmitted to the server 100 in step S1′ and received by the server 100 in step S1″.

[0545] Further, in step S1″, the server 100 determines at least one notification 14, 16, for example in the form of a response to a query or question. Thus, a notification provided in response to a query or question provided by the user device 10 may be considered a response notification.

[0546] In step S2'', at least one notification 14, 16 is transmitted to the user device 10 and is provided on the user device 10, for example displayed on the user interface 12, in step S2'.

[0547] Further, in step S3', reaction data is acquired by the user device 10, for example using sensor data from one or more sensors 24, 26, 28, and one or more (current) sentiment scores are computed based on the reaction data.

[0548] Further, in step S1, user-side sensor data is captured by one or more user-side sensors 50, 51, 53 and / or on one or more user-side devices 52, .

[0549] In step S2, one or more user-side sentiment scores are determined and transmitted to the user device 10 in step S3.

[0550] In step S4', one or more user-side sentiment scores are received at the user device 10 and compared with one or more (current) sentiment scores determined by the user device, wherein a final sentiment score may be determined in S4'.

[0551] In step S5', at least one of the final sentiment score, the one or more (current) sentiment scores, and the one or more user-side sentiment scores is transmitted to the server 100, which receives one or more of these sentiment scores in step S3''.

[0552] In step S4'', the reinforcement learning ("RL") model 110 of the server 100 is trained based on at least one of the final sentiment score, the one or more (current) sentiment scores, and the one or more user-side sentiment scores.

[0553] Further, in step S5'', a further notification 14, 16 may be sent to the user device 10 or the server 100 may wait for a further query from the user device 10. The further notification may be a response notification provided in response to a further query from the user device 10.

[0554] Various modifications to the method of Figure 4 are possible. For example, the user-side sentiment score may be transmitted to the server 100, and / or the (current) sentiment score may be transmitted to the server 100, which may then determine a final sentiment score.

[0555] Furthermore, one or more of the final sentiment score, the (current) sentiment score, and the user-side sentiment score may be verified on the user device 10 and / or the server 100 based on comparing at least two of these sentiment scores, as described herein above and in more detail herein below.

[0556] 5 shows a flowchart illustrating a method for interactive communication of a user device 10 with a server 100. In particular, the method illustrated in FIG. 5 may correspond to a method according to the first aspect of the present disclosure. The method of FIG. 5 may refer to a method of operating a user device 10, for example, a user device 10 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0557] Step S1 involves providing, on the user device 10, a notification 14, 16 to the user of the user device 10.

[0558] Step S2 involves obtaining reaction data indicative of the user's reaction to the notification 14,16.

[0559] Step S3 includes determining a sentiment score for transmission to the server 100 based on the obtained reaction data, the sentiment score indicating the user's sentiment in reaction to the notification 14,16.

[0560] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0561] 6 shows a flowchart illustrating a method for interactive communication of a user device 10 with a server 100. In particular, the method illustrated in FIG. 6 may correspond to a method according to the fourth aspect of the present disclosure. The method of FIG. 6 may refer to a method of operating a user device 10, for example, a user device 10 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0562] In step S1, one or more notifications 14, 16 are provided to the user on the user device 10.

[0563] Step S2 involves obtaining reaction data indicative of one or more reactions of the user to the one or more notifications 14,16.

[0564] Step S3 includes determining a plurality of sentiment scores based at least in part on the obtained reaction data, at least one of the plurality of sentiment scores indicating the user's sentiment in reaction to the one or more notifications 14, 16.

[0565] In step S4, a final sentiment score is determined for transmission to the server 100 based on comparing at least two of the plurality of sentiment scores with each other, and the final sentiment score is usable by the server 100 to train an artificial intelligence module 110 implemented on the server 100.

[0566] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0567] 7 shows a flowchart illustrating a method for interactive communication of the server 100 with the user device 10. In particular, the method illustrated in FIG. 7 may correspond to a method according to a seventh aspect of the present disclosure. The method of FIG. 7 may refer to a method of operating the server 100, for example, the server 100 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0568] Step S1 involves transmitting a notification 14, 16 from the server 100 to the user device 10.

[0569] Step S2 includes receiving, by the server 100, the sentiment score, which correlates with a reinforcement learning reward for training a reinforcement learning model 110 implemented on the server 100.

[0570] Step S3 involves training a reinforcement learning (“RL”) model 110 implemented on the server 100 based on the received sentiment scores.

[0571] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0572] 8 shows a flowchart illustrating a method for interactive communication of the server 100 with the user device 10. In particular, the method illustrated in FIG. 8 may correspond to a method according to a tenth aspect of the present disclosure. The method in FIG. 8 may refer to a method of operating the server 100, for example, the server 100 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0573] Step S1 involves transmitting one or more notifications 14, 16 from the server 100 to the user device 10.

[0574] Step S2 includes receiving, by the server 100, a plurality of sentiment scores, each of the plurality of sentiment scores correlated with a reinforcement learning reward for training a reinforcement learning model 110 implemented on the server 100.

[0575] Step S3 includes training a reinforcement learning (“RL”) model 110 implemented on the server 100 based on comparing at least two of the plurality of sentiment scores with each other.

[0576] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0577] 9 shows a flowchart illustrating a method between the user device 10 and the server 100. In particular, the method illustrated in FIG. 9 may correspond to a method according to a thirteenth aspect of the present disclosure. The method in FIG. 9 may refer to a method of operating a system 500, for example, the server 500 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0578] Step S1 involves transmitting a notification 14, 16 from the server 100 to the user device 10.

[0579] Step S2 involves providing, on the user device 10, a notification 14, 16 to the user of the user device 10.

[0580] Step S3 involves obtaining, using the user device 10 and / or using at least one user-side device 52, 54, reaction data indicative of a user's reaction to the notification 14, 16.

[0581] Step S4 includes determining, using the user device 10 and / or using at least one user-side device 52, 54, a sentiment score based on the acquired reaction data, the sentiment score indicating the user's sentiment in reaction to the notification 14, 16, and the sentiment score correlating with a reinforcement learning reward for training the reinforcement learning model 110 implemented on the server 100.

[0582] Step S5 involves transmitting the determined sentiment score from the user device 10 and / or from at least one user-side device 52, 54 to the server 100.

[0583] Step S6 involves receiving, by the server 100, the sentiment score.

[0584] Step S7 includes training a reinforcement learning (“RL”) model 110 implemented on the server 100 based on the sentiment scores received by the server 100.

[0585] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0586] 10 shows a flowchart illustrating a method between the user device 10 and the server 100. In particular, the method illustrated in FIG. 9 may correspond to a method according to a fourteenth aspect of the present disclosure. The method of FIG. 10 may refer to a method of operating a system 500, for example, the server 500 described with reference to any of the above figures or any of the first to twenty-fifth aspects of the present disclosure.

[0587] Step S1 involves transmitting one or more notifications 14, 16 from the server 100 to the user device 10.

[0588] Step S2 involves providing, on the user device 10, one or more notifications 14, 16 to the user of the user device 10.

[0589] Step S3 involves obtaining, using the user device 10 and / or using at least one user-side device 52, 54, reaction data indicative of one or more reactions of the user to the one or more notifications 14, 16.

[0590] Step S4 includes determining, using the user device 10 and / or using at least one user-side device 52, 54, a plurality of sentiment scores based on the acquired reaction data, each of the plurality of sentiment scores correlating with a reinforcement learning reward for training the reinforcement learning model 110 implemented on the server 100.

[0591] Step S5 includes transmitting at least one of the plurality of sentiment scores from the user device 10 and / or at least one user-side device 52, 54 to the server 100.

[0592] Step S6 includes receiving, by the server 100, at least one of the plurality of sentiment scores.

[0593] Step S7 includes training a reinforcement learning (“RL”) model 110 implemented on the server 100 based on at least one of the sentiment scores received by the server 100.

[0594] The method may include a number of further steps, for example as described with reference to any of the methods of the first, fourth, seventh, tenth, thirteenth and fourteenth aspects of the present disclosure.

[0595] For purposes of this specification and the appended claims, unless otherwise indicated, all numbers expressing amounts, quantities, percentages, and the like should be understood in all instances to be modified by the term "about." Also, all ranges include the disclosed maximum and minimum points, and include any intermediate ranges therein, which may or may not be specifically recited herein. Thus, in this context, the number A is understood as A ± 20%. Within this context, the number A may be considered to include values ​​that are within the general standard error for measurement of the property that the number A modifies. In some cases, such as when used in the appended claims, the number A may deviate by the percentages recited above, provided that the amount by which A deviates does not materially affect the basic and novel characteristics of the claimed invention. Also, all ranges include the disclosed maximum and minimum points, and include any intermediate ranges therein, which may or may not be specifically recited herein.

[0596] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is exemplary or representative and not restrictive, and the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art, from a study of the drawings, the disclosure, and the appended claims, and can be practiced within the scope of the claims.

[0597] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting their scope.

Claims

1. 1. A computer-implemented method for interactive communication of a user device with a server, the method comprising: providing, on the user device, a notification to a user of the user device; acquiring reaction data indicative of the user's reaction to the notification; determining a sentiment score for transmission to the server based on the obtained reaction data, the sentiment score being an anonymized numerical measure indicative of the user's sentiment in response to the notification; the sentiment score correlates with a reinforcement learning reward usable by the server to train a reinforcement learning model implemented on the server; obtaining the reaction data, receiving, using the user device, user-side sensor data from at least one user-side sensor of an aerosol-generating device communicatively coupleable to the user device; and deriving, using the user device, the response data from the received user-side sensor data of the at least one user-side sensor. method.

2. the determined sentiment score is transmitted from the user device; and / or The method of claim 1 , wherein the determined sentiment score is transmitted from a user-side device communicatively coupled to the user device.

3. determining the sentiment score, determining an intermediate sentiment score based on the obtained response data; and anonymizing the intermediate sentiment score, thereby generating the sentiment score.

4. The method of claim 3 , wherein the sentiment score is anonymized based on normalizing the intermediate sentiment score with a reference sentiment score.

5. The method of claims 3 and 4, further comprising removing the intermediate sentiment score from the user device upon anonymizing the intermediate sentiment score.

6. The method of any one of claims 1 to 5, further comprising removing the sentiment score from the user device when transmitting the sentiment score to the server.

7. obtaining the reaction data, capturing sensor data using at least one sensor of the user device; and deriving, using the user device, the response data from the captured sensor data of the at least one sensor of the user device.

8. deriving at least one environmental parameter from the reaction data, the at least one environmental parameter indicative of an environment of the user that influences the sentiment of the user; The method of any one of claims 1 to 7, wherein the sentiment score is determined based on the at least one environmental parameter.

9. The method according to any one of claims 1 to 8, wherein the reaction data is acquired by the user device based on sensor data of at least one sensor of the user device and based on user-side sensor data of at least one user-side sensor arranged in an environment of the user device.

10. A computer program comprising instructions which, when executed on one or more processors of a user device, cause the user device to perform the method of any one of claims 1 to 9.

11. A non-transitory computer readable medium storing the computer program of claim 10.

12. A user device for interactive communication with a server, said user device being configured to perform the method of any one of claims 1 to 9.

Images