Methods and means for assistance in digital interactions

Abstract

A method is disclosed for assisting a user involved in a digital interaction with a second person. The method is performed in a user device and comprises scanning for information in an area in proximity of the user for capturing data indicating that a third person is seeking the attention of the user, and determining, based on the captured data, whether at least one action is to be taken. A user device, a method in a managing node, a managing node, computer programs and computer program products are also disclosed.

Description

[0001] METHODS AND MEANS FOR ASSISTANCE IN DIGITAL INTERACTIONS

[0002] TECHNICAL FIELD

[0003] The technology disclosed herein relates generally to the field of communication, and in particular to means and methods for assisting a person involved in a digital interaction with another person.

[0004] BACKGROUND

[0005] Wearable technology, “wearables”, are electronic devices that are worn by a user, for instance in form of an accessory or embedded in the users’ clothes. The wearable is small and comprises miniaturized microprocessor(s) and has a wide range of applications, for instance, in medical use (e.g., as heart rate monitoring), as keys, as passwords, and as fitness tracker to mention only a few.

[0006] The wearables are becoming more and more capable, which creates numerous situations in which the user is occupied with digital interactions and less aware of his / her surroundings. It is also difficult for people around the user to know if he is carrying small wireless ear pods, smart glasses, or is occupied with some digital interaction. This is particularly the case as the wearables are small and becoming increasingly smaller. In a combination of, e.g., smart rings as a control entity for a near-eye display and / or ear pods, an approaching person typically does not notice that the person he is meeting is in some way interacting with his smartphone. Hence, when meeting a person, it is difficult to know whether he is available for physical person-to-person interactions. Conversely, it may be difficult for the user of the wearables to notify an approaching person that he has an ongoing communication session, which may cause awkwardness for both parties.

[0007] From the above, it is clear that there is a need for improvements in these regards, both for people using more or less hidden gadgets, such as wearables, and also for persons around them.

[0008] SUMMARY

[0009] An objective of embodiments herein is to address and improve various aspects of an increasing use of wearables that maybe hard to notice. A particular objective is the facilitate for a user using a wearable to let an approaching person know that he is engaged in a digital session, and this without having to interrupt the ongoing session. Another objective is to enable an approaching person, known to the person using the wearable, to easily join a digital session upon the joining being determined suitable. Still another objective is to prevent misinterpretations or confusion when a user of e.g., smart glasses, may appear to gaze at people around him. It maybe difficult for an approaching person to understand if the user, wearing see-through Extended Reality (XR) glasses, is looking at the physically approaching person or at content displayed in the XR glasses. In the latter case the user’s corresponding ongoing gazing may align with the approaching person’s direction just by coincidence. These objectives and others are achieved by the methods, devices, computer programs and computer program products according to the appended independent claims, and by the embodiments according to the dependent claims.

[0010] According to a first aspect there is presented a method for assisting a user that is involved in a digital interaction with a second person. The method is performed in a user device, e.g., ear pods or a smartphone, and comprises scanning for information in an area in proximity of the user for capturing data indicating that a third person is seeking the attention of the user. The method then comprises determining, based on the captured data, whether at least one action is to be taken, such as, for instance, inviting the third person to the digital interaction.

[0011] According to a second aspect there is presented a computer program for a user device for assistance in a digital interaction. The computer program comprises computer program code, which, when run on processing circuitry in the user device causes the user device to perform the method according to the first aspect.

[0012] According to a third aspect there is presented a computer program product comprising a computer program according to the second aspect, and a computer readable storage medium on which the computer program is stored.

[0013] According to a fourth aspect there is presented a user device for assisting a user involved in a digital interaction with a second person. The user device is configured to scan for information in an area in proximity of the user for capturing data indicating that a third person is seeking the attention of the user. The user device is further configured to determine, based on the captured data, whether at least one action is to be taken.

[0014] According to a fifth aspect there is presented a method in a managing node for assisting a user device in servicing a first user involved in a digital interaction with a second person. The method comprises receiving, from the user device, a request for information about at least a third person. The information relates to a preferred action to take in view of the third person when the first user is involved in the digital interaction with the second person. The method further comprises providing the requested information to the user device.

[0015] According to a sixth aspect there is presented a computer program for a managing node for assistance in a digital interaction. The computer program comprises computer program code, which, when run on processing circuitry in the managing node causes the managing node to perform the method according to the fifth aspect.

[0016] According to a seventh aspect there is presented a computer program product comprising a computer program according to the sixth aspect, and a computer readable storage medium on which the computer program is stored.

[0017] According to an eight aspect there is presented a managing node for assisting a user device in servicing a first user involved in a digital interaction with a second person. The managing node is configured to receive, from the user device, a request for information about at least a third person, the information relating to a preferred action to take in view of the third person when the first user is involved in the digital interaction with the second person. The managing node is further configured to provide the requested information to the user device.

[0018] These aspects provide several advantages and are applicable in various scenarios. For instance, a person involved in a digital interaction in form of a conversation with another person, is enabled to easily decide whether to invite a third person, whom she is about to meet, in the conversation. If she herself does not notice the third person, she may be notified by the user device, and she may also be provided with a suggested course of action, e.g., to invite the third person or to indicate to him that she is busy. Additional objectives, features and advantages of the enclosed embodiments will be apparent from the following detailed disclosure, from the attached dependent claims as well as from the drawings.

[0019] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / an / the element, apparatus, component, means, module, action, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, module, action, etc., unless explicitly stated otherwise. The actions of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

[0020] BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The inventive concept is now described, by way of example, with reference to the accompanying drawings, in which:

[0022] Fig. i is a schematic diagram illustrating a system and devices according to embodiments.

[0023] Fig. 2 is a flowchart of exemplary use cases.

[0024] Fig. 3 is a flowchart of embodiments of a method in a user device.

[0025] Fig. 4 is a schematic diagram showing functional units of a user device according to an embodiment.

[0026] Fig. 5 is a schematic diagram showing functional modules of a user device according to an embodiment.

[0027] Fig. 6 shows one example of a computer program product comprising computer readable means according to an embodiment.

[0028] Fig. 7 is a flowchart of embodiments of a method in a managing node.

[0029] Fig. 8 is a schematic diagram showing functional units of a managing node according to an embodiment. Fig. 9 is a schematic diagram showing functional modules of a managing node according to an embodiment.

[0030] Fig. io shows one example of a computer program product comprising computer readable means according to an embodiment.

[0031] DETAILED DESCRIPTION

[0032] The inventive concept will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the inventive concept are shown. This inventive concept may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art. Like numbers refer to like elements throughout the description. Any action or feature illustrated by dashed lines should be regarded as optional.

[0033] Briefly, methods and means are provided for handling various situations wherein a first person is engaged in, for instance, a phone call with a second person, while simultaneously a third person known to him is approaching. The approaching third person may be unaware of the first person being engaged in the phone call, who in turn might not be aware of the third person approaching, or, if he sees the third person approaching, it may be difficult for him to notify that he is busy in a call. The herein disclosed methods and means enable the first person interacting with the second person to perform an action, e.g., to notify the approaching third person about being engaged in a digital activity, such as a phone call. In other embodiments, the first person is enabled to invite the third person to join their conversation or other activity that they are involved in. The methods and means are applicable in various other scenarios as well.

[0034] Fig. 1 illustrates an exemplary scenario in which the present teachings may be applied. In a scenario such as, for instance, a train station or city square, a first person Pi, in the following also denoted first user, is involved in a digital interaction with a second person P2 (also denoted second user herein). In this scenario it is not obvious that they are interacting, and they may, for instance, be at different locations. A third person P3 (also denoted third user herein), not aware of the first person Pi being occupied, is voicing or otherwise expressing an intent for a person-to-person communication with the first person Pi. The first person Pi uses some type of user device 3a, which maybe any type of wearables, e.g., ear pods or a Near Eye Display (NED). The second person P2 also has some type of user device 3b in the digital interaction with the first person Pi. It may therefore be difficult for the third person P3, who may or may not have a user device 3c, to know that the first person Pi is engaged in the digital interaction with the second person P2 and therefore not available for a face-to-face conversation.

[0035] The digital interaction that the first person Pi has with the second person P2, may be of a private, business, or other context, and they may discuss a topic that is of private, business, or of some other nature. Hence, any combination of context, participant(s) and / or topic(s) for the digital session maybe seen as exemplary features that distinguishes their digital interaction.

[0036] In an exemplary situation, the third person P3 approaches the first person Pi and announces “Hey First Person” with an intent to interact with him. A mechanism hosted in, for instance, the first person’s user device 3a, and supported by e.g., a cloud node 2 of a cloud environment 1, may then determine various features on all involved persons Pi, P2, P3, e.g., their respective relations. The cloud node 2 may, for instance, determine the relation between the first person Pi and the second person P2 with respect to the ongoing digital interaction between them. For example:

[0037] - the first person Pi in relation to the second person P2, with respect to the digital interaction between them.

[0038] - the approaching third person P3 in relation to first person Pi, i.e., the announcing third person P3 in relation to the first person Pi,

[0039] - third person P3 in relation to second person P2 with whom the first person Pi has the digital interaction, e.g., a communication session;

[0040] - third person P3 in relation to both the first person Pi and the second person P2.

[0041] Based on the above exemplary relations, all involved person’s context, time of day, discussed topics, etc., maybe determined. The mentioned cloud node 2 may for instance be a host, i.e., a node that participates in user applications, either as a server, client, or both. A server 4 may also be used for providing resources to the cloud node 2. The cloud node 2 may, for instance, be a managing node 2 of, which receives a request from a user device 3a of the first person Pi (and other users) and provides the requested data in response.

[0042] The managing node 2 may hold various information related to each of the first, second and third person Pi, P2, P3. For instance, the managing node 2 may have information on each person’s social connections / network(s) via social platform entries, text messages, e-mail, contact lists, annotated media objects, etc., and by input received from each respective user. Further, the managing node 2 may store entries and / or information on how a given user preferably is responded to in case of being approached by a known person, whether the person is busy or not and / or in a state where he / she may have the possibility to physically (or digitally) respond to an incoming communication session; such information may be denoted a Communication Diversion Policy (CDP) between first, second and third persons, and that further may have contextual entries for the session e.g. for business, private, day, night, in car, etc.

[0043] The user device 3a, 3b, 3c may, for instance, be any type of wearables, e.g., selected among ear pods, smart glasses, smart contact lenses, Near Eye Displays (NEDs), smartwatch, and smartphone, to mention a few examples. The wearable 3a, 3b, 3c carried by the first user Pi and / or the second person P2 (and possibly also by the third person P3) is able to detect at least one feature of the approaching third person P3. The detectable feature may, for instance, be audio, spoken words or gestures from the third person P3 in the vicinity of the first user Pi. Still further examples on detectable features comprise the third person P3 wearing smart glasses and gazing at the first person Pi, and various combinations of the detectable features, such as a trigger event comprising both gazing and a spoken word. In order to detect such one or more features, the user device 3a, 3b, 3c is provided with suitable means, e.g., microphone to capture audio and spoken words, a camera to capture gestures etc. The herein provided methods and means are described next in relation to some exemplary scenarios.

[0044] Fig. 2 is a flow chart describing various scenarios and actions to take in these scenarios. In box 11, person Pi starts an online video communication with a remote person P2. Person 1 is walking towards her office and uses her ear pods and NED. When a triggering event happens, for instance a third person utters the words “Hello, Person i!”, flow continues to box 12.

[0045] In box 12, a scanning and response mechanism operative in the first persons’ Pi device (or devices) is activated, and flow continues to box 13. The mechanism maybe activated by the first person, e.g., when starting the digital interaction with the second person P2, or it maybe activated by detecting a sound or other feature related to an approaching third person P3.

[0046] In box 13, the scanning and response mechanism captures data. The type of data to be captured depends on the capabilities of the scanning and response mechanism. It may, for instance, capture data related to the first person Pi, such as: data on her motion pattern, location, and active apps, and / or data related to proximity, e.g., image and / or audio.

[0047] Meanwhile, in box 15, a third person P3 is walking and notices a person walking towards him. He recognizes that it is the first person Pi, somehow known by him. The third person P3 may also have a wearable at hand. Flow continues to box 16.

[0048] In box 16, the third person P3 gets closer to the first person Pi and he waves and says “Hi, person Pi!”. Flow continues to box 14.

[0049] In box 14, the scanning and response mechanism of the first persons Pi wearable detects one or both of the waving and the spoken words. Flow continues to box 17.

[0050] In box 17, the mechanism assesses the relevance of the third person P3. The third person P3 may be identified and determined whether the first person Pi and / or the second person P2 knows the third person P3. As described earlier, based on the determined relation(s), context, and topics, the managing node 2 may select a CDP attribute to be considered in a communication relation between the three or more considered users. More details on this are given later. Flow continues to decision box 18.

[0051] In decision box 18, if it is determined that the third person P3 is not relevant in view of the ongoing digital interaction between the first and second persons, flow continues to box 26, wherein a notification maybe sent to the third person P3 informing him that the first person is currently busy. If it is instead determined in decision box 18 that the third person P3 is to be invited to the ongoing digital communication, flow continues to box 19.

[0052] In box 19, the mechanism in the user device 3a of the first person Pi determines how to contact the third person, e.g., using a wearable, a smart phone, etc., and flow then continues to decision box 20.

[0053] In decision box 20, it is decided if the user device 3c of the third person is accessible for the digital conversation. If not, flow continues to box 26, wherein again (as in decision box 18) an action for informing the user device 3c of the third person is selected (e.g., notifying being busy). Else, if the user device 3c of the third person is accessible flow continues to box 27.

[0054] In box 27, a preferred communication method is determined. Again, this determination is performed using the mechanism in the in the first person’s user device 3a (and / or using the managing node 2). When communication method is decided on, flow continues to box 22.

[0055] In box 22, the third person receives an invitation, for instance an invitation through a device application (“app”). Flow then continues to box 23.

[0056] In box 23, the third person decides on whether to accept the invitation or rejecting it. Flow continues to box 24.

[0057] In decision box 24: if the third person P3 accepts the invitation, flow continues to box 28 wherein the first person Pi and the second person P2 are notified on that the third person P3 joins their session. If the third person P3 instead rejects the invitation, flow continues to box 25, where the first person Pi and the second person P2 are notified on that the third person P3 rejected the invitation.

[0058] In the following, the mechanism in the first persons Pi user device 3a (Box 17) is described in more details. Based on the determined relation, context and topics the managing node 2 (and / or managing servicer 4) may select a Communication Diversion Policy (CDP) attribute to be considered in a communication relation between the considered group of users. The managing node 2 may then according to information carried in the CDP depending on scenario select either one of allowing inclusion of the third person P3 into the digital interaction (e.g., digital meeting), or provide the selected CDP to the user device 3c of the third person and disclose the CDP to the third user.

[0059] The user device 3a of first person Pi: o determines intent of detected expressed / spoken utterance, such as: o Natural Language Understanding (NLU), Automatic Speech Recognition (ASR), intent recondition “Hey third person P3” o Triggers evaluation of CDP state, for attributes

[0060] ■ first, second and third persons’ respective context, time of day, discussed topics, etc. o Provides request of CDP evaluation towards the managing node 2

[0061] The managing node 2 may be located in any of the third user device 3c and / or cloud node 2 and has, for instance, following tasks: o Obtains CDP classification message from the first user device 3a, where the request message further carries attributes, such as

[0062] ■ Intent: “incoming request to communicate”

[0063] ■ Source object: “Third person P3”

[0064] • {private, business, other; context}

[0065] ■ Target object: “First person Pi”

[0066] ■ Other (fourth user): Fourth person (in the digital communication)

[0067] • {private, business, other; context, discussed topics}

[0068] If the managing server 4 is located partly in the first user device 3a, the above steps of determining relation, context, etc. are preferably executed in-device. However, if the steps are being performed in a cloud node 2, the first user device 3a may still provide certain attributes related to the inbound call for attention from the third user device 3c so that the managing server 4 may resolve user identity, and any affiliations and associations to the first person Pi. The managing function / server may, for instance, perform following for the first person Pi: o evaluate (for acquired CDP classification request for the third person P3):

[0069] ■ whether: private / business, context, known in phonebook, frequent entries in message service / mails, social network, historical data, etc., such, as for instance:

[0070] • third person P3 is family member, and during non-office hours; third person P3 is family member, and during office hours,

[0071] • first person Pi is business contact, and during office hours; first person Pi is business contact, and during non-office hours. o determine if the first person Pi is currently occupied with a digital communication session towards any of at least one second person P2 (i.e., not the third person P3):

[0072] ■ private / business, context, known in phonebook, frequent entries in message service / mails, social network, historical data, etc., such as, for instance:

[0073] • second person P2 is family member, and during non-office hours; second person P2 is family member, and during office hours,

[0074] • second person P2 is business contact, and during office hours; second person P2 is business contact, and during non- office hours. o determine relation of second person P2 (that is occupied with a digital interaction session with the first person Pi) towards the third person P3,

[0075] ■ private / business, context, known in phonebook, frequent entries in message service / mails, social network, historical data, etc., such as, for instance: • third person P3 is family member, and during non-office hours; second person P2 is family member, and during office hours,

[0076] • third person P3 is business contact, and during office hours; third person P3 is business contact, and during non-office hours. o determine topics / context of digital communication session first person Pi / second person P2 (throughout e.g., NLU, Natural Language Processing (NLP)), o evaluate matching between first person’s interaction / policy rules for respective second person P2 and third person P3.

[0077] ■ if third person P3 is a private / business affiliation and context of said face-to-face announcement, and if second person P2 is a private / business affiliation and context of ongoing digital communication,

[0078] ■ to determine any matching between communication policy for approaching third person P3, vs. ongoing communication session with second person P2.

[0079] The managing node 2 may then according to information carried in the selected CDP, and depending on scenario, decide either of:

[0080] • IF “OK”, allow inclusion of the third person P3 into the digital interaction.

[0081] • IF “NOK”, provide towards third person’s user device 3c the selected CDP, and disclose CDP to the third person 3c.

[0082] Exemplifying scenarios on the above (OK and NOK) are given below:

[0083] IF communication policy fulfilment matching == OK:

[0084] - second and third persons P2, P3 are related to each other via business affiliations, and are in same context,

[0085] - add third person P3 into session by methods that transfers control of second persons P2 user device 3b to first persons Pi user device 3a, or - if third person P3 carries no user device, open headset microphone of first person Pi and apply directional microphone capability in a direction towards third person P3 or deactivate noise suppression of first person Pi user device 3a.

[0086] IF communication policy fulfilment == NOK:

[0087] - select a CDP entry in respect to first person Pi and third person P3, and context, and

[0088] - provide toward third user device 3c (according to attributes for first person, ongoing activity, and relation to third person P3) a CDP message,

[0089] - third user device 3c receives, from managing server, a CDP message associated with relation and context of first person Pi and third person P3, as well as third person P3 / second person P2 relation and context (the latter not known / hidden for the third person), where said CDP message may be (depending on third user device capabilities) any combination of textual, audial, visual and / or haptic / tactile, and / or maybe selected depending on: preference of first person Pi, preference of third person P3, environmental info (such indoors / outdoors, dark / bright light), ambient sound levels / attributes, such as low noise (library) or noisy environment (steel mill)

[0090] - third user device 3c discloses the CDP message towards the third user P3.

[0091] Further, a solution is provided where a person being addressed by another person by any of the direct face-to-face or via technology approaches, may automatically respond to the announcer. In that case, the response may be selected in respect to e.g., what the first user device 3a is involved with (what type of digital interaction, business / private, other party, etc.), where the face-to-face interaction is uttered, and the equipment of the announcing third person P3.

[0092] In another set of embodiments, the first person Pi is occupied in a conversation using ear pods, which are not visible to third (announcing) person P3 that starts up a face- to-face communication despite the fact that the first person is occupied. In such scenario exemplary approaches may be:

[0093] Approach #1: announcing user makes use of his equipment and determines a call- through caller’s relevance, purpose and common social context are executed at any of announcing / receiving ends equipment. A third person P3 (announcing part) spots s friend (first person Pi) in a crowd at some distance, and utters “Hey, first person Pi!”

[0094] A user device 3c of the third person then determines intent of the expressed / spoken utterance using NLU and / or ASR, intent recondition “Hey first person Pi”. This triggers an “engage in call-through-communications“ (CTC) state, which CTC state is provided towards the managing server 4 in a CTC message.

[0095] The managing server 4 located in any of third user device 3c and / or cloud node 2 then obtains the CTC message from the first user device 3a, the message comprising e.g., intent, target information. The managing server 4 determines according to content of CTC message:

[0096] ■ Intent: “communicate”

[0097] ■ Target object: “First person Pi”

[0098] The managing server 4 located in any of the third user device 3c and / or managing server / cloud node then:

[0099] - receives from the first user device 3a, a CTC message carrying: Intent: “communicate”; Targetobject: “Third person”,

[0100] - determines probability for <TargetObject> corresponding to a known / addressable physical user identity relevant to first person, in that selects Targetobject: “third person”.

[0101] The managing function / server for the third person P3 then provides a connection request towards a managing server for the first person Pi, where the connection request may comprise attributes for communication to session according to the CTC message “intent” considering one of prioritized methods:

[0102] 1) direct over-the-air (OTA) speech

[0103] 2) speech over technology

[0104] 3) text,

[0105] 4) image / video,

[0106] 5) ... it is realized that several other methods maybe available. The managing function / server for the first person Pi receives a connection request for the third person P3, where the connection request may comprise attributes for communication to session according to CTC message “intent”. The managing function / server for the first person further evaluates connection request for the third person in context of, for instance private / business, context, known in phonebook, frequent entries in message service / mails, social network, historical data, family member, non-office hour, first person being business contact, and during office hours, in various combinations as described earlier. In an aspect it maybe determined if the first person is currently occupied in a digital interaction with another user (not the third user), and if engaged, the user device sets OccupiedTag: TRUE. IF OccupiedTag == TRUE, then the managing server and / or the first user device 3a selects CDP entry in respect to first user and third user, and the context of the first user’s digital interaction with another user, and provides towards third user device, the selected CDP message. The third user device 3c acquires from managing server a CDP message associated with the first person. The CDP message may (depending on capabilities of the third user device 3c) comprise any combination of textual, audial, visual and / or haptic / tactile information. The CDP message selection may furthermore be selected depending, for instance, on one or more of: first user preference, third user preference (if known via out-of-band (OOB) information; e.g. third user may be colour blind and hence a selection of green / red light flickering would be less effective), environmental info, such indoors / outdoors, dark / bright light, ambient sound levels / attributes, such as low noise (library) or noisy environment (steel mill).

[0107] In a related use case, a first and a third user are both occupied in separate communication sessions and, as the two users approach each other, their respective voices are overheard, which may cause confusion.

[0108] Such situations may be avoided if each device can determine either that there is no relation between the two users and mute overheard voice, or, in case the users indeed have a relation, convey information that users are occupied in respective separate calls.

[0109] According to the herein described solutions, any of the respective first user and / or second user devices may upon a trigger word, uttered from another physical party, being detected, the steps of determining if other physical user shows intent of communication are executed. In steps of determining CDP message, consider the other party as “unknown” and / or if known, select that particular context and / or topics in respective first user’s and second user’s digital in-meeting conversations towards their respective other users are non-matching.

[0110] Fig. 3 is a flowchart of various embodiments of a method for assisting a user Pi involved in a digital interaction with a second person P2. The method 30 maybe performed in a user device, such as, for instance, any type of wearable. Examples comprise ear pods, smart glasses, smart contact lenses, Near Eye Display (NED), smartwatch or smartphone, to mention a few. The method 30 comprises scanning 32 for information in an area in proximity of the user Pi. The scanning 32 is performed for capturing data that indicates that a third person P3 is seeking the attention of the user Pi. Such data indicating attention seeking may, for instance, be that the third person P3 is seeking eye contact with the user Pi or that his voice is recognized. The method 30 further comprises determining 34, based on the captured data, whether at least one action is to be taken.

[0111] The method 10 provides a number of advantages. For instance, the method 10 enables a person involved in a digital interaction in form of a conversation with another person, to easily decide whether to invite a third person, whom she is about to meet, in the conversation. If she herself does not notice the third person, she may be notified by her user device, and she may also be provided with a suggested course of action, e.g., to invite the third person or to indicate to him that she is busy.

[0112] In an embodiment, the action is at least one of: inviting the third person P3 to the digital interaction, notifying the third person P3 that the user Pi is unavailable, determining a communication mechanism for inviting the third person P3, notifying the second person P2 that the third person P3 is joining the digital interaction, notifying the third person P3 on being available within an indicated time and notifying the second person P2 that the user Pi needs to put the second person P2 on hold. These actions may be executed without the involvement of the user Pi, or the method 30 may require that the user allows the one or more actions.

[0113] In one embodiment, the user device 3a comprises glasses, e.g., smart glasses with various capabilities, such as providing a hands-free way to access information and interacting with information. The glasses may, for example, be glasses for extended reality (virtual reality, augmented reality, etc.), glasses able to take calls, pictures or videos or playing music. The smart glasses, may be worn by the first user Pi, and the action may comprise automatically changing degree of transparency of the smart glasses when it is detected that a third person P3 is seeking eye contact with the first user Pi. For instance, the smart glasses may automatically become non-transparent if it is determined that the user Pi is unavailable for contact with the third person P3.

[0114] In various embodiments, the determining 34 comprises identifying the third person P3 based on the captured data and the action is to invite or not invite her / him to the digital interaction based on the identification.

[0115] In variations of the above set of embodiments, the determining whether the third person P3 is to be invited to the digital interaction is based on one or more of: nature of the digital interaction, type of relation to the second person P2 and / or the user Pi, topic and / or context of the digital interaction, and availability of the third person P3.

[0116] In various embodiments, the scanning 32 for information comprises capturing data on the user Pi, wherein the data is associated with the digital interaction. For instance, the data may be one or more of: voice, audio, gaze attributes, haptics, gestures, data from an Inertial Measurement Unit, IMU, worn by the user Pi and / or data associated with the third person P3, wherein the data is one or more of: voice, audio, gaze attributes, haptics, gestures, movements, data from an IMU. It is noted that this gives a wide range of options on capturing data for use in deciding type of action. For some user devices 3a, it might be possible to capture only data in form of voice, since the user device 3a only has a microphone and lacks other capturing means, while other user devices 3a might have a large range of data capturing means.

[0117] In some embodiments, the scanning for information 32 further comprises analysing the captured data. The analysis is made for determining an availability level of the user Pi, which analysis may be based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities, to mention a few examples.

[0118] In various embodiments of the method 30, one or both of the scanning for information 32 and determining 34 is performed by the user device 3a. In other embodiments, the information scanning is performed by the user device 3a in cooperation with a managing node 2.

[0119] A user device 3a is also disclosed. The user device 3a is configured to implement the described method 30. In particular, the user device 3a is configured to assist a user Pi involved in a digital interaction with a second person P2. To this end, the user device 3a is provided with means, such as e.g., microphone, processing means, camera etc. The user device 3a is configured to scan for information in an area in proximity of the user for capturing data indicating that a third person P3 is seeking the attention of the user Pi. The user device 3a is further configured to determine, based on the captured data, whether at least one action is to be taken.

[0120] In an embodiment, the user device 3a is configured to select the action among: inviting the third person P3 to the digital interaction, notifying the third person P3 that the user Pi is unavailable, determining a communication mechanism for inviting the third person P3, notifying the second person P2 that the third person P3 is joining the digital interaction and, notifying the third person P3 on being available within x minutes and notifying the second person P2 that the user Pi needs to put the second person P2 on hold.

[0121] In a variation of the above embodiment, the user device 3a comprises smart glasses that are configured to automatically change degree of transparency of smart glasses when detecting a third person seeking eye contact with the user Pi. In some embodiments, wherein the smart glasses are configured to automatically become non-transparent if the user Pi is unavailable for contact with the third person.

[0122] In various embodiments, the user device 3a is configured to determine whether to take any action by identifying the third person P3 and to determine based thereon whether she / he is to be invited to the digital interaction. The user device 3a may be configured to determine whether to invite the third person P3 based on one or more of: nature of the digital interaction, type of relation to the second person P2 and / or the user Pi, topic and / or context of the digital interaction, and availability of the third person P3.

[0123] In various embodiments, the user device 3a is configured to scan for information by capturing data on the user Pi. The data is associated with the digital interaction, and selected among voice, audio, gaze attributes, haptics, gestures, data from an Inertial Measurement Unit IMU worn by the user Pi. Alternatively, or in addition to this data, the data maybe selected among data associated with the third person P3, for instance one or more of: voice, audio, gaze attributes, haptics, gestures, movements, data from an IMU worn by the third person P3.

[0124] In various embodiments, dependent on the above set, the user device 3a is configured to analyse the captured data, and to determine an availability level of the user Pi based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities.

[0125] In various embodiments, the user device 3a is configured to perform the scanning for information and / or the determining in the user device 3a or in the user device 3a in cooperation with a managing node 2.

[0126] In different embodiments, the user device 3a is a wearable selected among: ear pods, smart glasses, smart contact lenses, Near Eye Display, NED, smartwatch or smartphone.

[0127] Fig. 4 is a schematic diagram showing functional units of a user device 3a according to an embodiment. Processing circuitry no is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product 320 (as in Fig. 6), e.g., in the form of a storage medium 130. The processing circuitry 110 may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

[0128] Particularly, the processing circuitry no is configured to cause the user device 3a to perform a set of operations, or actions, as disclosed herein. For example, the storage medium 130 may store the set of operations, and the processing circuitry 110 maybe configured to retrieve the set of operations from the storage medium 130 to cause the user device 3a to perform the set of operations causing the user device 3a to, for instance, scan an area in proximity of the user device 3a and to determine, based on data from the scanning, whether any action is to be taken. The set of operations may be provided as a set of executable instructions. The processing circuitry no is thereby arranged to execute the method 30 as disclosed herein.

[0129] The storage medium 130 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory.

[0130] The user device 3a may further comprise a communications interface 120 for communications with other entities, functions, nodes, and devices, over suitable interfaces. As such the communications interface 120 may comprise one or more transmitters and receivers, comprising analogue and digital components.

[0131] The processing circuitry no controls the general operation of the user device 3a e.g., by sending data and control signals to the communications interface 120 and the storage medium 130, by receiving data and reports from the communications interface 120, and by retrieving data and instructions from the storage medium 130. Other components, as well as the related functionality, of the user device 3a are omitted in order not to obscure the concepts presented herein.

[0132] Fig. 5 is a schematic diagram showing functional modules of a user device 3a according to an embodiment. The user device 3a of figure 5 comprises a number of functional modules; a scanning module 210 configured to scan for information in an area, a determine module 220 configured to determine, based on the captured data, whether at least one action is to be taken. The user device 3a of Fig. 5 may further comprise a number of optional functional modules, such as any of an analysis module 230 configured to analyse captured data. In general terms, each functional module 210 - 230 may in one embodiment be implemented only in hardware and in another embodiment with the help of software, i.e., the latter embodiment having computer program instructions stored on the storage medium 130 which when run on the processing circuitry makes the user device 3a perform the steps mentioned earlier in relation to figure 4. Preferably, one or more or all functional modules 210 - 230 may be implemented by the processing circuitry no, possibly in cooperation with the communications interface 120 and / or the storage medium 130. The processing circuitry no may thus be arranged to from the storage medium 130 fetch instructions as provided by a functional module 210 - 230 and to execute these instructions, thereby performing any actions of the user device 3a as disclosed herein. Fig. 6 shows one example of a computer program product comprising computer readable means according to an embodiment. On this computer readable storage medium 1300 a computer program 1100 can be stored, which computer program 1100 can cause the processing circuitry no and thereto operatively coupled entities and devices, such as the communications interface 120 and the storage medium 130, to execute methods according to embodiments described herein. The computer program 1100 and / or computer program product 320. The processing circuitry no may thus be configured to from the storage medium 1300 fetch instructions as provided by a functional module 210 - 230, and to execute these instructions, thereby performing any steps as disclosed herein.

[0133] In the example of Fig. 6, the computer program product 320 is illustrated as an optical disc, such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. The computer program product 320 could also be embodied as a memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) and more particularly as a non-volatile storage medium of a device in an external memory such as a USB (Universal Serial Bus) memory or a Flash memory, such as a compact Flash memory. Thus, while the computer program 1100 is here schematically shown as a track on the depicted optical disk, the computer program 1100 can be stored in any way which is suitable for the computer program product 320.

[0134] Fig. 7 is a flowchart of embodiments of a method 40 in a managing node 2, which, as noted earlier, may be a cloud entity. The method 40 is usable in assisting a user device 3a in servicing a first user Pi who is involved in a digital interaction with a second person P2. The method 40 comprises receiving 42, from the user device 3a, a request for information about at least a third person P3. As described earlier, the third person P3 may, for instance, approaching the user of the user device 3a and the user device 3a may have detected this. The requested information may relate to a preferred action to take in view of the third person P3 when the first user Pi is involved in the digital interaction with the second person P2. The method 40 further comprises providing 44 the requested information to the user device 3a. In an embodiment, the receiving 42 further comprises a request for information on the second person P2. The information may, for instance, relate to preferences of the second person P2 on participation of the third person P3 in the digital interaction. As a particular example, the second person P2 may prefer not to be disturbed by the third person P3, who might be a friend, in the current activity, which is work related.

[0135] In other embodiments, the method 40 comprises obtaining the requested information from one or more of: social network platforms, text messages, e-mail, contact lists, annotated media objects, historical data, data on use of applications. Various other information sources could also be used.

[0136] In various embodiments, the method 40 comprises receiving further information for analysis and returning information selected among: an availability level of the first user Pi based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities. The managing node 2 may, for instance, receive information from the user device 3b on how the first user Pi moves, analyse this information and determine that the first user Pi is in a hurry and therefore not available for digital interaction.

[0137] A managing node 2 is also provided and maybe a cloud node or cloud server or the like. The managing node 2 may be used for assisting a user device 3a in servicing a first user Pi involved in a digital interaction with a second person P2. The managing node 2 is configured to receive, from the user device 3a, a request for information about at least a third person P3. The information relates to a preferred action to take in view of the third person P3 when the first user Pi is involved in the digital interaction with the second person P2. The managing node 2 is further configured to provide the requested information to the user device 3a.

[0138] In an embodiment, the managing node 2 is configured to obtain the requested information from one or more of: social network platforms, text messages, e-mail, contact lists, annotated media objects. Various similar means may also be used.

[0139] In various embodiments, the managing node 2 is configured to receive further information for analysis and to return information selected among: an availability level of the first user Pi based on one or more of his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities. Fig. 8 is a schematic diagram showing functional units of a managing node 2 according to an embodiment. Processing circuitry 410 is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product 420 (as in Fig. 10), e.g., in the form of a storage medium 430. The processing circuitry 410 may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

[0140] Particularly, the processing circuitry 410 is configured to cause the managing node 2 to perform a set of operations, or actions, as disclosed herein. For example, the storage medium 430 may store the set of operations, and the processing circuitry 410 may be configured to retrieve the set of operations from the storage medium 430 to cause the managing node 2 to perform the set of operations causing the managing node 2 to, for instance, receive a request for information about preferred action(s) for a third person, and to provide the requested information to the user device. The set of operations maybe provided as a set of executable instructions. The processing circuitry 410 is thereby arranged to execute the method 40 as disclosed herein.

[0141] The storage medium 430 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory.

[0142] The managing node 2 may further comprise a communications interface 420 for communications with other entities, functions, nodes, and devices, over suitable interfaces. As such the communications interface 420 may comprise one or more transmitters and receivers, comprising analogue and digital components.

[0143] The processing circuitry 410 controls the general operation of the managing node 2 e.g., by sending data and control signals to the communications interface 420 and the storage medium 530, by receiving data and reports from the communications interface 420, and by retrieving data and instructions from the storage medium 430. Other components, as well as the related functionality, of the managing node 2 are omitted in order not to obscure the concepts presented herein. Fig. 9 is a schematic diagram showing functional modules of a managing node 2 according to an embodiment. The managing node 2 of figure 9 comprises a number of functional modules; a receive module 210 configured to receive a request for information about a third person, a provide module 220 configured to provide the requested information to a user device. The managing node 2 of Fig. 9 may further comprise a number of optional functional modules (indicated by dashed line), such as an obtain module 530 configured to obtain the requested information from various sources, such as social network platforms, for instance. In general terms, each functional module 510 - 530 may in one embodiment be implemented only in hardware and in another embodiment with the help of software, i.e., the latter embodiment having computer program instructions stored on the storage medium 520 which when run on the processing circuitry makes the managing node 2 perform the steps mentioned earlier in relation to figure 7. Preferably, one or more or all functional modules 510 - 530 maybe implemented by the processing circuitry 410, possibly in cooperation with the communications interface 420 and / or the storage medium 430. The processing circuitry 410 may thus be arranged to from the storage medium 430 fetch instructions as provided by a functional module 510 - 530 and to execute these instructions, thereby performing any actions of the managing node 2 as disclosed herein.

[0144] Fig. 10 shows one example of a computer program product comprising computer readable means according to an embodiment. On this computer readable storage medium 2300 a computer program 2100 can be stored, which computer program 2100 can cause the processing circuitry 410 and thereto operatively coupled entities and devices, such as the communications interface 420 and the storage medium 430, to execute methods according to embodiments described herein. The processing circuitry 410 may thus be configured to from the storage medium 2300 fetch instructions as provided by a functional module 510 - 530, and to execute these instructions, thereby performing any steps and / or actions as disclosed herein.

[0145] In the example of Fig. 10, the computer program product 520 is illustrated as an optical disc, such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. The computer program product 520 could also be embodied as a memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) and more particularly as a non-volatile storage medium of a device in an external memory such as a USB (Universal Serial Bus) memory or a Flash memory, such as a compact Flash memory. Thus, while the computer program 2100 is here schematically shown as a track on the depicted optical disk, the computer program 2100 can be stored in any way which is suitable for the computer program product 520.

[0146] The inventive concept has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the inventive concept, as defined by the appended patent claims.

Claims

CLAIMS1. A method (30) for assisting a user (Pi) involved in a digital interaction with a second person (P2), the method (30) being performed in a user device (3a) and comprising:- scanning (32) for information in an area in proximity of the user (Pi) for capturing data indicating that a third person (P3) is seeking the attention of the user (Pi), and- determining (34), based on the captured data, whether at least one action is to be taken.

2. The method (30) as claimed in claim 1, wherein the action is at least one of: inviting the third person (P3) to the digital interaction, notifying the third person (P3) that the user (Pi) is unavailable, determining a communication mechanism for inviting the third person (P3), notifying the second person (P2) that the third person (P3) is joining the digital interaction, notifying the third person (P3) on being available within an indicated time and notifying the second person (P2) that the user (Pi) needs to put the second person (P2) on hold.

3. The method (30) as claimed in claim 1, wherein the user device (3a) comprises smart glasses, and the action comprises automatically changing degree of transparency of the smart glasses upon detecting a third person seeking eye contact with the user (Pi).

4. The method (30) as claimed in claim 3, wherein the smart glasses automatically become non-transparent if the user (Pi) is unavailable for contact with the third person (P3).

5. The method (30) as claimed in any of the preceding claims, wherein the determining (34) comprises identifying the third person (P3) and determining based thereon whether she / he is to be invited to the digital interaction.

6. The method (30) as claimed in claim 5, wherein the determining whether the third person (P3) is to be invited is based on at least one of: nature of the digital interaction, type of relation to the second person (P2) and / or the user (Pi), topic and / or context of the digital interaction, and availability of the third person (P3).

7. The method (30) as claimed in any of the preceding claims, wherein the scanning(32) for information:- comprises capturing data on the user (Pi), the data being associated with the digital interaction, and wherein the data is one or more of: voice, audio, gaze attributes, haptics, gestures, data from an Inertial Measurement Unit, IMU, worn by the user (Pi) and / or- data associated with the third person (P3), wherein the data is one or more of: voice, audio, gaze attributes, haptics, gestures, movements, data from an IMU.

8. The method (30) as claimed in claim 7, wherein the scanning for information (32) further comprises analysing the captured data for determining an availability level of the user (Pi) based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities.

9. The method (30) as claimed in any of the preceding claims, wherein one or both of the scanning for information (32) and determining (34) is performed by the user device (3a) or by the user device (3a) in cooperation with a managing node (2).

10. A computer program (100) for a user device (3a) for assistance in a digital interaction, the computer program (100) comprising computer program code, which, when run on processing circuitry in the user device (3a) causes the user device (3a) to perform the method (30) according to any of claims 1 - 9.

11. A computer program product (120) comprising a computer program (100) according to claim 10, and a computer readable storage medium (130) on which the computer program (100) is stored.

12. A user device (3a) for assisting a user (Pi) involved in a digital interaction with a second person (P2), the user device (3a) being configured to:- scan for information in an area in proximity of the user for capturing data indicating that a third person (P3) is seeking the attention of the user (Pi), and- determine, based on the captured data, whether at least one action is to be taken.

13. The user device (3a) as claimed in claim 12, configured to select the action among: inviting the third person (P3) to the digital interaction, notifying the third person (P3) that the user (Pi) is unavailable, determining a communication mechanismfor inviting the third person (P3), notifying the second person (P2) that the third person (P3) is joining the digital interaction and, notifying the third person (P3) on being available within x minutes and notifying the second person (P2) that the user (Pi) needs to put the second person (P2) on hold.

14. The user device (3a) as claimed in claim 12, wherein the user device (3a) comprises smart glasses being configured to automatically change degree of transparency of the smart glasses upon detecting a third person seeking eye contact with the user (Pi).

15. The user device (3a) as claimed in claim 14, wherein the smart glasses are configured to automatically become non-transparent if the user (Pi) is unavailable for contact with the third person.

16. The user device (3a) as claimed in claim any of claims 12 - 15, configured to determine by identifying the third person (P3) and to determine based thereon whether she / he is to be invited to the digital interaction.

17. The user device (3a) as claimed in claim 16, configured to determine whether the third person (P3) is to be invited based on at least one of: nature of the digital interaction, type of relation to the second person (P2) and / or the user (Pi), topic and / or context of the digital interaction, and availability of the third person (P3).

18. The user device (3a) as claimed in any of claims 12 - 17, configured to scan for information by:- capturing data on the user (Pi), the data being associated with the digital interaction, and wherein the data is one or more of: voice, audio, gaze attributes, haptics, gestures, data from an Inertial Measurement Unit, IMU, worn by the user (Pi) and / or- data associated with the third person (P3), wherein the data is one or more of: voice, audio, gaze attributes, haptics, gestures, movements, data from an IMU.

19. The user device (3a) as claimed in claim 18, configured to analyse the captured data, and to determine an availability level of the user (Pi) based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities.20.The user device (3a) as claimed in any of claims 12 - 19, configured to perform the scanning for information and / or the determining in the user device (3a) or in the user device (3a) in cooperation with a managing node (2).

21. The user device (3a) as claimed in any of claims 12 - 20, comprising a wearable selected among: ear pods, smart glasses, smart contact lenses, Near Eye Display, NED, smartwatch or smartphone.

22. A method (40) in a managing node (2) for assisting a user device (3a) in servicing a first user (Pi) involved in a digital interaction with a second person (P2), the method (40) comprising:- receiving (42), from the user device (3a), a request for information about at least a third person (P3), the information relating to a preferred action to take in view of the third person (P3) when the first user (Pi) is involved in the digital interaction with the second person (P2), and- providing (44) the requested information to the user device (3a).

23. The method (40) as claimed in claim 22, wherein the receiving (42) further comprises a request for information on the second person (P2), the information relating to preferences of the second person (P2) on participation of the third person (P3) in the digital interaction.

24. The method (40) as claimed in claim 22 or 23, comprising obtaining the requested information from one or more of: social network platforms, text messages, e-mail, contact lists, annotated media objects.

25. The method (40) as claimed in any of claims 22 - 24, comprising receiving further information for analysis and returning information selected among: an availability level of the first user (Pi) based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities.

26. A computer program (200) for a managing node (2) for assistance in a digital interaction, the computer program (200) comprising computer program code, which, when run on processing circuitry in the managing node (2) causes themanaging node (2) to perform the method (40) according to any of claims 22 -25-27. A computer program product (220) comprising a computer program (100) according to claim 26, and a computer readable storage medium (230) on which the computer program (200) is stored.

28. A managing node (2) for assisting a user device (3a) in servicing a first user (Pi) involved in a digital interaction with a second person (P2), the managing node (2) being configured to:- receive, from the user device (3a), a request for information about at least a third person (P3), the information relating to a preferred action to take in view of the third person (P3) when the first user (Pi) is involved in the digital interaction with the second person (P2), and- to provide the requested information to the user device (3a).

29. The managing node (2) as claimed in claim 28, configured to obtain the requested information from one or more of: social network platforms, text messages, e-mail, contact lists, annotated media objects.

30. The managing node (2) claimed in any of claims 28 or 29, configured to receive further information for analysis and to return information selected among: an availability level of the first user (Pi) based on one or more of: his / her motion pattern, occupancy in the digital interaction he / she is involved in, occupancy in further digital activities.

Images