Method and system for time-based personalized management in multi-device environment

Abstract

A method and system for time-based personalized management in a multi-device environment is disclosed. Based on the first user input, the method includes identifying at least one of the plurality of smart devices for performing a first action corresponding to the first user input. Further, the method includes determining one or more contextual information associated with the user, the multi-device environment, and the at least one smart device corresponding to the user input. Further, the method includes predicting, using a prediction model, an associated time span of the identified at least one smart device that needs to preserve the context of the first user input. Thereafter, the method includes combining the predicted correlation time span with the identified at least one smart device.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of Internet of Things (IoT) and more particularly to a method and system for time-based personalized management in a multi-device environment. BACKGROUND

[0002] Over the past few years, the development of wireless communication technologies such as Bluetooth and Wi-Fi has laid the foundation for the expansion of the Internet of Things (IoT). These technologies enable seamless connectivity between devices and open up new possibilities for IoT applications. Furthermore, with the increase in the use of smartphones and the accessibility of fast mobile data networks, IoT has gained attention in recent years. This allows users to remotely control and monitor their devices through mobile applications, thereby giving rise to the concept of multi-device IoT environments such as smart homes.

[0003] In IoT, a multi-device environment refers to a network of interconnected devices, wherein the multi-device environment facilitates automation, intelligence, and control of the interconnected devices to provide an immersive experience to the user. In non-limiting examples, the interconnected devices can correspond to, but are not limited to, a smartphone, a tablet computer, a laptop computer, a desktop computer, a smartwatch, a television (TV), an air conditioner (AC), a light, a curtain, a remote, and other connected devices.

[0004] In a conventional multi-device environment, a user can require one or more same devices among the interconnected devices in different rooms of a smart home to fulfill his / her requirements. In a non-limiting example, the user can require an AC in the bedroom and the living room of the smart home. In another non-limiting example, the user can require a TV in the bedroom as well as the living room. Therefore, if the user provides an ambiguous user input to a virtual assistant to control the operation of any of the one or more same devices, the virtual assistant is unable to take action on the intended device within the smart home. Accordingly, the virtual assistant can require follow-up queries to overcome the ambiguity in the user input. Subsequently, the user can provide another ambiguous user input to control a different operation of the intended device. In such a scenario, the virtual assistant can again require follow-up from the user to overcome the ambiguity in the other ambiguous user input. Therefore, such a conventional multi-device environment faces challenges in handling ambiguous user inputs and is therefore incompatible with handling the above-mentioned problem scenarios.

[0005] Reference is now made to Figure 1 , in accordance with the existing state of the art, an example scenario of a conventional multi-device environment is shown. As shown in Figure 1 , a user 102 provides a user input to a virtual assistant of a user device 104 to control an intended device in a multi-device environment. The preconditions of the example scenario correspond to a multi-device environment comprising two TVs, wherein a first TV is installed in a bedroom and a second TV is installed in a living room. Furthermore,Figure 1 Steps 106-120 of FIG. 1 combine to show the problem of follow-up queries with the user in a multi-device environment. In step 106, the user provides a user input to open a TV to the virtual assistant (i.e., Bixby). Since two identical devices are installed in the home, the virtual assistant of the user device 104 cannot identify the intended TV to be turned on. Therefore, to overcome the ambiguity, in step 108, the virtual assistant provides a follow-up query, i.e., "Which TV do you want to turn on?" In step 110, the user provides a user input to turn on the living room TV. In response, the user device 104 facilitates the multi-device environment to turn on the living room TV and provides feedback to the user in step 112. Subsequently, in step 114, the user provides another user input to increase the volume of the TV. Since the user provides a follow-up input command to increase the volume of the TV after providing an input command to turn on the living room TV, the virtual assistant of the user device 104 can relate the follow-up input command to the living room TV in this scenario. This happens because the virtual assistant of the user device 104 does not consider the historical context while processing the follow-up input command. According to the current state of the art solutions, there are challenges in storing the historical context due to various issues. For example, there is not enough memory to store the historical context or the time period for storing the historical context has expired, etc. Therefore, in step 116, the virtual assistant of the user device 104 provides another follow-up query to the user to confirm which TV volume needs to be increased. Further, in step 118, the user confirms that the volume of the living room TV needs to be increased. Thereafter, in step 120, the virtual assistant of the user device 104 confirms to increase the volume of the living room TV based on the user confirmation. Therefore, as disclosed in Figure 1 the conventional approach in

[0006] Additionally, according to the current state of the art, another example scenario of a conventional multi-device environment is shown in Figure 2 Figure 2 ​As shown, in step 202, the user provides a user input to open AC to the virtual assistant (i.e., Bixby). To overcome ambiguity of two same devices, in step 204, the virtual assistant provides a follow-up query, i.e., which AC do you want to turn on? In step 206, the user provides a user input to turn on the living room AC. In response, in step 208, the user device 104 facilitates the multi-device environment to turn on the living room AC and provides feedback to the user. Subsequently, in step 210, the user provides another user input to turn on the TV. Since the user provided a command to turn on the living room AC, the subsequent user input to turn on the TV should be related to the living room TV. Since the conventional multi-device environment fails to capture the historical context and the time period associated with the context, in step 212, the virtual assistant of the user device 104 provides another follow-up query to confirm which TV needs to be turned on. In step 214, the user confirms that the living room TV needs to be turned on. Further, in step 216, the user device 104 confirms turning on the living room TV while facilitating the multi-device environment.

[0007] Further, in accordance with the existing state of the art, in Figure 3 another example scenario of the conventional multi-device environment is shown. The preconditions of the example scenario correspond to a multi-device environment comprising two TVs, wherein the first TV is installed in the bedroom and the second TV is installed in the living room. Steps 302 to 316 are similar to steps 106 to 120. Therefore, for the sake of brevity, the explanation of steps 302 to 316 is omitted herein with respect to the explanation of steps 106 to 120. Further, in step 318, the user provides a user input to turn on the bedroom TV to the virtual assistant (i.e., Bixby). In step 320, the user device 104 confirms the bedroom TV is turned on to the user while facilitating the multi-device environment to turn on the bedroom TV. Further, in step 322, the user provides another user input to increase the TV volume. However, since the instruction is ambiguous, in step 324, the virtual assistant of the user device 104 provides another follow-up query to confirm the intended TV for which the TV volume needs to be increased. In step 326, the user provides a user input that the TV volume of the bedroom TV needs to be increased. Further, in step 328, the user device 104 confirms to the user that the TV volume of the bedroom TV is increased. Therefore, in accordance with the example scenario shown, Figure 3 the example scenario shown, the user needs to provide two user inputs to increase the TV volume, as shown in steps 310 and 322. This also results in increased user inconvenience and the level of user frustration when the user provides multiple answers to the follow-up queries asked by the virtual assistant of the user device 104.

[0008] Therefore, it would be advantageous to provide an improved method and system that overcomes the challenges, limitations, and above-mentioned problems associated with the conventional multi-device environment having multiple IoT-enabled devices. SUMMARY

[0009] TECHNICAL SOLUTION

[0010] This Summary is provided to introduce some aspects of one or more embodiments of the present disclosure, which are further described in the DETAILED DESCRIPTION. This Summary is not intended to identify key or essential inventive concepts of the present disclosure or to delineate the scope of the disclosure.

[0011] According to an embodiment, the present disclosure relates to a method for time-based personalization management in a multi-device environment. Based on a first user input, the method includes identifying at least one smart device, out of a plurality of smart devices in the multi-device environment, for performing a first action corresponding to the first user input. Further, in response to the first user input, the method includes determining one or more contextual information associated with a user corresponding to the user input, the multi-device environment, and the identified at least one smart device. Based on the determined one or more contextual information, the method includes predicting, using a prediction model, a relevant time span for which a context of the first user input needs to be preserved for the identified at least one smart device. Thereafter, the method includes associating the predicted relevant time span with the identified at least one smart device for performing the first action corresponding to the first user input.

[0012] According to another embodiment, the present disclosure relates to a multi-device system for time-based personalization management in a multi-device environment. The multi-device system includes a plurality of smart devices configured to communicate with each other in the multi-device environment. The multi-device system further includes a user device including at least one processor and configured with a virtual assistant. The user device is communicatively coupled with each of the plurality of smart devices via the virtual assistant. The at least one processor is configured to identify, based on a first user input, at least one smart device, out of a plurality of smart devices in the multi-device environment, for performing a first action corresponding to the first user input. In response to the first user input, the at least one processor is configured to determine one or more contextual information associated with a user corresponding to the user input, the multi-device environment, and the identified at least one smart device. Based on the determined one or more contextual information, the at least one processor is configured to predict, using a prediction model, a relevant time span for which a context of the first user input needs to be preserved for the identified at least one smart device. The at least one processor is configured to associate the predicted relevant time span with the identified at least one smart device for performing the first action corresponding to the first user input.

[0013] To further clarify the advantages and features of the present disclosure, a more particular description of the disclosure will be rendered by reference to specific embodiments thereof, which are illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the disclosure and are therefore not to be considered limiting of its scope. The disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0014] These and other features, aspects, and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0015] Figure 1 An example scenario of a conventional multi-device environment is shown in accordance with the state of the art;

[0016] Figure 2 Another example scenario of a conventional multi-device environment is shown in accordance with the state of the art;

[0017] Figure 3 Yet another example scenario of a conventional multi-device environment is shown in accordance with the state of the art;

[0018] Figure 4 A schematic block diagram of a multi-device system for time-based personalized management in a multi-device environment is shown in accordance with embodiments of the present disclosure;

[0019] Figure 5 A schematic block diagram of modules as shown in Figure 4 accordance with embodiments of the present disclosure is shown;

[0020] Figure 6 A dynamic correlation time predictor module based on an artificial intelligence (AI) model as shown in Figure 5 accordance with embodiments of the present disclosure is shown;

[0021] Figure 7 A flowchart of a method for time-based personalized management in a multi-device environment is shown in accordance with embodiments of the present disclosure;

[0022] Figure 8 An example scenario depicting time-based personalization in a multi-device environment is shown in accordance with embodiments of the present disclosure;

[0023] Figure 9 Another example scenario depicting time-based personalization in a multi-device environment is shown in accordance with embodiments of the present disclosure;

[0024] Figure 10 Yet another example scenario depicting time-based personalization in a multi-device environment is shown in accordance with embodiments of the present disclosure; and

[0025] Figure 11 An example scenario depicting time-based personalization based on a rule-based model is shown in accordance with embodiments of the present disclosure.

[0026] Furthermore, those skilled in the art will appreciate that the elements in the figures are shown for the purpose of simplicity and clarity and can not have been necessarily drawn to scale. For example, flow diagrams typically show only those acts performed by the machine or apparatus in accordance with the most prominent procedure to help to improve the understanding of the various aspects of the present disclosure. Also, with respect to the apparatus, one or more components of the apparatus can have been represented in the figures by conventional symbols, and the figures can show only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. DETAILED DESCRIPTION

[0027] For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to various embodiments, and specific language will be used to describe the various embodiments. It will nevertheless be understood that no limitation of the scope of the disclosure is intended by this specification of embodiments, and such alterations and further modifications in the illustrated system, and such further applications of the principles of the present disclosure as illustrated herein are contemplated as would normally occur to one skilled in the art to which the disclosure relates.

[0028] The term "some" or "one or more" as used herein is defined as "one," "more than one," or "all." Thus, the terms "more than one," "one or more," or "all" will fall within the definition of "some" or "one or more." The terms "embodiment," "another embodiment," "some embodiments," or "in one or more embodiments" can refer to one embodiment or several embodiments or all embodiments. Thus, the term "some embodiments" is defined as meaning "one embodiment, or more than one embodiment, or all embodiments."

[0029] The terms and structures employed herein are used for the purpose of describing, teaching, and illustrating some embodiments and particular features and elements thereof, and do not limit, restrict, or reduce the spirit and scope of the claims or their equivalents. The phrase "exemplary" can indicate an example.

[0030] More specifically, any terms used herein, such as but not limited to "comprise," "comprise," "have," "consist of," "include," and grammatical variants thereof, do not specify an exact limitation or constraint, and certainly do not exclude the possible addition of one or more features or elements, unless otherwise stated, and must not be considered as excluding the possible removal of one or more listed features and elements, unless otherwise stated, and except for the limiting language "must comprise" or "need to comprise" to state.

[0031] Whether or not a certain feature or element is used only once, it can still be referred to as “one or more features,” “one or more elements,” “at least one feature,” or “at least one element,” regardless of how it is used. Also, the use of the term “one or more” or “at least one” feature or element does not preclude the existence of the feature or element, unless otherwise specified by limiting language such as “one or more” or “one or more elements are required.”

[0032] Unless otherwise defined, all terms used herein, including technical and / or scientific terms, can be interpreted according to the same meaning as commonly understood by one of ordinary skill in the art.

[0033] Embodiments of the present disclosure will now be described in detail in the following, with reference to the accompanying drawings.

[0034] Figure 4 A schematic block diagram of a multi-device system 400 for time-based personalization management in a multi-device environment, in accordance with embodiments of the present disclosure, is shown. In accordance with embodiments, the multi-device system 400 includes a user device 402, and a plurality of smart devices configured to communicate with each other in a multi-device environment over a communication network 424. Each of the plurality of smart devices corresponds to an electronic device that can be connected to an internet connection and perform various tasks such as controlling various operations of home appliances, monitoring energy consumption, and the like. The plurality of smart devices can be controlled by the user device 402. In accordance with another embodiment, the plurality of smart devices can also be integrated with each other to create a connected and automated smart home environment or an IoT environment. In a non-limiting example, the plurality of smart devices includes, but is not limited to, a TV 404, a remote control 406, a light source 408, and a window curtain 410. The window curtain 410 generally refers to a window covering made of fabric or vinyl, which can be adjusted to control light and privacy.

[0035] In an example embodiment, the user device 402 can correspond to, but is not limited to, a smartphone, other mobile devices, a laptop, a tablet, a computer, and the like.

[0036] In accordance with embodiments, the user device 402 includes at least one processor 412 (hereinafter referred to as processor 412), an input / output (I / O) interface 416, and a memory 418. The processor 412, the I / O interface 416, and the memory 418 are communicatively coupled to each other. The processor 412 includes one or more modules 414 (hereinafter referred to as modules 414) for performing operations for time-based personalization management in a multi-device environment.

[0037] According to an embodiment, the processor 412 can be operatively coupled to the modules 414 for processing, running or executing a set of operations. In another embodiment, the processor 412 can comprise at least one data processor for running processes in a virtual storage area network. The processor 412 can include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units etc. In yet another embodiment, the processor 412 can comprise a central processing unit (CPU), a graphics processing unit (GPU) or both. The processor 412 can be one or more general processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, servers, networks, digital circuits, analog circuits, combination thereof, or other now known or later developed devices for analyzing and processing data. The processor 412 can run one or more instructions such as codes, programs, routines and / or software stored in a memory to perform one or more operations disclosed herein throughout the disclosure.

[0038] According to an embodiment, the term "module" or "a plurality of modules" used herein can mean a unit including one of or a combination of two or more of, for example, hardware, software and firmware. The "module" or "a plurality of modules" can be interchangeably used with terms such as logic, a logical block, a component, etc. The "module" or "a plurality of modules" can be the smallest unit of a device component for performing one or more functions, or can be a part thereof. The processor 412 can control the modules 414 to perform a set of specific operations as described below in the forthcoming paragraphs of the disclosure.

[0039] According to an embodiment, the I / O interface 416 refers to a hardware or software component that enables data communication between the user device 402 and any other device or system. The I / O interface 416 serves as a communication medium for exchanging information, commands, or data with other devices or systems. According to another embodiment, the I / O interface 416 can be a part of the processor 412 or can be a separate component. The I / O interface 416 can be created in software or can be a physical connection of hardware. The I / O interface 416 can be configured to connect with an external network, an external medium, a display, or any other component or combination thereof. The external network can be a physical connection, such as a wired Ethernet connection, or can be established wirelessly. In a non-limiting example, the user device 402 can be configured to receive one or more user inputs for performing one or more desired operations as an upcoming paragraph of the present disclosure. The one or more user inputs can be alternatively disclosed as a first user input, a second user input, and so on throughout the disclosure without deviating from the scope of the present invention. The first user input can correspond to any one of a voice input, a text input, a graphical user interface (GUI) input, a remote control input, and a gesture input of a user. Further, the second user input corresponds to any one of a voice input, a text input, and a gesture input of a user that causes disambiguation. According to an alternative embodiment, the first user input can cause disambiguation when receiving an input from any one of a voice input, a text input, a GUI input, and a gesture input of a user. However, the first user input can not cause disambiguation when received through a remote control input.

[0040] According to an embodiment, the memory 418 can include any non-transitory computer-readable medium known in the art including, for example, volatile memory such as static random access memory (SRAM) and dynamic random access memory (DRAM), and / or non-volatile memory such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disks, optical disks, and magnetic tapes. The memory 418 is communicatively coupled with the processor 412 to store bit streams or processing instructions for completing one or more processes. Further, the memory 418 includes an operating system 422 for performing one or more tasks of the device 402 as performed by a general operating system in a communication domain. Further, the memory 418 includes a database 420 to store information required by the modules 414 and the processor 412 to perform one or more functions for time-based personalization management in a multi-device environment. Further, the memory 418 can store one or more values such as, but not limited to, one or more intermediate data generated by the modules 414, parameters required by the modules 414, threshold values, and the like. Further, the memory 418 can store one or more models for performing operations as disclosed throughout the present disclosure.

[0041] According to an embodiment, the communication network 424 refers to any entity performing one or more functions of network connectivity between the user device 402 and the plurality of smart devices. Further, the network connectivity can be established between the user device 402 and the plurality of smart devices via a communication port or interface or using a bus (not shown). The communication port can be configured to connect with a network, an external medium, a memory, or any other component in the system or a combination thereof. The network connectivity can be a physical connection such as a wired Ethernet connection or can be established wirelessly. Likewise, additional connections with other components of the multi-device system 400 can be physical or can be established wirelessly.

[0042] Figure 5 A schematic block diagram of the module 414 is shown as Figure 4 performed by the module 414 according to an embodiment of the present disclosure.

[0043] According to an embodiment, the module 414 includes an ambiguity resolver module 512, an action executor module 514, and a dynamic correlation time predictor module 516. The module 414 communicates with the user device 402, the virtual assistant 506, and the smart controller 508 to perform a set of operations for time-based personalized management in a multi-device environment. According to an embodiment, the virtual assistant 506 can also be referred to as an artificial intelligence (AI) assistant or a digital assistant. The smart controller 508 can correspond to a controller in the multi-device environment to control operation of at least one smart device in the plurality of smart devices. According to another embodiment, the virtual assistant 506 and the smart controller 508 can also be a part of the user device 402. However, for ease of explanation, the virtual assistant 506 and the smart controller 508 are disclosed as different components in the present disclosure without deviating from the scope of the present disclosure. In a non-limiting example, the virtual assistant 506 can relate to, but can not be limited to, Siri, Bixby, and the like. Figure 5

[0044] ​According to an embodiment, the user device 402 receives the first user input from the user 102. The virtual assistant 506 receives the first user input from the user device 402. Further, the smart controller 508 receives the first user input from the virtual assistant 506 to perform an operation on an intended user device. Thereafter, the module 414 receives the first user input from the smart controller 508. Subsequently, the module 414 determines, via the decision block 510, whether the first user input is an ambiguous user input or a partial user input for performing a first action by at least one smart device. The ambiguous user input or the partial user input involves a user input that does not specifically define the intended at least one smart device among the plurality of smart devices to perform any action. For example, if the multi-device environment includes two identical devices, such as two TVs, the user 102 provides the ambiguous user input or the partial user input as "turn on TV" without specifically disclosing which TV needs to be turned on. Based on the determination of the decision block 510, if the first user input is the ambiguous user input, the flow moves to the ambiguity resolver module 512. Based on the determination of the decision block 510, if the first user input is not ambiguous, the flow moves to the action performer module 514.

[0045] According to an embodiment, the at least one smart device has a same functionality with respect to a group of smart devices among the plurality of smart devices. In a non-limiting example, a TV and a speaker among the plurality of smart devices have a same functionality of increasing and decreasing the volume. Accordingly, the multi-device system 400 is configured to identify at least one smart device among the group of smart devices to perform the first action corresponding to the first user input.

[0046] According to an embodiment, based on the first user input, the ambiguity resolver module 512 identifies at least one smart device among the plurality of smart devices in the multi-device environment for performing the first action corresponding to the first user input. The ambiguity resolver module 512 determines whether corresponding data is available in the database 420 to perform the first action. If the corresponding data is not available, the ambiguity resolver module 512 initiates a prompt to the user for resolving the ambiguity in order to overcome the ambiguity. For example, if the first user input involves "turn on TV" without specifying which TV needs to be turned on, the ambiguity resolver module 512 prompts the user "which TV do you want to turn on?". Based on the prompt resolution response, the action performer module 514 controls the virtual assistant 506 for performing the first action corresponding to the first user input and the prompt resolution response. Alternatively, if the corresponding data is available in the database 420, the ambiguity resolver module 512 fetches the explicit data from the database 420 and sends the explicit data to the action performer module 514 to perform the first action.

[0047] According to an embodiment, if the action executor module 514, after identifying the at least one smart device in the multi-device environment, provides the input to the virtual assistant 506 to perform the first action based on the first user input. The action executor module 514 provides the input to the virtual assistant 506 to perform the first action based on the first user input after resolving the ambiguity from the prompt resolution or from the database 420 via the ambiguity resolver module 512. Alternatively, if there is no ambiguity in the first user input, the action executor module 514 provides the input to the virtual assistant 506 to perform the first action based on the first user input. Further, the action executor module 514 triggers the dynamic relevance time predictor module 516 to predict the relevance time span of the identified at least one smart device. The action executor module 514 triggers the dynamic relevance time predictor module 516 under two conditions. The first condition of the two conditions corresponds to the case when the ambiguity occurs for the first time and there is no data available in the database 420 for the relevance time span. The second condition of the two conditions corresponds to when the previously stored data in the database 420 needs to be updated to modify the relevance time span of the respective at least one smart device.

[0048] According to an embodiment, in response to the first user input, the dynamic relevance time predictor module 516 determines one or more contextual information associated with the user, the multi-device environment, and the identified at least one smart device. Accordingly, the dynamic relevance time predictor module 516 determines the one or more contextual information by retrieving the information from the context provider 522 associated with the user, the multi-device environment, and the at least one smart device. The dynamic relevance time predictor module 516 retrieves the contextual information from the user’s context 524. Further, the dynamic relevance time predictor module 516 retrieves the contextual information from the context of the environment in the multi-device environment 526. Additionally, the dynamic relevance time predictor module 516 retrieves the contextual information from the operational context of the identified at least one smart device 528. The context provider 522 includes the user’s context 524, the context of the environment in the multi-device environment 526, and the operational context of the identified at least one smart device 528. The context provider 522 can involve a database for storing the corresponding one or more contextual information. In a non-limiting example, the user’s context 524 includes the historical user interaction with the plurality of smart devices in the multi-device environment. Further, the dynamic relevance time predictor module 516 determines the one or more contextual information by retrieving the information from the user’s context 524.

[0049] According to an embodiment, the dynamic related time predictor module 516 assigns a dynamic weight to each of the user’s context 524, the context of the environment in the multi-device environment 526, and the operational context of the identified at least one smart device 528. In a non-limiting example, if the user has not used any user input throughout the day, then the operational context of the identified at least one smart device 528 can become dominant based on the assigned dynamic weight. In another non-limiting example, if the user has recently provided a user input to the TV, then the user’s context 524 and the operational context of the identified at least one smart device 528 become dominant.

[0050] In a non-limiting example, the user’s context 524 relates to information about the user’s historical activity on the at least one smart device through the first user input. An example of the user’s context 524 is shown in Table 1 below. As shown in Table 1, the user’s context 524 includes a historical user context, such as a first user input from the user. In addition, the user’s context 524 includes an executing device identification (ID), a user’s voice intent, and a related time span (old). In a non-limiting example, as shown in row 1 of Table 1, the historical user context corresponds to “turn on living room TV”. The dynamic related time predictor module 516 identifies the device ID of “living room TV”. Thereafter, based on the historical user context, the dynamic related time predictor module 516 discerns the user’s intent, i.e., to turn on the device. Accordingly, based on the user’s intent, the related time span (old) is predicted earlier.

[0051] [Table 1]

[0052]

[0053] In another non-limiting example, the context of the environment in the multi-device environment 526 relates to the environment of the plurality of smart devices. An example of the context of the environment in the multi-device environment 526 is shown in Table 2 below. As shown in Table 2, the context of the environment in the multi-device environment 526 corresponds to a time of 7 PM, a day of Saturday, a location of living room, and a season of summer. In addition, the dynamic related time predictor module 516 identifies a corresponding label encoding of the environment context. As an example, the label encoding corresponds to an encoded label of a voice intent, a device location, a device ID, and an environmental context, such as a time of day, a day, and the like. The label encoding is provided as an input into the AI model.

[0054] [Table 2]

[0055]

[0056] In yet another non-limiting example, the operation context 528 of the identified at least one smart device relates to an operation context of the at least one smart device. Examples of the operation context 528 of the identified at least one smart device are shown in Table 3 below. As shown in Table 3, the operation context of the TV in the living room relates to an “on” state and a “family” channel. Further, the state of the AC is “on” for the “cool” mode.

[0057] [Table 3]

[0058]

[0059] According to an embodiment, the dynamic relevant time predictor module 516 further predicts the relevant time span using a prediction model based on the determined one or more context information. The relevant time span is predicted for which the context of the first user input needs to be preserved for the identified at least one smart device.

[0060] According to an embodiment, the dynamic relevant time predictor module 516 predicts the relevant time span from the information retrieved from the context provider 522 and thereby stores the relevant time span in the database 420.

[0061] According to an embodiment, the prediction model can correspond to an AI model for predicting the relevant time span based on the first user input. The AI model is trained to predict the relevant time span of the identified at least one smart device based on the determined one or more context information. Figure 6 Fig. 6 illustrates an AI model based prediction of the relevant time span for the identified at least one smart device based on the determined one or more context information, according to an embodiment of the present disclosure. Figure 5the AI model receives inputs from the user's context 524, the context of the environment in the multi-device environment 526, and the operational context of the identified at least one smart device 528. Based on the received inputs, the AI model dynamically determines the relevant time span 602 by utilizing at least two hidden layers, such as layer 1 and layer 2. An example of the relevant time span 602 is shown in Table 4 below. As shown in Table 4, the relevant time span 602 for the living room TV is set to 7 minutes. Thus, for any subsequent ambiguous user input related to the TV within 7 minutes, the multi-device system 400 resolves the ambiguity by performing the action on the living room TV. Further, the relevant time span 602 changes based on the sequence of remaining time periods. Further, if any ambiguity occurs between the entries present in the relevant time span 602, the entry present in row 1 gets the highest priority. For example, the relevant time span 602 for the living room TV is 7 minutes and the relevant time span 602 for the bedroom TV is 3 minutes. In this scenario, the multi-device system 400 considers the ambiguous user input related to "increase TV volume" as "increase living room TV volume" because the relevant time span 602 for the living room TV is greater than the bedroom TV. The relevant time span 602 can be stored in the database 420 for subsequent use by the ambiguity resolver module 512.

[0062] [Table 4]

[0063]

[0064] In a non-limiting example, the dynamic relevant time predictor module 516 predicts the relevant time span 602 for the curtains 410 based on the context of the environment in the multi-device environment 526. If the context of the environment in the multi-device environment 526 involves cloudy weather, the relevant time span 602 for the curtains 410 can be set to 25 minutes. Alternatively, if the context of the environment in the multi-device environment 526 involves sunny weather, the relevant time span 602 for the curtains 410 can be set to 15 minutes. The relevant time span 602 is longer for cloudy weather because the user can provide the second user input over a longer period of time than sunny weather.

[0065] According to an embodiment, the prediction model corresponds to a rule-based model for predicting a relevant time span based on the first user input. In the rule-based model, a priority is assigned to the at least one smart device that is last used to perform an action corresponding to the user input. For example, the action performer module 514 controls the TV volume of the living room TV according to the user input. Thus, for the living room TV, the relevant time span 602 is set to 10 minutes (10 minutes is considered as a default time). Subsequently, if the user input involves "turning on the bedroom TV". Thus, for the bedroom TV relevant time span 602 is set to 5 minutes (less than the default time set for the earlier instance). Further, the bedroom TV is set to the highest priority. Thus, the action performer module 514 considers the bedroom TV for any ambiguous user input related to TV for the next 5 minutes. In Figure 11 An exemplary scenario for the rule-based model for predicting the relevant time span is shown in

[0066] According to an embodiment, the dynamic relevant time predictor module 516 combines the predicted relevant time span with the identified at least one smart device for performing the first action corresponding to the first user input. Thus, the relevant time span is stored in the database 420 for the identified at least smart device for performing the first action.

[0067] According to an embodiment, the ambiguity resolver module 512 determines whether a second user input is subsequently received after the first user input within the predicted relevant time span. If the second user input is ambiguous and is received within the predicted relevant time span, the action performer module 514 controls the identified at least one smart device to perform the second action. In a non-limiting example, the relevant time span 602 for the living room TV is set to 10 minutes. If a second user input is received within 10 minutes and the second user input is ambiguous, the ambiguity resolver module 512 determines "living room TV" for performing the second action.

[0068] Figure 7 A flow diagram of a method 700 for time-based personalized management in a multi-device environment according to an embodiment of the present disclosure is shown. As Figure 7 shown, the method 700 includes a series of steps 702-708 for time-based personalized management. The details of the method 700 have been explained in the paragraphs below. The order in which the method steps are described below is not intended to be construed as a limitation, and any number of the described method steps can be combined in any appropriate order to perform the method or an alternative method. Additionally, individual steps can be deleted from the method without departing from the scope of the present disclosure. The method steps 700 begin with a start block and in step 702 the execution of the operations begins as Figure 7 shown.

[0069] At step 702, the method 700 includes identifying, based on the first user input, at least one smart device of the plurality of smart devices in the multi-device environment for performing a first action corresponding to the first user input. The ambiguity resolver module 512 identifies the at least one smart device for performing the first action. The first user input can involve an ambiguous user input. Accordingly, the ambiguity resolver module 512 identifies the at least one smart device by resolving the ambiguity based on the cues or from the database 420. The flow of the method 700 now proceeds to step 704.

[0070] At step 704, the method 700 determines one or more contextual information associated with a user corresponding to the first user input, the multi-device environment, and the identified at least one smart device in response to the first user input. The dynamic relevance time predictor module 516 determines the one or more contextual information. The contextual information is determined to predict the relevant time span 602. The flow of the method 700 now proceeds to step 706.

[0071] At step 706, the method 700 includes predicting, using a prediction model, a relevant time span of the identified at least one smart device for which a context of the first user input needs to be preserved. The dynamic relevance time predictor module 516 predicts the relevant time span of the identified at least one smart device. The flow of the method 700 now proceeds to step 708.

[0072] At step 708, the method 700 includes combining the predicted relevant time span with the identified at least one smart device for performing the first action corresponding to the first user input. In particular, the dynamic relevance time predictor module 516 combines the predicted relevant time span with the identified at least one smart device. Accordingly, the relevant time span is stored in the database 420 for the identified at least smart device for performing the first action.

[0073] It should be noted that the method steps 702 to 708 and other operations disclosed herein are performed by the processor 412 of the user device 402.

[0074] Although the above steps in Figure 7 are shown and described in a particular order, these steps can occur in varying orders according to various embodiments. Moreover, detailed descriptions related to various steps of Figure 7 have been covered in the description related to Figures 4-6 and are omitted here for brevity.

[0075] Figure 8 An example scenario depicting time-based personalization in a multi-device environment is shown in accordance with an embodiment of the present disclosure.

[0076] According to an example embodiment, the sequence of exemplary steps 800 is depicted in a line diagram. Figure 8 As shown, user 102 provides user input to the virtual assistant 506 of user device 402 to control a desired device in a multi-device environment. The premise of the exemplary scenario corresponds to a multi-device environment including two TVs, where the first TV is installed in the bedroom and the second TV is installed in the living room. In step 806, the user provides the virtual assistant 506 (e.g., Bixby) with the first user input to turn on the TV. Because the relevant time span 602 is not present in database 420 and two identical devices are installed in the home, the ambiguity resolver module 512 cannot identify the desired TV to turn on. Therefore, to overcome the ambiguity, in step 808, user device 402 provides a prompt to resolve the query, i.e., which TV do you want to turn on? In step 810, the user responds to the prompt to resolve the query to turn on the living room TV. In response, in step 812, the action actuator module 514 facilitates the multi-device environment to turn on the living room TV and provides feedback to the user. Additionally, the dynamic relevant time predictor module 516 identifies the relevant time span 602 of the identified at least one smart device (i.e., the living room TV). In step 814, user 102 provides a second user input within the relevant time span 602 to increase the TV volume. Thus, in step 816, action actuator module 514 facilitates increasing the TV volume on the living room TV without prompting the user to resolve ambiguity. Similarly, in step 818, user 102 provides a third user input within the relevant time span 602 to change the channel to HBO. Therefore, in step 820, action actuator module 514 facilitates playing the HBO channel on the living room TV without prompting the user to resolve ambiguity. Therefore, this disclosure improves the user experience and reduces the time required to facilitate an appropriate action corresponding to user input.

[0077] Figure 9 Another example scenario depicting time-based personalization in a multi-device environment according to embodiments of the present disclosure is shown.

[0078] Another exemplary scenario corresponds to a multi-device environment including two TVs and two ACs, where the first TV and the first AC are installed in the main room. Furthermore, the second TV and the second AC are installed in the living room. For example... Figure 9As shown, in step 902, the user provides a first user input to open AC to the virtual assistant 506 (i.e., Bixby). The relevant time span 602 is not available in the database 420, and thus, the ambiguity resolver module 512 cannot overcome the ambiguity of two identical devices. Accordingly, in step 904, the virtual assistant 506 provides a prompt resolution query, i.e., "Which AC do you want to open?" In step 906, the user responds to the prompt resolution query to open the living room AC. Subsequently, the dynamic relevant time predictor module 516 predicts the relevant time span 602 for one or more contextual information and saves the relevant time span 602 in the database 420. In response, in step 908, the action executor module 514 facilitates the multi-device environment to open the living room AC and provides feedback to the user. Subsequently, in step 910, the user provides a second user input to open the TV within the predicted relevant time span 602. In step 912, the action executor module 514 facilitates the multi-device environment to open the living room TV without prompting the prompt resolution query. Accordingly, the present disclosure reduces further steps to save time and energy of the user device 402.

[0079] Figure 10 Another example scenario depicting time-based personalization in a multi-device environment is shown in accordance with an embodiment of the present disclosure.

[0080] The preconditions of the further example scenario correspond to the multi-device environment comprising two TVs, wherein the first TV is installed in the bedroom and the second TV is installed in the living room. Steps 1002 to 1016 are similar to steps 806 to 820. Accordingly, for brevity, the explanation of steps 1002 to 1016 is omitted herein with respect to the explanation of steps 806 to 820. Further, in step 1018, the user provides a fourth user input to open the bedroom TV to the virtual assistant 506 (i.e., Bixby). The fourth user input is an explicit user input. Accordingly, in step 1020, the action executor module 514 facilitates the multi-device environment to open the bedroom TV and thereby provides feedback to the user 102. Additionally, the dynamic relevant time predictor module 516 predicts the relevant time span 602 for the bedroom TV based on one or more contextual information. Further, in step 1022, the user provides a fifth user input to increase the TV volume within the relevant time span 602. Further, in step 1024, the action executor module 514 facilitates to increase the TV volume of the bedroom TV based on the relevant time span 602 and confirms to the user that the TV volume of the bedroom TV is increased. Accordingly, the present disclosure enhances the user experience by dynamically determining the relevant time span 602 based on the latest user input.

[0081] Figure 11 An example scenario depicting time-based personalization based on a rule-based model is shown in accordance with an embodiment of the present disclosure.

[0082] A precondition of yet another example scenario 1100 corresponds to a multi-device environment including two TVs, where a first TV is installed in a bedroom and a second TV is installed in a living room. Steps 1102 through 1112 are similar to steps 806 through 816. Thus, for brevity, explanations of steps 1102 through 1112 are omitted herein with respect to the explanations of steps 806 through 816. However, the rule-based model sets a priority to the living room TV during the operations of steps 1106 through 1112, because the living room TV is last used to perform an operation corresponding to a user input. Further, based on the rules defined in the rule database 1122, the relevant time span 602 is set to 10 minutes for the living room TV. As such, at step 1110, when the user provides a third user input for increasing a TV volume within the relevant time span 602 of 10 minutes, the action executor module 514 facilitates increasing the TV volume of the living room TV at step 1112 without prompting the user to resolve ambiguity. Further, at step 1114, the user provides a fourth user input to the virtual assistant 506 (i.e., Bixby) to turn on the bedroom TV. The fourth user input is an unambiguous user input. Thus, at step 1116, the action executor module 514 facilitates the multi-device environment to turn on the bedroom TV, and thereby provides feedback to the user 102. Further, the rule-based model sets the highest priority to the bedroom TV that is last used to perform an action corresponding to the fourth user input. Further, based on the rules defined in the rule database 1122, the relevant time span 602 is set to 5 minutes for the bedroom TV. Further, at step 1118, the user provides an ambiguous fifth user input to increase the TV volume within the relevant time span 602 of 5 minutes from the fourth user input, with the bedroom TV having the highest priority. Further, at step 1120, the action executor module 514 facilitates increasing the TV volume of the bedroom TV based on the priority, the relevant time span 602, and thereby confirms to the user that the TV volume of the bedroom TV is increased. Thus, the present disclosure enhances user experience by dynamically determining the relevant time span 602 according to the rule-based model.

[0083] Now referring to the technical capabilities and effectiveness of the method 700 and the multi-device system 400 as disclosed herein. The following technical advantages are provided over conventional and existing solutions. The method 700 as disclosed above helps improve the user experience by eliminating the prompt resolution query if any subsequent ambiguous user input is provided within the relevant time span 602. Additionally, the relevant time span 602 is dynamically predicted based on the personalized context (i.e., one or more contextual information). The method 700 determines the priority of at least one smart device among the plurality of smart devices based on the capabilities and the wait / listen time in order to obtain the most relevant smart device in case of disambiguation. Further, the method 700 reduces the overall execution time by storing the relevant time span 602 to eliminate the disambiguation scenario for the device. Moreover, the present disclosure saves the energy of the user device 402 as the number of prompt resolution queries is reduced.

[0084] Although the present disclosure has been described using specific language, it is intended that no limitation, with regard to the scope of the inventive subject matter, is created in this respect. It is apparent that those having ordinary skill in the art can devise many alterations, modifications, additions, or subtractions to the method described without departing from the spirit and scope of the inventive subject matter as taught by the present disclosure.

[0085] The accompanying drawings and preceding description give examples of embodiments. Those skilled in the art will understand that one or more of the described elements can well be combined in a single function element. Alternatively, certain elements can be separated into multiple function elements. Elements from one embodiment can be added to another embodiment. For example, the order of process described herein can be changed, and not all of the described acts can be performed. The scope of embodiments is accordingly not limited to what is described in this document.

[0086] Moreover, the acts of any of the processes can not be performed in the order indicated; nor can all of the acts be performed; and those acts that are performed can be performed in a different order. Additionally, those acts can be performed concurrently, and / or included in other acts or structures as is useful to one of ordinary skill in the art. Nothing is to be construed as requiring a departure from the scope of the embodiments. Moreover, the scope of the embodiments is not to be limited to the specific examples contained herein. Many variations, modifications, additions, and / or omissions can be made to the embodiments described herein without departing from the scope of the embodiments as recited in the claims.

[0087] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems and any component(s) that can cause any one of the benefits, advantages, or solutions to occur or become more pronounced are not to be construed as a critical, required, or essential feature or component of any or all the claims.

Claims

1. A method for time-based personalization management in a multi-device environment, the method comprising: identifying, based on a first user input, at least one smart device of a plurality of smart devices in the multi-device environment for performing a first action corresponding to the first user input; determining, in response to the first user input, one or more contextual information associated with a user corresponding to the first user input, the multi-device environment, and the identified at least one smart device; predicting, using a prediction model, a relevant time span for which a context of the first user input needs to be preserved for the identified at least one smart device based on the determined one or more contextual information; and combining the predicted relevant time span with the identified at least one smart device for performing the first action corresponding to the first user input.

2. The method of claim 1, further comprising: determining whether a second user input is subsequently received after the first user input within the predicted relevant time span; and controlling the identified at least one smart device to perform a second action based on a determination that the second user input is subsequently received after the first user input within the predicted relevant time span. identifying the at least one smart device of the plurality of smart devices comprises: determining whether the first user input is an ambiguous user input for performing the first action by the at least one smart device; and 3. The method of claim 1, wherein, identifying the at least one smart device in the multi-device environment based on a determination that the first user input is the ambiguous user input. determining the one or more contextual information comprises: determining a context of the user, a context of an environment in the multi-device environment, and an operational context of the identified at least one smart device, wherein the context of the user is determined based on historical user interactions with the plurality of smart devices in the multi-device environment; and 4. The method of claim 1, wherein, determining the one or more contextual information based on the context of the user, the context of the environment in the multi-device environment, and the operational context of the identified at least one smart device.

5. The method of claim 4, further comprising assigning a dynamic weight to each of the determined context of the user, the context of the multi-device environment, and the operational context of the identified at least one smart device. the prediction model corresponds to a rule-based model for predicting the relevant time span based on the first user input. the prediction model corresponds to an artificial intelligence (AI) model for predicting the relevant time based on the first user input, 6. The method of claim 1, wherein, wherein the AI model is trained to predict the relevant time span for the identified at least one smart device based on the determined one or more contextual information.

7. The method of claim 1, wherein, 8. The method of claim 1, wherein: the multi-device environment corresponds to one of a smart home environment or an Internet of Things (IoT) environment, and the at least one smart device has a same functionality with respect to a group of smart devices of the plurality of smart devices. the first user input corresponds to any one of a voice input, a text input, a graphical user interface (GUI) input, a remote control input, and a gesture input of the user; and wherein the second user input corresponds to any one of the voice input, the text input, and the gesture input of the user that causes disambiguation.

9. The method of claim 1, wherein, ​ ​ ​ 10. A multi-device system for time-based personalization management in a multi-device environment, the multi-device system comprising: a plurality of smart devices configured to communicate with each other in the multi-device environment; and a user device comprising at least one processor and configured with a virtual assistant, the user device communicatively coupled with each of the plurality of smart devices via the virtual assistant, and the at least one processor configured to: identify, based on a first user input, at least one smart device of the plurality of smart devices in the multi-device environment for performing a first action corresponding to the first user input; determine, in response to the first user input, one or more contextual information associated with a user corresponding to the user input, the multi-device environment, and the identified at least one smart device; predict, using a prediction model, a relevant time span for which a context of the identified at least one smart device needs to be preserved based on the determined one or more contextual information of the first user input; and combine the predicted relevant time span with the identified at least one smart device for performing the first action corresponding to the first user input. the at least one processor is configured to:

11. The multi-device system of claim 10, wherein, determine whether a second user input is subsequently received after the first user input within the predicted relevant time span; and based on a determination that the second user input is subsequently received after the first user input within the predicted relevant time span, control the identified at least one smart device to perform a second action. to identify the at least one smart device of the plurality of smart devices, the at least one processor is configured to: determine whether the first user input is an ambiguous user input for performing the first action by the at least one smart device; and 12. The multi-device system of claim 10, wherein, based on a determination that the first user input is the ambiguous user input, identify the at least one smart device in the multi-device environment. to determine the one or more contextual information, the at least one processor is configured to: determine a context of the user, a context of an environment in the multi-device environment, and an operational context of the identified at least one smart device, wherein the context of the user is determined based on historical user interactions with the plurality of smart devices in the multi-device environment; and 13. The multi-device system of claim 10, wherein, based on the context of the user, the context of the environment in the multi-device environment, and the operational context of the identified at least one smart device, determine the one or more contextual information. the at least one processor is further configured to assign a dynamic weight to each of the determined context of the user, the context of the multi-device environment, and the operational context of the identified at least one smart device. ​ 14. The multi-device system of claim 13, wherein, ​

Images