User terminal device provision system, method and computer readable medium with usage habit analysis

Abstract

The invention discloses a user terminal device provision system, method and computer readable medium with usage habit analysis. A usage habit analysis module of a user terminal software provision subsystem uses a deep reinforcement learning method to train a usage habit analysis model based on usage data of a user terminal device and traffic data of a operation support system, and the usage habit analysis module uses the usage habit analysis model trained by the deep reinforcement learning method to predict software change time of the user terminal device, and a software provision module performs software changes or software installation on the user terminal device when the user terminal device is idle based on the software change time predicted by the usage habit analysis model.

Description

[Technical Field]

[0001] This invention relates to a user terminal equipment supply technology, and more particularly to a user terminal equipment supply system, method, and computer-readable medium with usage habit analysis. [Previous Technology]

[0002] With the development of the Internet, daily life is now closely related to smart homes. This makes the integration of a good smart home system and user experience one of the important factors for users to choose. Therefore, how to accurately analyze user behavior and avoid interfering with users' use of services has also become a very important issue.

[0003] The prior art proposes a network quality testing method. However, when measuring the quality of network services, this prior art usually ignores the user's device usage, which leads to a negative impact on the user's experience.

[0004] Furthermore, this prior art does not utilize deep reinforcement learning methods to train a usage habit analysis model based on user terminal device usage data and operation support system service data. It also cannot predict the software change time of user terminal device through the usage habit analysis model, nor can it perform software changes or software installation when the user terminal device is idle. As a result, it is difficult to avoid interruption due to the operation of the user terminal device when the user is using this user terminal device, and it is even more difficult to avoid the user experience being reduced due to the operation of the user terminal device.

[0005] Therefore, how to provide an innovative user terminal equipment installation technology to solve any of the above problems, or to provide related systems and methods, has become a major research topic for those skilled in the art. [Summary of the Invention]

[0006] The user terminal device installation system with usage habit analysis according to the present invention includes: at least one user terminal device; and a user terminal software installation subsystem, which is communicatively connected to the user terminal device. The user terminal software installation subsystem has a usage habit analysis module and a software installation module. The usage habit analysis module has a usage habit analysis model. The usage habit analysis module uses a deep reinforcement learning method to train the usage habit analysis model based on the usage data of the user terminal device and the communication data of the external operation support system. The usage habit analysis model trained by the usage habit analysis module through the deep reinforcement learning method predicts the software changeable time of the user terminal device. The software installation module performs software modification or software installation of the user terminal device when the user terminal device is idle, based on the software changeable time predicted by the usage habit analysis model.

[0007] The user terminal device installation method with usage habit analysis according to the present invention includes: providing a user terminal device installation system including at least one user terminal device and a user terminal software installation subsystem, wherein the user terminal software installation subsystem has a usage habit analysis module and a software installation module; and the usage habit analysis module uses a deep reinforcement learning method to train a usage habit analysis model based on the usage data of the user terminal device and the communication data of an operations support system, and then the usage habit analysis model trained by the usage habit analysis module through the deep reinforcement learning method predicts the software changeable time of the user terminal device, so that the software installation module performs software change or software installation of the user terminal device when the user terminal device is idle, based on the software changeable time predicted by the usage habit analysis model.

[0008] The computer-readable medium of the present invention is applied in a computing device or computer, and stores instructions. The computer-readable medium is executed through a processor and memory so as to execute the above-mentioned user terminal device installation method with usage habit analysis when the computer-readable medium is executed.

[0009] Therefore, the present invention provides an innovative user terminal device installation system, method, and computer-readable medium with usage habit analysis. The usage habit analysis module uses deep reinforcement learning to effectively train the usage habit analysis model based on the user terminal device's usage data (such as processor / memory usage, network traffic, software usage records, etc.) and the operation support system's communication data (such as internet access data / internet behavior data / internet traffic data related to the user or user device, etc.). The usage habit analysis model can also quickly predict the software change time of the user terminal device and perform software changes or software installation when the user terminal device is idle.

[0010] Furthermore, the usage habit analysis model of the usage habit analysis module of the present invention can reduce the impact on the user's use of the user terminal device when the device operation is performed. In addition to ensuring the normal operation of the user terminal device, it also avoids interruption of the user's use of the user terminal device due to the device operation, and also avoids the user's experience being reduced due to the device operation.

[0011] To make the above-described features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings. Additional features and advantages of the invention will be set forth in part in the following description, and such features and advantages will be partly apparent from the description or may be learned by practice of the invention. It should be understood that both the foregoing general description and the following detailed description are illustrative and explanatory, and are not intended to limit the scope of the invention. [Simplified Explanation of the Diagram]

[0081] Figure 1 is a schematic diagram of the architecture of the user terminal equipment supply system with usage habit analysis according to the present invention.

[0082] Figure 2 is a block diagram of the usage scenarios in the user terminal equipment supply system with usage habit analysis described in this invention.

[0083] Figure 3 is a schematic diagram of the process of equipment management in the user terminal equipment supply and installation method with usage habit analysis according to the present invention.

[0084] Figure 4 is a schematic diagram of the user subscription process in the user terminal device supply method with usage habit analysis described in this invention.

[0085] Figure 5 is a schematic diagram of the software installation process in the user terminal device installation method with usage habit analysis described in this invention.

[0086] Figure 6 is a flowchart illustrating the software change scheduling process in the user terminal device installation method with usage habit analysis described in this invention.

[0087] Figure 7 is a schematic diagram of the training process of the user terminal equipment supply method with user habit analysis according to the present invention.

Implementation Method

[0012] The embodiments of the present invention are described below by way of specific specific embodiments. Those skilled in the art can understand other advantages and effects of the present invention from the content disclosed in this specification, and can therefore implement or use it by other different specific equivalent embodiments.

[0013] Figure 1 is a schematic diagram of the architecture of the user terminal device supply system 1 with usage habit analysis according to the present invention, and Figure 2 is a block diagram of the "usage scenario" in the user terminal device supply system 1 with usage habit analysis according to the present invention.

[0014] As shown in Figures 1 and 2, the user terminal device installation system 1, which includes usage habit analysis, can communicatively connect at least one (or multiple) user A's user device A1 with an external operation support system B, and includes a user terminal software installation subsystem 10, at least one (or multiple) user terminal devices 20, and a remote device management server 30, etc., which are interconnected. In one embodiment, the user terminal device 20 can be communicatively connected to the user terminal software installation subsystem 10 and the remote device management server 30, etc.

[0015] In one embodiment, the user terminal software installation subsystem 10 may have a core module 11, a remote management module 12, a software installation module 13, an application software library 14, and a usage habit analysis module 15, etc. The core module 11 may have a scheduling unit 11a and a database 11b, etc., and the usage habit analysis module 15 may have a usage habit analysis model 16, etc. In addition, the user terminal device 20 may have multiple hardware 21 (such as processors / memory, etc.) and an operating system 22, and may also have various software such as a core software 23, a remote management agent software 24, an isolation software 25, a status monitoring software 26, and at least one (such as multiple) value-added software 27, etc., and the status monitoring software 26 may be built into the isolation software 25 or linked to the isolation software 25.

[0016] In one embodiment, the user terminal software installation subsystem 10 may be a user terminal software installation platform, etc. The core module 11 may be a terminal core, etc., the scheduling unit 11a may be a scheduling component (chip / circuit), scheduling software (program), etc., and the database 11b may be a data storage device, hard disk, memory, memory card, etc. The remote management module 12 may be a remote management component (chip / circuit), remote management software (program), etc., the software installation module 13 may be a software installation component (chip / circuit), software installation software (program), etc., and the application software library 14 may be a database, hard disk, memory, memory card, server, etc., used to store software (such as value-added software 27).

[0017] In one embodiment, the user terminal device 20 may be a modem 20a, a wireless router 20b (or wireless router 20c), a network camera 20d, a smart speaker, a TV set-top box, etc., but does not include smartphones, tablets, personal computers, laptops, and other user devices A1. The remote management agent software 24 may be a remote management agent program, the isolation software 25 may be a software isolation engine, etc., and the status monitoring software 26 may be a status monitoring program, etc. The value-added software 27 may be a value-added service, value-added application software, etc., such as audio-visual software, game software, remote work software, etc.

[0018] In one embodiment, user A may be a user, customer, member, etc., user device A1 may be a smartphone, tablet computer, personal computer, laptop computer, etc., and service provider C may be a service provider, network service provider, etc.

[0019] In this invention, "at least one" means one or more (e.g., one, two, or three or more), "plural" or "multiple" means two or more (e.g., two, three, four, or ten or more), and "communication connection" means mutual communication or connection through various forms such as data, signal, electrical, wired (e.g., wired network), or wireless (e.g., wireless network). However, this invention is not limited to those mentioned in the various embodiments.

[0020] In the embodiment of FIG1, the user terminal software supply subsystem 10 may have [1] a core module 11, [2] a remote management module 12, [3] a software supply module 13, [4] an application software library 14, [5] a usage habit analysis module 15 and a usage habit analysis model 16.

[0021] [1] The scheduling unit 11a (e.g., scheduler) of the core module 11 can schedule the maintenance tasks of the user terminal equipment 20, and the database 11b of the core module 11 can store the management data of the user terminal equipment 20 and the subscription information of software (e.g., value-added software 27). [2] The remote management module 12 can handle the periodic reporting and maintenance instructions of the user terminal equipment 20, and can also perform remote settings, troubleshooting, restart and / or firmware updates of the user terminal equipment 20. [3] The software installation module 13 can be responsible for the software (e.g., value-added software 27) installation task of the user terminal equipment 20, and can also initiate software change commands. The software installation module 13 can perform software change or software installation of the user terminal equipment 20 when the user terminal equipment 20 is idle, based on the software changeable time (e.g., software installation time) predicted by the user habit analysis model 16. [4] The application software library 14 can store various software (such as value-added software 27). [5] The usage habit analysis model 16 of the usage habit analysis module 15 analyzes the software usage habits of the user terminal device 20. The usage habit analysis module 15 can use deep reinforcement learning methods to train the usage habit analysis model 16 based on the usage data of the user terminal device 20 (such as processor / memory usage rate, network traffic, software usage records, etc.) and the traffic data of the external operation support system B (such as the internet access data / internet behavior data / internet traffic data of user A or user device A1, etc.), so that the usage habit analysis model 16 can provide a prediction of the software availability time of the user terminal device 20 (such as the software availability time). That is, the usage habit analysis module 15 can predict the software changeable time of the user terminal device 20 through the usage habit analysis model 16 trained by deep reinforcement learning method, so that the software installation module 13 can perform software change or software installation of the user terminal device 20 when the user terminal device 20 is idle, based on the software changeable time predicted by the usage habit analysis model 16.

[0022] Furthermore, the user terminal device 20 may be composed of hardware 21 (such as processor / memory), operating system 22, and multiple software including core software 23 from bottom to top, and the multiple software may include [1] core software 23, [2] remote management agent software 24, [3] isolation software 25 and status monitoring software 26, [4] value-added software 27.

[0023] [1] Core software 23 can be responsible for providing various core functions, such as system resource allocation function, network function and device fault alarm function. [2] Remote management agent software 24 can periodically report to user terminal software installation subsystem 10 and receive software installation commands from user terminal software installation subsystem 10. That is, remote management agent software 24 can receive instructions from user terminal software installation subsystem 10 to perform remote management and software installation of user terminal device 20, and can also assist in software changes and other device operations of user terminal device 20. [3] Isolation software 25 (such as software isolation engine) can be built into or linked to status monitoring software 26 so that status monitoring software 26 can periodically report device information of user terminal device 20. [4] Value-added software 27 can be the value-added services or value-added application software managed by user terminal software installation subsystem 10.

[0024] In the embodiment of Figure 2, the remote device management server 30 can manage user terminal devices 20 (such as modem 20a / wireless router 20b / wireless router 20c / network camera 20d, etc.) used by at least one (e.g., two) user A or user device A1 in user environment D. The remote device management server 30 can also perform actions such as monitoring the device status of user terminal devices 20, querying network topology, restarting devices, and / or activating software (such as value-added software 27 or value-added services). In addition, the types of user terminal devices 20 may include modem 20a, wireless router 20b (wireless router 20c), network camera 20d, as well as smart speakers, TV set-top boxes, etc. (not shown).

[0025] For example, in today’s booming development of smart homes, service providers C (such as service providers / network service providers) listen to market demands and hope that various user terminal devices 20, such as modems 20a or wireless routers 20b (wireless routers 20c), can provide more diverse smart home services in addition to providing the original network functions.

[0026] Therefore, the user terminal device supply system 1 with usage habit analysis described in this invention is applied to the field of smart homes, etc. It can also automatically install, update and / or remove software (such as value-added software 27) from the user terminal device 20 by the user terminal software supply subsystem 10, thereby improving the management efficiency of the user terminal device 20 and ensuring the security and reliability of the user terminal device 20.

[0027] This invention combines user A's usage habits with software (such as value-added software 27) to facilitate personalized and precise installation of user terminal devices 20 used by user A. It also enables the segmentation and advanced behavior analysis of user terminal devices 20 in a de-identified environment. In one embodiment, this invention can assess the degradation cycle of user terminal devices 20 based on heavy, moderate, and light usage rates, or recommend software (such as value-added software 27) based on applications such as video / audio, games, or remote work. Alternatively, it can utilize the idle state of user terminal devices 20 (such as non-usage periods) to perform maintenance on user terminal devices 20 to achieve energy-saving purposes (energy-saving effect).

[0028] Figure 3 is a schematic diagram of the process of "device management" in the user terminal equipment supply method with usage habit analysis according to the present invention, and includes the following steps S11 to S17, and is described together with reference to the user terminal equipment supply system 1 with usage habit analysis shown in Figures 1 and 2.

[0029] In the process of "device management", the present invention performs remote device management by periodically accessing the remote device management server 30 through the user terminal device 20. The user terminal software installation subsystem 10 determines whether to manage the device or not by strictly verifying the legitimacy of the user terminal device 20 and the user A (such as the tenant). In addition to providing the user terminal device 20 with daily maintenance functions, it can also install the software subscribed by the user A (such as value-added software 27) during the management period of the user terminal device 20, which helps to reduce the manpower and time costs of the installation personnel of the service provider C (such as the service provider).

[0030] As shown in Figure 3, when the user terminal device 20 used by user A is installed for the first time, the user terminal device 20 can actively send a request message to the remote management module 12 of the user terminal software installation subsystem 10 to request the remote management module 12 to remotely manage the user terminal device 20 (see step S11).

[0031] Next, the user terminal software installation subsystem 10 can confirm whether the user terminal device 20 is provided by the service provider C (such as a service provider) through the request message transmitted by the user terminal device 20, so as to confirm the legitimacy of the user terminal device 20, that is, to determine whether the user terminal device 20 is legitimate (see step S12). If not (the user terminal device 20 is illegitimate), then proceed to step S13. Otherwise, if yes (the user terminal device 20 is legitimate), then proceed to step S14.

[0032] That is, if the user terminal software installation subsystem 10 determines that the user terminal device 20 is not provided by the service provider C (such as a service supplier) and therefore determines that the user terminal device 20 is illegitimate, then the remote management module 12 of the user terminal software installation subsystem 10 will not manage the user terminal device 20 (see step S13). Conversely, if the user terminal software installation subsystem 10 determines that the user terminal device 20 is provided by the service provider C (such as a service supplier) and therefore determines that the user terminal device 20 is legitimate, then the user terminal software installation subsystem 10 can further confirm whether the user A (such as a tenant) has rented the user terminal device 20 from the service provider C (such as a service supplier) to confirm the legitimacy of the user A, that is, to determine whether the user A is legitimate (see step S14). If not (user A is illegitimate), then proceed to step S13; otherwise, if yes (user A is legitimate), then proceed to step S15.

[0033] In other words, if the user terminal software installation subsystem 10 determines that user A has not leased user terminal equipment 20 from service provider C (such as a service provider) (such as user terminal equipment 20 not belonging to user A) and this is illegal, then the remote management module 12 of the user terminal software installation subsystem 10 will not manage user terminal equipment 20 (see step S13). Conversely, if the user terminal software installation subsystem 10 determines that user A has leased user terminal equipment 20 from service provider C (such as a service provider) and this is legal, then proceed to step S15.

[0034] After confirming the legitimacy of the user terminal device 20 and the user A, the user terminal software installation subsystem 10 immediately installs parameters on the user terminal device 20 (see step S15) to provide basic services or Internet access services to the user terminal device 20. Then, the user terminal software installation subsystem 10 initiates (e.g., immediately initiates) software changes on the user terminal device 20 (see step S16) to install software (e.g., value-added software 27 or value-added services) on the user terminal device 20, thereby completing the management of the user terminal device 20 (see step S17).

[0035] Figure 4 is a schematic diagram of the process of "user subscription" in the user terminal device supply method with usage habit analysis according to the present invention, and includes the following steps S21 to S26, and is described together with reference to the user terminal device supply system 1 with usage habit analysis shown in Figures 1 and 2.

[0036] In the "user subscription" process of this invention, user A can proactively initiate a software / hardware service request to the service provider C (such as a service provider). The operation support system B assists in recording and deciding whether to distribute a new user terminal device 20, allowing user A to install the user terminal device 20 through simple device installation, or dispatching installation personnel from the service provider C (such as a service provider) when needed. Combined with the automatic software supply process, the software installation task can be completed quickly, thereby improving user A's satisfaction.

[0037] As shown in Figure 4, when user A or its user device A1 subscribes to software (such as value-added software 27) from service provider C (such as a service provider) (see step S21), the external operation support system B sends a software installation request to the remote management module 12 of the user terminal software installation subsystem 10. The remote management module 12 checks whether the user terminal device 20 used by user A (such as a tenant) supports the installation of the software (such as value-added software 27) (see step S22). If not (user terminal device 20 does not support the installation of the software), then proceed to steps S23 and S24 in sequence; otherwise, if yes (user terminal device 20 supports the installation of the software), then proceed directly to step S24.

[0038] That is, if the user terminal device 20 does not support the installation of software (such as value-added software 27), the service provider C (such as a service provider) will provide a new user terminal device 20 for software installation (see step S23), and the user terminal software installation subsystem 10 will manage the new user terminal device 20 and then register (record) the device information of the new user terminal device 20 to the external operation support system B (see step S24). Conversely, if the user terminal device 20 supports the installation of software (such as value-added software 27), the device information of the user terminal device 20 will be directly registered (recorded) to the external operation support system B (see step S24).

[0039] If the user terminal device 20 has been managed by the remote management module 12 of the user terminal software installation subsystem 10, the software installation module 13 of the user terminal software installation subsystem 10 can initiate (e.g., immediately initiate) software changes to the user terminal device 20 (see step S25). Then, the software installation module 13 of the user terminal software installation subsystem 10 provides software (e.g., value-added software 27) to the user terminal device 20 through the application software library 14 to complete the subscription (see step S26).

[0040] Figure 5 is a flowchart of the "software installation" process in the user terminal device installation method with usage habit analysis according to the present invention, and includes the following steps S31 to S34, and is described together with reference to the user terminal device installation system 1 with usage habit analysis shown in Figures 1 and 2.

[0041] In the process of "software installation", the present invention combines the remote management module 12 and the software installation module 13 of the user terminal software installation subsystem 10 to provide automated software installation services and interfaces with the external operation support system B, so that the user terminal equipment 20 can perform automated software installation, which helps to reduce the need for dispatching installation personnel in the past.

[0042] As shown in Figure 5, when the remote management module 12 receives the software change message initiated by the user terminal software installation subsystem 10 (see step S31), the software installation module 13 of the user terminal software installation subsystem 10 can provide the software download path to the user terminal device 20 through the application software library 14 to issue the software change command (see step S32).

[0043] Then, the software supply module 13 can initiate a software change schedule to the user terminal device 20 based on the software changeable time of the user terminal device 20 analyzed by the usage habit analysis model 16 (see step S33). The user terminal device 20 then obtains software (such as value-added software 27 or value-added services) from the application software library 14 for updating, thereby ending the software change command (see step S34).

[0044] Figure 6 is a flowchart of the "software change scheduling" in the user terminal device installation method with usage habit analysis according to the present invention, and includes the following steps S41 to S46, and is described together with reference to the user terminal device installation system 1 with usage habit analysis shown in Figures 1 and 2.

[0045] In the process of "software change scheduling", the present invention combines the software change command issued by the user terminal software installation subsystem 10 with the software changeable time of the user terminal device 20 analyzed by the usage habit analysis model 16, and performs software installation or software installation on the user terminal device 20 when the user terminal device 20 is idle (such as no traffic or low traffic not used by the user).

[0046] As shown in Figure 6, when the user terminal device 20 receives a message / command for changing the device scheduling software (see step S41), the software installation module 13 will perform software changes on the user terminal device 20 during the software changeable time (such as the software changeable time period) provided by the user terminal software installation subsystem 10's usage habit analysis model 16.

[0047] When making software changes to user terminal device 20, user terminal device 20 may first check whether its dependent software is busy (see step S42). If not (dependent software is not busy), proceed to step S43, whereby the software installation module 13 performs software changes on user terminal device 20 and reports the results of the software changes (see step S43), thereby ending the scheduling task (see step S46). Conversely, if it is (dependent software is busy), then it seeks whether there is any remaining (other) software change time (see step S44).

[0048] If there is remaining software modification time, the scheduling task is terminated directly (see step S46). Conversely, if there is no remaining software modification time, the user terminal device 20 can request or obtain software modification time from the remote management module 12 (see step S45). The remote management module 12 then queries the usage habit analysis model 16 for software modification time and sends the information back to the user terminal device 20. After receiving the next software modification time predicted by the usage habit analysis model 16, the user terminal device 20 terminates the scheduling task (see step S46). That is, when the user terminal device 20 waits for the next software modification time (scheduling time) predicted by the usage habit analysis model 16, if the user terminal device 20 detects that its dependent software is in an idle state, the user terminal device 20 can perform software modification and report the results of the software modification, thereby terminating the scheduling task (see step S46).

[0049] Figure 7 is a flowchart of the "training of the usage habit analysis model 16" in the user terminal equipment supply method with usage habit analysis according to the present invention, and includes the following steps S51 to S54, and is described together with reference to the user terminal equipment supply system 1 with usage habit analysis shown in Figures 1 and 2.

[0050] In the process of "training the usage habit analysis model 16", the usage habit analysis module 15 can use the device status of the user terminal device 20, the traffic data of the operation support system B, and the replay memory of the usage habit analysis model 16 to perform deep reinforcement learning training (such as deep reinforcement learning model training). It also uses the real online data of the user terminal device 20 to continuously train the usage habit analysis model 16, so that the usage habit analysis model 16 approaches or matches the user A's actual usage habits of the user terminal device 20, so that the usage habit analysis model 16 can provide prediction results for the software installation schedule of the user terminal device 20.

[0051] As shown in Figure 7, the usage habit analysis module 15 can use deep reinforcement learning methods to train the usage habit analysis model 16 based on the usage data of the user terminal device 20 and the communication data of the operation support system B (see step S51). Next, after the remote management module 12 remotely manages the user terminal device 20 (see step S52), the usage habit analysis model 16 can use the data of the user terminal device 20 in the previous time interval to predict the usage of the user terminal device 20 in the current time interval. Then, the remote management module 12 feeds back the prediction results of the usage habit analysis model 16 based on the monitoring information of the device status (such as the real-time device status) of the user terminal device 20. Then, the usage habit analysis module 15 can combine the prediction results of the usage habit analysis model 16 with the replay memory of the usage habit analysis model 16 to make the loss function of the usage habit analysis model 16 closer to the actual result.

[0052] The usage habit analysis model 16 can provide a predicted operating time (equipment operation time) for the user terminal device 20, and can also monitor the device status of the user terminal device 20 (such as the real-time device status) to prevent user A from being affected by the operation of the user terminal device 20. In addition, the usage habit analysis module 15 can use the device status of the user terminal device 20 (such as the real-time device status), the traffic data of the operation support system B, and the replay memory of the usage habit analysis model 16 to perform deep reinforcement learning training (such as deep reinforcement learning model training) (see step S53). The training process of the usage habit analysis model 16 can be continuously (in real-time) trained with the real online data of the user terminal device 20 (see step S54), which is conducive to the analysis experience of the usage habit analysis module 15 continuously matching the user A's actual usage habits of the user terminal device 20, thereby enabling the device to operate in the idle state of the user terminal device 20 (such as no traffic or non-user use period).

[0053] For example, the following embodiments one to three are provided to illustrate the user terminal equipment supply system 1 and method with usage habit analysis described in this invention.

[0054] I. Taking the user terminal software installation subsystem 10 actively initiating software change as an example: User A has multiple user terminal devices 20 (such as modem 20a / wireless router 20b / wireless router 20c / network camera 20d) in user environment D from service provider C (such as service provider), and the user terminal software installation subsystem 10 wants to make a software change to a certain user terminal device 20 (such as wireless router 20b).

[0055] [1] The software installation module 13 of the user terminal software installation subsystem 10 initiates software change, obtains the software download path from the application software library 14, and obtains the software changeable time of the user terminal device 20 to be changed from the usage habit analysis model 16. Then, through the instructions of the remote management module 12, it initiates software change scheduling to the remote management agent software 24 in the user terminal device 20 (such as the wireless router 20b).

[0056] [2] After receiving the software change, the user terminal device 20 schedules the process. The user terminal device 20 can check whether its dependent software is busy (in a busy state or idle state) at the scheduling trigger time. If the dependent software is busy (in a busy state), it will be postponed to the next scheduling.

[0057] [3] The user terminal device 20 checks again whether its dependent software is busy (in a busy state or an idle state). If the user terminal device 20 finds that the dependent software is not busy (in an idle state), the user terminal device 20 can immediately perform differential updates through the software download path to complete the software change, and then report the change result of the software change.

[0058] 2. Taking the operation of the energy-saving software (such as value-added software 27) of the user terminal equipment 20 as an example: User A has multiple user terminal equipment 20 (such as modem 20a / wireless router 20b / wireless router 20c / network camera 20d) of service provider C (such as service provider) in user environment D, and the energy-saving software (such as value-added software 27) of the user terminal equipment 20 wants to schedule to reduce power consumption and achieve energy saving purpose (energy saving effect).

[0059] [1] Use the habit analysis model 16 to collect data and train the model on the multiple user terminal devices 20 that have been included in the management (such as modem 20a / wireless router 20b / wireless router 20c).

[0060] [2] Using the habit analysis model 16, it is determined that a user terminal device 20 (such as a wireless router 20b) is not used at night, while other user terminal devices 20 (such as a modem 20a / wireless router 20c / network camera 20d) have a fixed traffic at night. Using the habit analysis model 16 in conjunction with the user terminal software installation subsystem 10, it is determined that this fixed traffic is the traffic of a certain user terminal device 20 (such as network camera 20d). Therefore, energy saving can be carried out by referring to the minimum traffic (such as the minimum network traffic) of the user terminal device 20 (such as network camera 20d).

[0061] [3] The energy-saving software (such as value-added software 27) obtains from the usage habit analysis model 16 the time period when the user terminal device 20 (such as network camera 20d) wants to save energy and the minimum traffic (such as minimum network traffic) during this period, and monitors the device status (such as real-time device status) of the user terminal device 20 (such as network camera 20d) before the user terminal device 20 (such as network camera 20d) reduces load for energy saving, so as to confirm whether the user terminal device 20 (such as network camera 20d) is in an idle state (such as a non-use state).

[0062] [4] If the energy-saving software (such as value-added software 27) confirms that the user terminal device 20 (such as the webcam 20d) is in an idle state (such as a non-use state), the energy-saving software (such as value-added software 27) performs load reduction and energy saving on the user terminal device 20. It can also predict the next usage time of the user terminal device 20 (such as the webcam 20d) through the usage habit analysis model 16, so as to effectively release the load reduction state of the user terminal device 20 (such as the webcam 20d) in advance.

[0063] III. Taking the behavior of user A restarting user terminal device 20 as an example: User 80 has multiple user terminal devices 20 (such as modem 20a / wireless router 20b / wireless router 20c / network camera 20d) from service provider C (such as service provider) in user environment D, and uses habit analysis model 16 to analyze the restart behavior of user terminal devices 20 used by user A.

[0064] [1] Use the habit analysis model 16 to collect data and train the model on the managed user terminal equipment 20 (such as modem 20a / wireless router 20b / wireless router 20c).

[0065] [2] User A connects to user terminal device 20 (such as wireless router 20b) through user device A1 (such as smartphone / notebook / tablet computer, etc.) to access the Internet. After a period of time, user A feels that the network is abnormal and restarts user terminal device 20 (such as wireless router 20b). Then, user A reconnects to user terminal device 20 (such as wireless router 20b) through user device A1 to use it.

[0066] [3] The habit analysis model 16 predicts that the user terminal device 20 (such as wireless router 20b) should be in continuous use. This is different from the predicted behavior. Therefore, the restart behavior of this user terminal device 20 (such as wireless router 20b) is recorded.

[0067] [4] Subsequently, after feedback from user A, it was found that the user terminal device 20 (such as wireless router 20b) was unstable in function / network. This is a sign of degradation of the user terminal device 20 (such as wireless router 20b). Therefore, user terminal devices 20 (such as wireless router 20b) with this characteristic (such as unstable function / network or signs of degradation) were included in the replacement target.

[0068] Furthermore, the present invention also provides a computer-readable medium for a user terminal device installation method with usage habit analysis, which is applied in a computing device or computer having a processor and memory. The computer-readable medium stores instructions and can be executed by the computing device or computer through the processor and memory, so that when the computer-readable medium is executed, the content of the aforementioned user terminal device installation method with usage habit analysis is executed. In another embodiment, this computer-readable medium is a non-transitory computer-readable storage medium.

[0069] In one embodiment, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MPU), a microcontroller (MCU), an artificial intelligence (AI) processor, etc., and the memory may be random access memory (RAM), read-only memory (ROM), flash memory, memory card, hard disk (such as cloud / network / external hard disk), optical disk, USB flash drive, database, etc., and the computing device or computer may be a computer, smartphone, tablet computer, personal computer, notebook computer, desktop computer, server (such as cloud / remote / network / AI server), etc.

[0070] In summary, the user terminal device supply system, method and computer-readable medium system with usage habit analysis described in this invention have at least the following features, advantages or technical effects.

[0071] First, the remote management module 12 of the present invention can provide remote device management functions for user terminal device 20, and can also support remote settings, troubleshooting, restart and / or firmware updates for user terminal device 20. The software installation function of the software installation module 13 can ensure the consistency of software installation / removal and the correct installation / configuration for large-scale deployment.

[0072] Second, the automated software installation method of the present invention for user terminal equipment 20 can reduce the manual deployment costs of new user subscriptions, equipment vulnerability patching and other software changes, as well as the risk of equipment failure caused by human error, and can also improve user A's satisfaction.

[0073] Third, the usage habit analysis model 16 of the usage habit analysis module 15 of the present invention can upload usage data (such as processor / memory usage rate, network traffic, software usage records, etc.) to the user terminal software installation subsystem 10 through the user terminal device 20, and cooperate with the communication data of the external operation support system B (such as the internet access data / internet access behavior data / internet traffic data of user A or user device A1, etc.) to train and analyze the usage habit analysis model 16 using deep reinforcement learning methods, thereby avoiding interruption due to the operation of the user terminal device 20 when user A uses this user terminal device 20.

[0074] Fourth, the usage habit analysis model 16 of the usage habit analysis module 15 of the present invention can reduce the impact on the user A who is about to use the device when the user terminal device 20 is operating. In addition to ensuring the normal operation of the user terminal device 20, it also avoids reducing the user A's experience due to the operation of the user terminal device 20.

[0075] Fifth, the remote management agent software 24 of the present invention can automatically execute the commands sent by the remote device management server 30, and can also quickly obtain the updated software of the user terminal device 20 from the application software library 14 through differential update technology when the user terminal device 20 is updated.

[0076] Sixth, the present invention can make good use of the usage habit analysis model 16 to predict the working time of the user terminal device 20, and can also combine the monitoring of the device status of the user terminal device 20 (such as the real-time status of the device) to make accurate judgments, which is conducive to maintaining a good experience for user A and ensuring the smooth operation of the user terminal device 20.

[0077] VII. The isolation software 25 (such as a software isolation engine) of the present invention can effectively manage various software such as remote management agent software 24, status monitoring software 26, and value-added software 27 (such as value-added services) using software isolation technology. Moreover, this software isolation technology can isolate various software such as core software 23 (such as terminal core), built-in software, and value-added software 27 (such as value-added services) of user terminal device 20, so as to reduce interference between various software, enhance the security and stability of user terminal device 20, simplify the deployment and maintenance of software of user terminal device 20, and improve the user experience of user terminal device 20.

[0078] Eighth, the present invention can automatically analyze the usage status of user A through the reporting status of user terminal device 20 and the communication data of operation support system B, so as to avoid user A performing equipment operation on user terminal device 20 when using this user terminal device 20, and also to improve the quality of equipment operation of user terminal device 20 and user A's experience.

[0079] IX. This invention can be applied to the field of smart homes or other fields. It can also automatically install, update and / or remove software (such as value-added software 27) on user terminal equipment 20 by user terminal software installation subsystem 10, thereby improving the management efficiency of user terminal equipment 20 and ensuring the security and reliability of user terminal equipment 20.

[0080] The above embodiments are merely illustrative of the principles, features, and effects of the present invention and are not intended to limit the scope of the invention. Anyone skilled in the art can modify and alter the above embodiments without departing from the spirit and scope of the present invention. Any equivalent changes and modifications made using the content disclosed in this invention should still be covered by the claims. Therefore, the scope of protection of this invention should be as listed in the claims.

Claims

1. A user terminal equipment supply system with usage habit analysis capabilities, comprising: A user terminal device includes core software, remote management agent software, isolation software, status monitoring software, and at least one value-added software. The isolation software uses software isolation technology to manage the remote management agent software, status monitoring software, and value-added software of the user terminal device, while isolating the core software and value-added software of the user terminal device. A user terminal software installation subsystem is also provided, communicatively connected to the user terminal device. The user terminal software installation subsystem includes a usage habit analysis module and a software installation module. The usage habit analysis module has a usage habit analysis model, which uses a deep reinforcement learning method to analyze the core software, remote management agent software, isolation software, and value-added software. The user habit analysis model is trained using usage data of the user terminal device containing status monitoring software and value-added software, as well as traffic data from external operation support systems. The user habit analysis module then uses the deep reinforcement learning method to train the user habit analysis model to predict the software change-progress time of the user terminal device containing the core software, remote management agent software, isolation software, status monitoring software, and value-added software. Based on the software change-progress time predicted by the user habit analysis model, the software installation module performs software changes or software installations on the user terminal device when it is idle.

2. The user terminal equipment supply system as described in claim 1, wherein, The user terminal software installation subsystem further verifies the legitimacy of the user terminal device and the legitimacy of the user of the user terminal device, and then installs parameters on the user terminal device to provide basic services or Internet services to the user terminal device, so that the software installation module can initiate software changes on the user terminal device to install software on the user terminal device.

3. The user terminal equipment supply system as described in claim 1, wherein, The user terminal software installation subsystem further includes a remote management module. When the user terminal device is first installed, the user terminal device actively sends a request message to the remote management module to request the remote management module to remotely manage the user terminal device.

4. The user terminal equipment supply system as described in claim 1, wherein, The user terminal software installation subsystem further includes a remote management module. When a user or their user device subscribes to software from a service provider, the operation support system sends a software installation request to the remote management module. The remote management module then checks whether the user terminal device used by the user supports the installation of the software. If the user terminal device supports the installation of the software, the device information of the user terminal device is logged into the operation support system.

5. The user terminal equipment supply system as described in claim 1, wherein, The user terminal software installation subsystem further includes a remote management module and an application software library. When the remote management module receives a software change message initiated by the user terminal software installation subsystem, the software installation module provides a software download path to the user terminal device through the application software library. Then, the software installation module initiates a software change schedule to the user terminal device based on the software change time determined by the usage habit analysis model, so that the user terminal device can obtain the software from the application software library for updating.

6. The user terminal equipment supply system as described in claim 1, wherein, The user terminal software installation subsystem further includes a remote management module, which remotely manages the user terminal device. The usage habit analysis model uses data from the previous time interval of the user terminal device to predict the usage of the user terminal device in the current time interval. The remote management module then provides feedback on the prediction results of the usage habit analysis model based on the monitoring information of the device status of the user terminal device.

7. The user terminal equipment supply system as described in claim 1, wherein, When software changes are made to the user terminal device, the user terminal device first checks whether its dependent software is not busy or is busy. When the dependent software is not busy, the software installation module makes the software changes to the user terminal device.

8. The user terminal equipment supply system as described in claim 1, wherein, The usage habit analysis module further utilizes the device status of the user terminal device, the communication data of the operation support system, and the replay memory of the usage habit analysis model to perform deep reinforcement learning training on the usage habit analysis model. It also uses real online data from the user terminal device to continuously train the usage habit analysis model, so that the usage habit analysis model can approach or match the user's usage habits of the user terminal device.

9. A method for supplying user terminal equipment with usage habit analysis capabilities, comprising: A user terminal device installation system is provided, comprising a user terminal device and a user terminal software installation subsystem. The user terminal device has core software, a remote management agent software, an isolation software, a status monitoring software, and at least one value-added software. The isolation software uses software isolation technology to manage the remote management agent software, status monitoring software, and value-added software of the user terminal device, while isolating the core software and value-added software. The user terminal software installation subsystem has a usage habit analysis module and a software installation module. The usage habit analysis module uses a deep reinforcement learning method to analyze the core software, remote management agent software, isolation software, status monitoring software, and value-added software. A usage habit analysis model is trained using usage data of the user terminal device containing the core software and value-added software, as well as traffic data of an operations support system. The usage habit analysis model, trained by the deep reinforcement learning method, predicts the software change-progress time of the user terminal device containing the core software, remote management agent software, isolation software, status monitoring software, and value-added software. Based on the software change-progress time predicted by the usage habit analysis model, the software installation module performs software changes or software installations on the user terminal device when it is idle.

10. The user terminal equipment installation method as described in claim 9 further includes the user terminal software installation subsystem confirming the legitimacy of the user terminal equipment and the legitimacy of the user of the user terminal equipment, and then installing parameters on the user terminal equipment to provide basic services or Internet access services to the user terminal equipment, so that the software installation module can initiate software changes on the user terminal equipment to install software on the user terminal equipment.

11. The user terminal device installation method as described in claim 9 further includes, when a user or its user device subscribes to software from a service provider, the operation support system sends a software installation request to the remote management module, and the remote management module checks whether the user terminal device used by the user supports the installation of the software, so that if the user terminal device supports the installation of the software, the device information of the user terminal device is logged to the operation support system.

12. The user terminal device installation method as described in claim 9 further includes, when a remote management module of the user terminal software installation subsystem receives a software change message initiated by the user terminal software installation subsystem, the software installation module provides a software download path to the user terminal device through an application software library of the user terminal software installation subsystem, and then the software installation module initiates a software change schedule to the user terminal device based on the software changeable time analyzed by the usage habit analysis model, so that the user terminal device can obtain the software from the application software library for updating.

13. The user terminal device installation method as described in claim 9 further includes remote device management of the user terminal device by a remote management module of the user terminal software installation subsystem, so that the usage habit analysis model can predict the usage of the user terminal device in the current time interval using data from the previous time interval of the user terminal device, and then the remote management module can provide feedback on the prediction results of the usage habit analysis model based on the monitoring information of the device status of the user terminal device.

14. The user terminal device installation method as described in claim 9 further includes, when making software changes to the user terminal device, the user terminal device first checks whether its dependent software is not busy or is busy, so that when the dependent software is not busy, the software installation module performs software changes to the user terminal device.

15. The user terminal equipment installation method as described in claim 9 further includes the user habit analysis module using the device status of the user terminal equipment, the service data of the operation support system and the replay memory of the user habit analysis model to perform deep reinforcement learning training on the user habit analysis model, and using real online data of the user terminal equipment to continuously train the user habit analysis model so that the user habit analysis model approaches or matches the user's usage habits of the user terminal equipment.

16. A computer-readable medium, applied in a computing device or computer, storing instructions and executing the computer-readable medium via a processor and memory, so as to perform, when executing the computer-readable medium, the user terminal device supply method with usage habit analysis as described in any one of claims 9 to 15.

Images