Intelligent awakening method based on multi-mode perception, storage medium and electronic device

Abstract

The invention discloses an intelligent wake-up method based on multi-modal perception, a storage medium and an electronic device, and relates to the technical field of smart home / smart home, and the intelligent wake-up method based on multi-modal perception comprises the steps: collecting preset user wake-up decision related data; obtaining a user wake-up strategy according to the user wake-up decision related data; wherein the user wake-up strategy comprises a plurality of wake-up modes, an execution sequence of the plurality of wake-up modes and parameter values of the wake-up modes; executing the corresponding wake-up mode according to the execution sequence and the parameter value of the wake-up mode; wherein whether the user is successfully awakened or not is verified after each awakening mode is executed; in response to successful awakening of the user, stopping execution of the subsequent awakening mode; and in response to the fact that the user is not successfully awakened, executing the next awakening mode according to the execution sequence. According to the intelligent wake-up method based on multi-mode perception, the storage medium and the electronic device provided by the invention, the success rate of user wake-up is improved.

Description

Technical Field

[0001] The present application relates to the technical field of smart home / smart family, and in particular, to an intelligent wake-up method, a storage medium, and an electronic device based on multi-modal perception. Background Art

[0002] Existing wake-up systems usually adopt only one wake-up mode. A relatively common method is to set an alarm clock for waking up. However, if the alarm sound is too small, it may cause the user not to be awakened. If the user turns off the alarm clock after being awakened by the alarm, they may fall asleep again after waking up briefly. Another example is to use light wake-up to wake up the user. Due to individual differences, some users are not sensitive to light, and even turning on the light or opening the curtain cannot wake up the user. Or some users wear eye masks when sleeping, or sleep covered in the quilt. In these scenarios, the light wake-up strategy cannot be executed.

[0003] It can be seen that how to improve the success rate of waking up has become an urgent problem to be solved. Summary of the Invention

[0004] The present application provides an intelligent wake-up method, a storage medium, and an electronic device based on multi-modal perception, which are used to solve the defect of low wake-up success rate in the prior art and realize the improvement of the wake-up success rate.

[0005] The present application provides an intelligent wake-up method based on multi-modal perception, including: collecting preset user wake-up decision-related data; obtaining a user wake-up strategy according to the user wake-up decision-related data; wherein the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; executing the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; wherein, after each wake-up method is executed, it is verified whether the user is successfully awakened; in response to the user being successfully awakened, the execution of the subsequent wake-up methods is stopped; in response to the user not being successfully awakened, the next wake-up method is executed according to the execution order.

[0006] According to the intelligent wake-up method based on multi-modal perception provided by the present application, the user wake-up decision-related data at least includes one of the following data: user physiological data, user sleep behavior data, and user sleep environment data.

[0007] An intelligent wake-up method based on multi-modal perception provided by the present application, wherein the user's physiological data includes at least one of the following data: heart rate, respiratory data, blood oxygen saturation, blood pressure, body temperature, height, weight, and exercise data; the user's sleep behavior data includes at least one of the following data: bedtime, sleep duration, sleep cycle, number of awakenings, number of turns, and mattress pressure; the sleep environment data includes at least one of the following data: temperature, humidity, light intensity, and sound.

[0008] An intelligent wake-up method based on multi-modal perception provided by the present application, obtaining a user wake-up strategy according to the user wake-up decision-related data, including: obtaining an evaluation result of the user's sleep state according to a combination of one or more of the user's physiological data, the user's sleep behavior data, and the user's sleep environment data; wherein, the evaluation result of the user's sleep state includes a combination of one or more of the following evaluation results: an evaluation result of the user's sleep quality, an evaluation result of the user's physiological state, and an evaluation result of the user's mental state; obtaining a user wake-up strategy according to the evaluation result of the user's sleep state.

[0009] An intelligent wake-up method based on multi-modal perception provided by the present application, obtaining a user wake-up strategy according to the evaluation result of the user's sleep state, including: obtaining an initial user wake-up strategy according to the evaluation result of the user's sleep state; adjusting the initial user wake-up strategy by using the wake-up method user configuration data to obtain the user wake-up strategy.

[0010] An intelligent wake-up method based on multi-modal perception provided by the present application, the method further includes: giving at least one feedback information of health condition reminder, medical advice, exercise advice, rest advice, diet advice according to the evaluation result of the user's sleep state.

[0011] An intelligent wake-up method based on multi-modal perception provided by the present application, the wake-up method is a combination of one or more of the following wake-up methods: light wake-up, sound wake-up, vibration wake-up, and temperature adjustment wake-up.

[0012] An intelligent wake-up method based on multi-modal perception provided by the present application, the method further includes: adaptively adjusting the user wake-up strategy according to the verification result of whether the user is successfully awakened.

[0013] An intelligent wake-up method based on multi-modal perception provided by the present application, verifying whether the user is successfully awakened, including: comprehensively verifying whether the user is successfully awakened through at least one preset verification mode.

[0014] According to an intelligent wake-up method based on multi-modal perception provided by the present application, comprehensively verifying whether the user is successfully woken up through at least one preset verification mode includes: obtaining the wake-up success rate judgment result of each preset verification mode in the at least one preset verification mode; calculating the weighted sum of the wake-up success rate judgment results of the at least one preset verification mode; in response to the weighted sum being greater than a preset success rate threshold, determining that the user is successfully woken up; otherwise, determining that the user is not successfully woken up.

[0015] According to an intelligent wake-up method based on multi-modal perception provided by the present application, obtaining the wake-up success rate judgment result of each preset verification mode in the at least one preset verification mode includes: obtaining the wake-up success rate judgment result of the corresponding preset verification mode according to the change situation of the preset data index corresponding to each preset verification mode.

[0016] According to an intelligent wake-up method based on multi-modal perception provided by the present application, the preset verification mode includes one or a combination of more than one of the following verification methods: verifying by detecting the weight change of a weight sensor placed under the mattress, verifying by detecting the user's movement situation, and verifying by detecting the user's bioelectric signal.

[0017] The present application also provides an intelligent wake-up device based on multi-modal perception, including: a data acquisition module, configured to: acquire preset user wake-up decision-related data; a wake-up decision module, configured to: obtain a user wake-up strategy according to the user wake-up decision-related data; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; a wake-up execution module, configured to: execute the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; wherein, verify whether the user is successfully woken up after each wake-up method is executed; in response to the user being successfully woken up, stop the execution of the subsequent wake-up methods; in response to the user not being successfully woken up, execute the next wake-up method according to the execution order.

[0018] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to execute to implement the intelligent wake-up method based on multi-modal perception as described in any one of the above.

[0019] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, and wherein the program executes to implement the intelligent wake-up method based on multi-modal perception as described in any one of the above when running.

[0020] The present application also provides a computer program product, including a computer program, which when executed by a processor, implements the intelligent wake-up method based on multi-modal perception as described in any one of the above.

[0021] The intelligent wake-up method, storage medium and electronic device based on multi-modal perception provided by the present application collect preset user wake-up decision-related data, and obtain a user wake-up strategy according to the user wake-up decision-related data; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of multiple wake-up methods, and the parameter values of the wake-up methods, execute the corresponding wake-up methods according to the execution order and the parameter values of the wake-up methods, verify whether the user is successfully woken up after each wake-up method is executed, in response to the user being successfully woken up, stop the execution of subsequent wake-up methods, and in response to the user not being successfully woken up, execute the next wake-up method according to the execution order, thereby improving the success rate of user wake-up. Description of the Drawings

[0022] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a schematic diagram of the hardware environment of an intelligent wake-up method based on multi-modal perception according to an embodiment of the present application;

[0025] Figure 2 It is one of the flow diagrams of the intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of user wake-up decision-related data in the intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application;

[0027] Figure 4 It is another flow diagram of the intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application;

[0028] Figure 5 It is a third flow diagram of the intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application;

[0029] Figure 6 It is a schematic diagram of the structure of an intelligent wake-up device based on multi-modal perception provided by an embodiment of the present application;

[0030] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0031] Figure 8 It is a schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] According to one aspect of an embodiment of the present application, a smart wake-up method based on multimodal perception is provided. The smart wake-up method based on multimodal perception is widely used in smart home (Smart Home), smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned smart wake-up method based on multimodal perception can be applied to Figure 1 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 1 As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0035] The above network may include, but is not limited to, at least one of the following: a wired network, a wireless network. The above wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, a local area network. The above wireless network may include, but is not limited to, at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing device, a smart dishwasher, a smart projection device, a smart TV, a smart drying rack, a smart curtain, smart audio and video, a smart socket, a smart speaker, a smart sound box, a smart fresh air device, smart kitchen and bathroom equipment, smart bathroom equipment, a smart floor sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification device, a smart steam box, a smart microwave oven, a smart kitchen water heater, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0036] Figure 2 It is one of the flow diagrams of the intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application. As Figure 2 shown, the method includes:

[0037] Step S1, collect data related to the user wake-up decision preset.

[0038] The data related to the user wake-up decision includes various data on which the user wake-up strategy depends. It is possible to preset which data needs to be collected and obtain it according to the corresponding collection method.

[0039] Step S2, obtain the user wake-up strategy according to the data related to the user wake-up decision; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods.

[0040] The user wake-up strategy is obtained by analyzing the data related to the user wake-up decision. The given user wake-up strategy includes multiple wake-up methods, among which the multiple wake-up methods include at least two wake-up methods. The user wake-up strategy also includes the execution order of the given multiple wake-up methods and the parameter values of the wake-up methods. Among them, the data related to the user wake-up decision can be input into a prediction model to obtain the user wake-up strategy.

[0041] For the parameter values of the wake-up method, for example, for the method of waking up the user by sound, what kind of music to play, how many seconds to play, which segment to play; for the method of waking up by light, whether to wake up by turning on the light or by opening the curtain, etc.

[0042] Step S3: Execute the corresponding wake-up method according to the execution order and the parameter value of the wake-up method; wherein, after each wake-up method is executed, verify whether the user is successfully awakened; in response to the user being successfully awakened, stop the execution of the subsequent wake-up methods; in response to the user not being successfully awakened, execute the next wake-up method according to the execution order.

[0043] Execute each wake-up method according to the execution order of multiple wake-up methods, and execute based on the corresponding parameter values when executing each wake-up method. When executing each wake-up method according to the execution order of multiple wake-up methods, verify whether the user is successfully awakened after each wake-up method is executed. If it is verified that the user has been successfully awakened, stop executing the subsequent wake-up methods; if it is verified that the user has not been successfully awakened, execute the next wake-up method according to the execution order of multiple wake-up methods. If the user has not been successfully awakened all the time, all the multiple wake-up methods in the user wake-up strategy will be executed.

[0044] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application obtains a user wake-up strategy by collecting preset user wake-up decision-related data; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of multiple wake-up methods, and the parameter values of the wake-up methods. Execute the corresponding wake-up method according to the execution order and the parameter values of the wake-up method, verify whether the user is successfully awakened after each wake-up method is executed, in response to the user being successfully awakened, stop the execution of the subsequent wake-up methods, in response to the user not being successfully awakened, execute the next wake-up method according to the execution order, which improves the success rate of user wake-up.

[0045] According to an intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the user wake-up decision-related data at least includes one of the following data: user physiological data, user sleep behavior data, and user sleep environment data.

[0046] In order to consider the individual differences of users when giving the user wake-up strategy, improve the pertinence and adaptability of the user wake-up strategy, enhance the user experience, and give the user wake-up strategy by integrating various factors to improve the accuracy of the user wake-up strategy. In the embodiments of the present application, when collecting the preset user wake-up decision-related data, collect at least one of user physiological data, user sleep behavior data, and user sleep environment data.

[0047] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application improves the pertinence, adaptability, and accuracy of the user wake-up strategy by collecting at least one of user physiological data, user sleep behavior data, and user sleep environment data.

[0048] An intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application, where the user's physiological data at least includes one of the following data: heart rate, respiratory data, blood oxygen saturation, blood pressure, body temperature, height, weight, and exercise data; the user's sleep behavior data at least includes one of the following data: bedtime, sleep duration, sleep cycle, number of awakenings, number of turns, and mattress pressure; the sleep environment data at least includes one of the following data: temperature, humidity, light intensity, and sound.

[0049] The user's physiological data, user's sleep behavior data, and user's sleep environment data can be obtained in various ways. This includes obtaining them through direct measurement, obtaining them from other devices, or further analyzing the measured data, etc.

[0050] Figure 3 It is a schematic diagram of data related to user wake-up decision-making in the intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application. Among them, the collected user's physiological data includes heart rate, respiratory data, blood oxygen saturation, blood pressure, body temperature, height, weight, and exercise data. The collected user's sleep behavior data includes bedtime, sleep duration, sleep cycle, number of awakenings, number of turns, and mattress pressure. The collected sleep environment data includes temperature, humidity, light intensity, and sound.

[0051] For the user's physiological data, for example, by connecting with the user's mobile phone or devices such as smart watches and bracelets worn, to collect the user's physiological data in real time, and these data include but are not limited to height, weight, daily exercise data, heart rate, respiratory data, blood oxygen saturation, blood pressure, etc.

[0052] For the user's sleep behavior data, for example, by installing sensors in the sleep environment or using devices such as smart mattresses, data such as the number of turns of the user, mattress pressure, and sleep duration are collected, and these data can provide information about the user's sleep quality, sleep posture, and possible sleep problems.

[0053] Algorithms and technologies such as machine learning, artificial neural networks, or support vector machines can be used to construct and train a model based on known sleep state data. The data related to user wake-up decision-making is input into a predefined model for comparison and classification to determine the user's sleep cycle, and these sleep cycles include the falling asleep period, light sleep period, deep sleep period, REM period (rapid eye movement sleep period), etc.

[0054] For the sleep environment data, for example, environmental sensors such as temperature and humidity sensors, light sensors, and sound sensors are installed in the user's sleep environment to collect environmental data in real time. These data include but are not limited to temperature, humidity, light intensity, sound (breathing sound, snoring sound, environmental noise), etc.

[0055] The original data collected can be pre - processed, including removing noise, filtering, noise reduction, etc., and then extracting features in the data, such as frequency, amplitude, phase, etc., to improve the quality and accuracy of the data.

[0056] The intelligent wake - up method based on multi - modal perception provided by the embodiments of the present application further improves the pertinence, adaptability and accuracy of the user wake - up strategy by setting specific data of user physiological data, user sleep behavior data and user sleep environment data.

[0057] According to an intelligent wake - up method based on multi - modal perception provided by the embodiments of the present application, obtaining a user wake - up strategy according to the user wake - up decision - related data includes: obtaining a user sleep state evaluation result according to a combination of one or more of the user physiological data, the user sleep behavior data and the user sleep environment data; wherein, the user sleep state evaluation result includes a combination of one or more of the following evaluation results: user sleep quality evaluation result, user physiological state evaluation result and user mental state evaluation result; obtaining a user wake - up strategy according to the user sleep state evaluation result.

[0058] Obtaining a user sleep state evaluation result according to a combination of one or more of the user physiological data, the user sleep behavior data and the user sleep environment data, and the user sleep state evaluation result includes a combination of one or more of the user sleep quality evaluation result, the user physiological state evaluation result and the user mental state evaluation result. Among them, the user physiological data, the user sleep behavior data and the user sleep environment data can be input into a machine learning model to obtain the user sleep quality evaluation result, the user physiological state evaluation result and the user mental state evaluation result.

[0059] For example, by using machine learning or deep - learning algorithms, by analyzing respiratory data, the respiratory rate, respiratory depth and respiratory pattern of the user can be identified, etc., to judge the user's sleep quality, fatigue level and mental state, etc.; by monitoring the user's body movements and postures, the user's sleep behavior patterns can be identified, such as the number of turns, the type of limb movement, etc., to judge the user's sleep quality, sleep habits and potential sleep problems. By analyzing the sound data in the sleep environment, the user's sound characteristics, such as snoring, teeth grinding, etc., can be identified, to judge the user's sleep state and possible sleep disorders; by analyzing the user's physiological data, such as heart rate, blood pressure, body temperature, etc., the user's physiological state and health condition can be identified, to judge the user's sleep quality and mental state. In addition, according to the user's sleep cycle, combined with relevant indicators, such as sleep time, sleep efficiency, sleep onset time, etc., the user's sleep quality can be evaluated.

[0060] Obtain the user wake-up strategy based on the evaluation result of the user's sleep state. Different user wake-up strategies corresponding to different evaluation results of the user's sleep state can be preset. After obtaining the evaluation result of the user's sleep state, the corresponding user wake-up strategy can be obtained through matching.

[0061] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application obtains the evaluation result of the user's sleep state based on at least one of the user's physiological data, the user's sleep behavior data, and the user's sleep environment data. The evaluation result of the user's sleep state includes one or a combination of more of the evaluation result of the user's sleep quality, the evaluation result of the user's physiological state, and the evaluation result of the user's mental state. The user wake-up strategy is obtained based on the evaluation result of the user's sleep state, which further improves the pertinence, adaptability, and accuracy of the user wake-up strategy.

[0062] According to an intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the obtaining of the user wake-up strategy based on the evaluation result of the user's sleep state includes: obtaining an initial user wake-up strategy based on the evaluation result of the user's sleep state; and adjusting the initial user wake-up strategy by using the user configuration data of the wake-up method to obtain the user wake-up strategy.

[0063] The learning result of the model can be continuously adjusted according to the user's configuration data of the wake-up method, and the system automatically adjusts the user wake-up decision according to the feedback, such as adjusting the wake-up time and optimizing the wake-up mode.

[0064] Therefore, when obtaining the user wake-up strategy based on the evaluation result of the user's sleep state, an initial user wake-up strategy is obtained based on the evaluation result of the user's sleep state, and the initial user wake-up strategy is adjusted by using the user configuration data of the wake-up method to obtain the user wake-up strategy.

[0065] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application obtains an initial user wake-up strategy based on the evaluation result of the user's sleep state, and adjusts the initial user wake-up strategy by using the user configuration data of the wake-up method to obtain the user wake-up strategy, so that the intelligent output wake-up strategy is combined with the user's needs, which is more user-friendly and improves the user experience.

[0066] According to an intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the method further includes: giving at least one feedback information of health status reminder, medical advice, exercise advice, rest advice, and diet advice according to the evaluation result of the user's sleep state.

[0067] The obtained evaluation result of the user's sleep state can be used to remind the user to pay attention to their health and give suggestions on medical treatment, rest, exercise, diet, etc. when necessary.

[0068] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application gives at least one piece of feedback information such as health status reminder, medical advice, exercise advice, rest advice, and diet advice according to the evaluation result of the user's sleep state, improving the user experience.

[0069] An intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the wake-up method is one or a combination of the following wake-up methods: light wake-up, sound wake-up, vibration wake-up, and temperature adjustment wake-up.

[0070] The user wake-up strategy is carried out in a multi-mode combination way, and the multiple wake-up methods include at least one of light wake-up, sound wake-up, vibration wake-up, and temperature adjustment wake-up. Among them:

[0071] Light wake-up: The brightness of the bedroom can be gradually increased to imitate natural daylight to gradually wake up the user. This method can be realized by using smart bulbs or smart curtains.

[0072] Music or sound wake-up: The user's favorite music or sounds, such as natural bird songs, gentle melodies, etc., can be played to attract the user's attention. It can be realized through a smart speaker or a mobile application.

[0073] Vibration wake-up: Devices such as a smart mattress or a vibrator can be used to generate slight vibrations to stimulate the user's physical sensations and achieve the waking-up effect.

[0074] Temperature adjustment wake-up: The temperature of the bedroom can be appropriately adjusted according to the user's body temperature and the ambient temperature to help the user wake up more comfortably.

[0075] The above various wake-up methods can be combined to use to improve the success rate and comfort of waking up. For example, gentle music can be played first, then the light intensity can be gradually increased, and finally the wake-up process can be ended with slight vibrations. It can be understood that if it is judged that the user has been woken up after a certain wake-up method is executed, there is no need to execute the subsequent wake-up methods.

[0076] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application further improves the success rate of waking up the user by setting the wake-up method to include at least one of light wake-up, sound wake-up, vibration wake-up, and temperature adjustment wake-up.

[0077] According to an intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the method further includes: adaptively adjusting the user wake-up strategy according to the verification result of whether the user is successfully woken up.

[0078] The user wake-up strategy can be adaptively adjusted according to the verification result of whether the user is successfully awakened. For example, if it is known that the user cannot be successfully awakened by using the output user wake-up strategy, the parameters in the wake-up method can be adjusted, such as turning up the volume. Or adjust the wake-up method in the user wake-up strategy. If the user still cannot be awakened by the optimized user wake-up strategy, further optimization and adjustment will be carried out on this basis to ensure that the user is successfully awakened.

[0079] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application further improves the success rate of user wake-up by adaptively adjusting the user wake-up strategy according to the verification result of whether the user is successfully awakened.

[0080] According to an intelligent wake-up method based on multi-modal perception provided by an embodiment of the present application, verifying whether the user is successfully awakened includes: comprehensively verifying whether the user is successfully awakened through at least one preset verification mode.

[0081] According to the given user wake-up strategy, execute the corresponding wake-up method according to the execution order of multiple wake-up methods and the parameter values of the wake-up methods. Among them, after each wake-up method is executed, verify whether the user is successfully awakened; if the user is successfully awakened, stop the execution of the subsequent wake-up methods; if the user is not successfully awakened, execute the next wake-up method according to the execution order.

[0082] Among them, when verifying whether the user is successfully awakened after each wake-up method is executed, comprehensively verify whether the user is successfully awakened through at least one preset verification mode.

[0083] It is very important to mutually verify the wake-up results with multiple preset verification modes, such as the following examples.

[0084] Example 1: After the user hears the alarm clock and gets up to open the curtain, the light intensity in the room changes at this time, but the user goes back to sleep. If only the light intensity in the room is used as the standard for whether the user gets up, the judgment will be incorrect in this scenario.

[0085] Example 2: After the user hears the alarm clock, gets up and gets out of bed, and continues to sleep on the sofa. At this time, the pressure on the mattress decreases. If only the pressure on the mattress is used as the only standard for judging whether the user gets up, the judgment will be incorrect in this scenario.

[0086] Therefore, it is necessary to adopt a method of mutually verifying with at least one preset verification mode to ensure that the user gets up successfully and improve the success rate of wake-up.

[0087] By comprehensively verifying whether the user is successfully awakened through at least one preset verification mode, the accuracy of judging whether the user is successfully awakened can be improved. Thus, when it is judged that the user is not successfully awakened, subsequent awakening methods can be further executed to improve the success rate of user awakening. When it is judged that the user is successfully awakened, subsequent awakening methods are no longer executed to avoid disturbing the user and enhance the user experience.

[0088] The intelligent awakening method based on multi-modal perception provided by the embodiments of this application comprehensively verifies whether the user is successfully awakened through at least one preset verification mode, further improving the success rate of user awakening and enhancing the user experience.

[0089] According to an intelligent awakening method based on multi-modal perception provided by the embodiments of this application, the comprehensive verification of whether the user is successfully awakened through at least one preset verification mode includes: obtaining the awakening success rate judgment results of each verification mode in the at least one preset verification mode; calculating the weighted sum of the awakening success rate judgment results of the at least one preset verification mode; in response to the weighted sum being greater than a preset success rate threshold, determining that the user is successfully awakened; otherwise, determining that the user is not successfully awakened.

[0090] When comprehensively verifying whether the user is successfully awakened through at least one preset verification mode, each verification mode can give an awakening success rate judgment result. A weight is set for each verification mode, and the higher the weight, the more credible the awakening success rate judgment result of this verification mode. Calculate the weighted sum of the awakening success rate judgment results of at least one preset verification mode and compare it with the preset success rate threshold. If the weighted sum of the awakening success rate judgment results of at least one preset verification mode is greater than the preset success rate threshold, it is determined that the user is successfully awakened; if the weighted sum of the awakening success rate judgment results of at least one preset verification mode is less than or equal to the preset success rate threshold, it is determined that the user is not successfully awakened.

[0091] For example, use A = {a1, a2, a3, a4,..., a n} to represent n preset verification modes, and ai represents the i-th verification mode. In addition, different weights are set for each verification mode. P = {p1, p2, p3, p4,... pn} is a weight set, and each pi corresponds to ai in the A set. This weight is used to represent the degree of recognition of this verification method, and the higher the degree of recognition, the greater the weight. Among them, p1 + p2 +... + pn = 1, and this weight can be adaptively adjusted.

[0092] Define a success rate threshold β. When β < a1 * p1 + a2 * p2 +... + an * pn, it means that the n preset verification modes mutually verify successfully, the user has successfully woken up, and the system can stop the awakening operation on the user.

[0093] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application calculates the weighted sum of the wake-up success rate judgment results of at least one preset verification mode by obtaining the wake-up success rate judgment results of each verification mode in at least one preset verification mode. In response to the weighted sum being greater than the preset success rate threshold, it is determined that the user is successfully woken up; otherwise, it is determined that the user is not successfully woken up, improving the judgment accuracy of whether the user is successfully woken up.

[0094] According to an intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the obtaining of the wake-up success rate judgment results of each preset verification mode in the at least one preset verification mode includes: obtaining the wake-up success rate judgment results of the corresponding preset verification mode according to the change of the preset data index corresponding to each preset verification mode.

[0095] Each preset verification mode corresponds to a data index, and the wake-up success rate judgment result of this verification mode can be determined by the change of the data index corresponding to this preset verification mode.

[0096] For example, the light wake-up strategy. The light intensity of the surrounding environment before waking up is S1, and the light intensity increases to S2 after waking up. Then the light intensity gain △S = S2 - S1. The larger △S is, the higher the success rate of waking up the user in this way. Therefore, there is a non-linear functional relationship between △S and the wake-up success rate, and this functional relationship can be fitted by data. Therefore, there is a non-linear functional relationship between the information gain of the data index corresponding to each verification mode and the wake-up success rate of this verification mode. By obtaining this non-linear functional relationship, the wake-up success rate judgment results of the corresponding verification modes are obtained according to the change of the preset data index corresponding to each verification mode.

[0097] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application further improves the judgment accuracy of whether the user is successfully woken up by obtaining the wake-up success rate judgment results of the corresponding preset verification modes according to the change of the preset data index corresponding to each preset verification mode.

[0098] According to an intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application, the preset verification mode includes one or a combination of more than one of the following verification methods: verifying by detecting the weight change of a weight sensor placed under the mattress, verifying by detecting the user's movement situation, and verifying by detecting the user's bioelectric signal.

[0099] Mutual verification is performed through at least one preset verification mode to ensure that the user gets up successfully. For example, the preset verification mode can include the following methods:

[0100] Adopt a weight sensor: Place a weight sensor under the mattress. When a change in weight is detected, it is determined that the user has gotten up.

[0101] Utilize motion detection: Detect the user's motion through a camera or motion sensor. If actions such as the user turning over or getting up in bed are detected, it is determined that the user has woken up.

[0102] Bioelectrical signal monitoring: Determine whether the user has woken up by monitoring the user's bioelectrical signals (such as electrocardiogram, electroencephalogram, etc.).

[0103] The intelligent wake-up method based on multi-modal perception provided by the embodiments of this application further improves the accuracy of judging whether the user has been successfully woken up by setting at least one preset verification mode, including verifying by detecting the weight change of the weight sensor placed under the mattress, verifying by detecting the user's motion situation, and verifying by detecting the user's bioelectrical signals.

[0104] Figure 4 It is the second flow diagram of the intelligent wake-up method based on multi-modal perception provided by the embodiments of this application. As Figure 4 shown, the method includes:

[0105] Input the user's physiological data, user's sleep behavior data, and user's sleep environment data into the prediction model to obtain the user wake-up strategy output by the prediction model; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of multiple wake-up methods, and the parameter values of the wake-up methods.

[0106] Perform a wake-up operation on the user according to the wake-up strategy, and comprehensively verify whether the user has been successfully woken up by using at least one preset verification mode. At the same time, adaptively optimize the user wake-up strategy according to the result of whether the user has been successfully woken up to improve the success rate and comfort of waking up.

[0107] Figure 5 It is the third flow diagram of the intelligent wake-up method based on multi-modal perception provided by the embodiments of this application. As Figure 5 shown, the method includes:

[0108] Collect the user's physiological data, sleep behavior data, and sleep environment data through a variety of sensors and monitoring devices;

[0109] Perform processing such as cleaning, filtering, and noise reduction on the collected original data, and extract the feature data related to waking up;

[0110] Utilize machine learning algorithms to classify and identify the extracted features, identify the user's sleep state and behavior patterns, and design personalized combined wake-up methods;

[0111] When the preset wake-up time set by the user arrives, the user is stimulated through a preset combined wake-up method (such as sound, light, etc.) to achieve the wake-up effect;

[0112] Multiple verification modes are mutually verified to ensure successful wake-up, and according to the user's feedback on the wake-up method, the wake-up strategy is adaptively adjusted and optimized to improve the success rate and comfort of wake-up;

[0113] Continuously monitor various data, and according to the evaluation results of the user's sleep quality, physiological state, and mental state obtained through analysis, remind the user to pay attention to their health status, and give suggestions on seeking medical treatment and rest when necessary.

[0114] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present invention can comprehensively utilize multiple perception modes, consider the individual differences of users, accurately judge the user's state, and has the functions of health consideration and customization, so as to improve the wake-up success rate, provide personalized services, reduce the false wake-up rate, pay attention to the user's health, simplify the operation, and improve the user experience.

[0115] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application has the following characteristics:

[0116] (1) Multi-mode perception: Comprehensively utilize multiple perception modes, including various methods such as light, music, vibration, etc., to achieve a more accurate and comprehensive wake-up effect. This multi-mode design can adapt to the individual differences of different users and different wake-up requirements.

[0117] (2) Artificial intelligence collaborative judgment: By introducing artificial intelligence technology, it can process and analyze the user's physiological data, sleep behavior data, and sleep environment data in real time, so as to more accurately judge the user's sleep state and individual characteristics. This collaborative judgment method can improve the adaptability and personalization of wake-up services.

[0118] (3) Personalized wake-up service: It not only considers the user's different sleep states, but also further considers the individual differences of users, such as age, gender, living habits, etc. By providing personalized wake-up services, it can better meet the user's needs and improve the user's satisfaction.

[0119] (4) Health consideration: The connection of health monitoring devices and the health reminder function are introduced, and wake-up services can be provided on the premise of ensuring the user's health. This health consideration design can increase the user's trust and loyalty to the product.

[0120] (5) Customization and remote control: By providing customization and remote control functions, users can set and control the product according to their own preferences and needs. This design can improve the usability and user experience of the product.

[0121] (6) Adaptive learning: It can automatically correct errors according to the learning situation, reducing user operations and human intervention.

[0122] The intelligent wake-up method based on multi-modal perception provided by the embodiments of the present application has the following advantages:

[0123] (1) Improve the wake-up success rate: By comprehensively using various wake-up modes, such as light, music, vibration, etc., it can provide a more comprehensive wake-up service for different user individuals and wake-up scenarios, thus improving the wake-up success rate.

[0124] (2) Personalized service: By introducing artificial intelligence technology, it can analyze users' physiological data and behavior patterns to provide personalized wake-up services. For example, for users who like to get up early to exercise, their favorite music can be played or the light can be set to gradually increase in intensity when waking up; for users with sleep disorders, a gentle wake-up method and reminder service can be provided, etc.

[0125] (3) Health consideration: By connecting health monitoring devices, it can obtain users' health data and remind users to pay attention to their health status when necessary. This can improve users' trust and loyalty.

[0126] (4) Customizability: It provides a customizable function that allows users to set and control the product according to their own preferences and needs. This design can improve usability and user experience while meeting the needs of different users.

[0127] (5) Improve user experience: By introducing advanced technologies such as multi-modal perception and artificial intelligence collaborative judgment, it can provide more accurate and personalized wake-up services, thus improving the user experience.

[0128] (6) Reduce the false wake-up rate: By comprehensively analyzing users' physiological data and environmental data, it can more accurately judge whether the user has really woken up, thus avoiding the occurrence of false wake-up situations. This design of reducing the false wake-up rate can improve reliability and stability.

[0129] (7) Energy conservation and environmental protection: It can effectively utilize energy by intelligently controlling parameters such as the wake-up time and light intensity, achieving the effect of energy conservation and environmental protection.

[0130] (8) Safe and reliable: It can connect health monitoring devices to obtain users' health data, and at the same time can set a safe and reliable remote control method to ensure users' safety and privacy.

[0131] The intelligent wake-up device based on multi-modal perception provided by the present application will be described below. The intelligent wake-up device described below can be mutually corresponding and referred to the intelligent wake-up method based on multi-modal perception described above.

[0132] Figure 6 This is a schematic structural diagram of an intelligent wake-up device based on multimodal perception provided by an embodiment of the present application. As Figure 6 shown, the device includes a data acquisition module 10, a wake-up decision module 20, and a wake-up execution module 30, where: The data acquisition module 10 is configured to: acquire preset user wake-up decision-related data; The wake-up decision module 20 is configured to: obtain a user wake-up strategy based on the user wake-up decision-related data; where, the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; The wake-up execution module 30 is configured to: execute the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; where, after each wake-up method is executed, it is verified whether the user is successfully woken up; in response to the user being successfully woken up, the execution of subsequent wake-up methods is stopped; in response to the user not being successfully woken up, the next wake-up method is executed according to the execution order.

[0133] The intelligent wake-up device provided by the embodiment of the present application acquires preset user wake-up decision-related data, and obtains a user wake-up strategy based on the user wake-up decision-related data; where, the user wake-up strategy includes multiple wake-up methods, the execution order of multiple wake-up methods, and the parameter values of the wake-up methods, executes the corresponding wake-up method according to the execution order and the parameter values of the wake-up method, verifies whether the user is successfully woken up after each wake-up method is executed, in response to the user being successfully woken up, the execution of subsequent wake-up methods is stopped, in response to the user not being successfully woken up, the next wake-up method is executed according to the execution order, thereby improving the success rate of waking up the user.

[0134] Figure 7 This is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 7 shown, the entire electronic device is controlled by a bus. First, various sensors are used to collect the required data, and the data is stored, mined, and analyzed on the console, and then a personalized wake-up strategy set is designed based on the analysis conclusion. The strategy commands are sent to the wake-up devices in the house, and then the results of the wake-up are fed back and cross-checked to ensure that the user is successfully woken up.

[0135] Figure 8 Illustrates a schematic structural diagram of an electronic device entity, as Figure 8As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communications interface 820, and the memory 830 complete communication with each other through the communication bus 840. The processor 810 may call the logical instructions in the memory 830 to execute an intelligent wake-up method based on multi-modal perception. The method includes: collecting preset user wake-up decision-related data; obtaining a user wake-up strategy according to the user wake-up decision-related data; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; executing the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; wherein, after each wake-up method is executed, it is verified whether the user is successfully woken up; in response to the user being successfully woken up, the execution of the subsequent wake-up methods is stopped; in response to the user not being successfully woken up, the next wake-up method is executed according to the execution order.

[0136] In addition, when the logical instructions in the above-mentioned memory 830 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0137] On the other hand, the present application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intelligent wake-up method based on multimodal perception provided by the above-mentioned various methods. The method includes: collecting preset user wake-up decision-related data; obtaining a user wake-up strategy according to the user wake-up decision-related data; wherein the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; executing the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; wherein, after each wake-up method is executed, it is verified whether the user is successfully woken up; in response to the user being successfully woken up, the execution of the subsequent wake-up methods is stopped; in response to the user not being successfully woken up, the next wake-up method is executed according to the execution order.

[0138] In another aspect, the present application also provides a computer-readable storage medium, which includes a stored program. When the program runs, it executes the intelligent wake-up method based on multimodal perception provided by the above-mentioned various methods. The method includes: collecting preset user wake-up decision-related data; obtaining a user wake-up strategy according to the user wake-up decision-related data; wherein the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; executing the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; wherein, after each wake-up method is executed, it is verified whether the user is successfully woken up; in response to the user being successfully woken up, the execution of the subsequent wake-up methods is stopped; in response to the user not being successfully woken up, the next wake-up method is executed according to the execution order.

[0139] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0140] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. An intelligent wake-up method based on multimodal perception, characterized in that, Including: Collecting preset user wake-up decision-related data; Obtaining a user wake-up strategy based on the user wake-up decision-related data; wherein, the user wake-up strategy includes multiple wake-up methods, the execution order of the multiple wake-up methods, and the parameter values of the wake-up methods; Executing the corresponding wake-up method according to the execution order and the parameter values of the wake-up method; wherein, after each wake-up method is executed, it is verified whether the user is successfully awakened; in response to the user being successfully awakened, the execution of the subsequent wake-up methods is stopped; in response to the user not being successfully awakened, the next wake-up method is executed according to the execution order.

2. The intelligent wake-up method based on multi-modal perception according to claim 1, wherein The user wake-up decision-related data includes at least one of the following data: User physiological data, user sleep behavior data, and user sleep environment data.

3. The intelligent wake-up method based on multi-modal perception according to claim 2, characterized in that The user physiological data includes at least one of the following data: heart rate, respiratory data, blood oxygen saturation, blood pressure, body temperature, height, weight, and exercise data; The user sleep behavior data includes at least one of the following data: sleep onset time, sleep duration, sleep cycle, number of awakenings, number of turns, and mattress pressure; The sleep environment data includes at least one of the following data: temperature, humidity, light intensity, and sound.

4. The intelligent wake-up method based on multi-modal perception according to claim 2, wherein The obtaining of the user wake-up strategy according to the user wake-up decision-related data includes: Obtaining a user sleep state evaluation result according to a combination of one or more of the user physiological data, the user sleep behavior data, and the user sleep environment data; wherein, the user sleep state evaluation result includes a combination of one or more of the following evaluation results: user sleep quality evaluation result, user physiological state evaluation result, and user mental state evaluation result; Obtaining a user wake-up strategy according to the user sleep state evaluation result.

5. The intelligent wake-up method based on multimodal perception according to claim 4, wherein The obtaining of the user wake-up strategy according to the user sleep state evaluation result includes: Obtaining an initial user wake-up strategy according to the user sleep state evaluation result; Adjusting the initial user wake-up strategy using wake-up method user configuration data to obtain the user wake-up strategy.

6. The intelligent wake-up method based on multi-modal perception according to claim 4, wherein The method further includes: Giving at least one feedback message such as a health condition reminder, medical advice, exercise advice, rest advice, and diet advice according to the user sleep state evaluation result.

7. The intelligent wake-up method based on multi-modal perception according to claim 1, wherein The wake-up method is a combination of one or more of the following wake-up methods: light wake-up, sound wake-up, vibration wake-up, and temperature adjustment wake-up.

8. The intelligent wake-up method based on multimodal perception according to claim 1, wherein, The method further includes: Adapting and adjusting the user wake-up strategy according to the verification result of whether the user is successfully awakened.

9. The intelligent wake-up method based on multi-modal perception according to claim 1, characterized in that The verifying whether the user is successfully awakened includes: Comprehensively verifying whether the user is successfully awakened through at least one preset verification mode.

10. The intelligent wake-up method based on multi-modal perception according to claim 9, wherein, The comprehensively verifying whether the user is successfully awakened through at least one preset verification mode includes: Obtaining the wake-up success rate judgment result of each preset verification mode in the at least one preset verification mode; Calculating the weighted sum of the wake-up success rate judgment results of the at least one preset verification mode; In response to the weighted sum being greater than the preset success rate threshold, it is determined that the user is successfully awakened; otherwise, it is determined that the user is not successfully awakened.

11. The intelligent wake-up method based on multi-modal perception according to claim 10, characterized in that, Obtaining the wake-up success rate judgment result of each preset verification mode in the at least one preset verification mode includes: Obtaining the wake-up success rate judgment result of the corresponding preset verification mode according to the change of the preset data index corresponding to each preset verification mode.

12. The intelligent wake-up method based on multi-modal perception according to claim 9, characterized in that, The preset verification mode includes one or a combination of more than one of the following verification methods: verifying by detecting the weight change of a weight sensor placed under the mattress, verifying by detecting the user's movement, and verifying by detecting the user's bioelectrical signal.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when running, executes the method according to any one of claims 1 to 12.

14. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 12 through the computer program.

Images