Equipment energy-saving control method and device based on multi-modal data, equipment and medium

Abstract

The invention relates to the technical field of equipment control, and discloses an equipment energy-saving control method and device based on multi-modal data, equipment and a medium, and the method comprises the steps: obtaining target detection data corresponding to a target space; performing living body behavior analysis according to the target detection data to obtain a target result; and if the target result is that all the intelligent devices leave, obtaining energy-saving control configuration corresponding to the target space, and performing energy-saving control on each intelligent device in the target space according to the energy-saving control configuration. Therefore, the situation that normal use is affected by early shutdown of the equipment due to misjudgment or energy is wasted due to idle running of the equipment when no living body exists is avoided.

Description

Technical Field

[0001] The present invention relates to the field of equipment control technology, and in particular to an equipment energy-saving control method, device, equipment and medium based on multimodal data. Background Art

[0002] With the continuous growth of energy demand and the gradual enhancement of environmental awareness, solving this pain point has become increasingly important. The existing single sensor trigger or user manual operation mode is generally used, which may cause misjudgment due to reasons such as the stillness of the living body, environmental occlusion, and multiple entrances, causing the device to shut down prematurely or continue to operate when there is no living body.

[0003] In places such as home rooms, B&B rooms, shared mahjong halls, and shared conference rooms, living people often leave the room but forget to turn off the equipment, which in turn causes a lot of energy waste and increased electricity bills. Nowadays, energy demand continues to rise and environmental awareness is also increasing, so it is particularly urgent to solve this problem. At present, a single sensor trigger or user manual operation mode is usually adopted. However, due to factors such as the stationary living body, obstructions in the environment, and multiple entrances and exits in the room, misjudgment is prone to occur, which causes the device to either shut down prematurely or continue to run when there is no living person. Summary of the invention

[0004] Based on this, it is necessary to propose a device energy-saving control method, device, equipment and medium based on multimodal data to address the technical problem that the existing technology adopts a single sensor trigger or user manual operation mode, which is prone to misjudgment, causing the device to either shut down prematurely or continue to run when there is no living body.

[0005] In a first aspect, a device energy-saving control method based on multimodal data is provided, the method comprising: Obtain target detection data corresponding to the target space; Performing a live behavior analysis based on the target detection data to obtain a target result; If the target result is that all devices leave, the energy-saving control configuration corresponding to the target space is obtained, and energy-saving control is performed on each smart device in the target space according to the energy-saving control configuration.

[0006] In a second aspect, a device energy-saving control device based on multimodal data is provided, the device comprising: A data acquisition module, used to acquire target detection data corresponding to the target space; A behavior analysis module, used to perform a live behavior analysis based on the target detection data to obtain a target result; The control module is used to obtain the energy-saving control configuration corresponding to the target space if the target result is that all smart devices leave, and perform energy-saving control on each smart device in the target space according to the energy-saving control configuration.

[0007] According to a third aspect, a target device is provided, which includes a communication component, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is electrically connected to the communication component, and implements the steps of the above-mentioned device energy-saving control method based on multimodal data when executing the computer program.

[0008] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned device energy-saving control method based on multimodal data are implemented.

[0009] The energy-saving control method, device, equipment and medium of the equipment based on multimodal data of the present application first obtain the target detection data corresponding to the target space. This step can integrate information from many aspects, unlike the one-sided information of a single sensor; then, the target result is obtained by analyzing the behavior of the living body based on the target detection data. This analysis is based on multimodal data (that is, target detection data), which can avoid misjudgment caused by a single factor; when the target result is that all leave, the energy-saving control configuration corresponding to the target space is obtained and energy-saving control is performed on each smart device in the target space. Through such a process, the state of the living body in the target space can be accurately judged, and the situation that the equipment is shut down prematurely due to misjudgment, affecting normal use, or the equipment is idling and wasting energy when there is no living body, can be avoided, thereby effectively achieving energy saving, improving energy utilization efficiency, and ensuring the reasonable matching of equipment operation and living body use needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] in: Figure 1 This is an application environment diagram of a device energy-saving control method based on multimodal data in one embodiment; Figure 2 is a flow chart of a device energy-saving control method based on multimodal data in one embodiment; Figure 3is a flow chart of a device energy-saving control method based on multimodal data in one embodiment; Figure 4 is a structural block diagram of a device energy-saving control device based on multimodal data in one embodiment; Figure 5 FIG. 4 is a structural block diagram of a target device in an embodiment. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The device energy-saving control method based on multimodal data provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 The application environment includes: a target device 1, a client 3, a server 4, and smart devices (e.g., a lamp 21, an air conditioner 22, and a TV 23). The smart devices are installed in a target space 5. The target device 1 is connected to the client 3, the server 4, and each smart device in the target space 5, and the client 3 is connected to the server 4.

[0014] The target device 1 is configured to implement the device energy-saving control method based on multimodal data of the present application, specifically: obtaining target detection data corresponding to the target space; performing living behavior analysis based on the target detection data to obtain a target result; if the target result is that all have left, obtaining the energy-saving control configuration corresponding to the target space, and performing energy-saving control on each smart device in the target space according to the energy-saving control configuration.

[0015] The energy-saving control method of the equipment based on multimodal data of the present application first obtains the target detection data corresponding to the target space. This step can integrate information from many aspects, unlike the one-sided information of a single sensor; then, the target result is obtained by analyzing the behavior of the living body based on the target detection data. This analysis is based on multimodal data (that is, target detection data), which can avoid misjudgment caused by a single factor; when the target result is that all leave, the energy-saving control configuration corresponding to the target space is obtained and energy-saving control is performed on each smart device in the target space. Through such a process, the state of the living body in the target space can be accurately judged, avoiding the situation where the equipment is shut down prematurely due to misjudgment, affecting normal use, or the equipment is idling and wasting energy when there is no living body, thereby effectively achieving energy saving and improving energy utilization efficiency, while ensuring the reasonable match between equipment operation and living body use needs.

[0016] The client 3 may include but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server 4 may be implemented by an independent server or a server cluster consisting of multiple servers.

[0017] The target device 1 includes but is not limited to a personal computer, a server, an industrial computer, and an intelligent gateway.

[0018] The user can input data through the client 3 , and the client 3 transmits the data input by the user to the target device 1 and / or the server 4 ; the server 4 can also transmit data to the client 3 and the target device 1 .

[0019] Figure 1 The architecture shown is only for expressing the environment used in the device linkage control method and the display method. The target device 1, client 3, server 4 and smart device involved are only schematically displayed. The specific structure / size / shape / location / installation method can be adaptively adjusted according to actual needs, and this application does not limit it here.

[0020] The present invention is described in detail below through specific embodiments.

[0021] See also Figure 2 As shown, Figure 2 A schematic flow chart of a device energy-saving control method based on multimodal data provided by an embodiment of the present invention includes the following steps: S1: Obtain target detection data corresponding to the target space; The target space is the space where you want to perform energy-saving control.

[0022] Target detection data is the data obtained by monitoring the target space. Target detection data is multimodal data.

[0023] Target detection data includes: target networking data. Target networking data is used to describe the data related to the device connecting to Bluetooth or connecting to WiFi (mobile hotspot). The data generated when connecting to Bluetooth is called Bluetooth connection data, and the data generated when connecting to WiFi is called WiFi connection data.

[0024] In addition, target detection data may also include liveness detection data. Liveness detection data is further divided into millimeter wave detection data and / or infrared detection data. Millimeter wave detection data is data obtained by using a detector based on millimeter wave radar technology, and infrared detection data is data obtained by using a detector based on infrared technology.

[0025] Multimodal data refers to multiple types of data collected through different sensors or technical means. Multimodal data includes target networking data and liveness detection data. Target networking data is further divided into Bluetooth connection data and WiFi connection data. This type of data belongs to network communication data and is used to reflect the connection status and behavior of the device. Liveness detection data includes millimeter wave detection data and / or infrared detection data. Millimeter wave detection data is collected through millimeter wave radar technology, and infrared detection data is collected through infrared technology. This type of data belongs to sensor data and is used to detect the presence and activity status of living things.

[0026] Bluetooth connection data is usually stored in structured text or log format, such as JSON (JavaScript Object Notation), CSV (Comma-Separated Values), or XML (Extensible Markup Language). WiFi connection data is usually stored in structured text or log format, such as JSON, CSV, or XML format. Millimeter wave detection data is usually stored in binary data stream or matrix form. Infrared detection data is usually stored in the form of image or thermal imaging data, such as JPEG, PNG, or RAW format.

[0027] JPEG (Joint Photographic Experts Group) is a widely used image file format with a file extension of .jpg or .jpeg. PNG (Portable Network Graphics) is a lossless compressed bitmap graphic format. RAW is a file format for digital cameras to record raw image data, which means "unprocessed". It saves the raw data of the light signal captured by the camera sensor converted into a digital signal.

[0028] It is understandable that after data collection is completed, the collected data is preprocessed according to a preset method (for example, one or more methods of cleaning, normalization, and alignment), the usability of the data is improved through preprocessing, and the preprocessed data is used as target detection data. The preprocessing method can be selected from the existing technology according to the sensor or technical means used to collect the data, and will not be elaborated here.

[0029] In the process of multimodal data collection, the data synchronization of devices used for Bluetooth, devices used for WiFi connection and detectors is crucial. Bluetooth devices, devices used for WiFi connection and detectors are all called data terminals. If the data of each data terminal is not synchronized, it may lead to misjudgment of the on-site situation. To ensure data synchronization, a unified clock source can be used. For example, each data terminal can be equipped with a high-precision clock module so that it can collect data under the same time reference. Or use the Network Time Protocol (NTP) to connect all data terminals to a network, calibrate the time through the NTP server, and ensure the clock synchronization of each data terminal, so that when collecting data, it can accurately correspond to different modal data at the same time, thereby improving the accuracy of target detection data. Further avoid the situation where the device is shut down prematurely due to misjudgment, affecting normal use, or the device is idle and wastes energy when there is no living body.

[0030] The data detected by each data terminal for the target space and within the preset time window is determined as the target detection data corresponding to the target space. It can be understood that the current time is within the preset time window.

[0031] Optionally, detection data can be obtained from each data terminal, and then data within a preset time window (the current time is within the preset time window) can be extracted based on the obtained detection data, and the extracted data can be used as target detection data corresponding to the target space.

[0032] Optionally, detection data can also be obtained from a preset storage space, and data within a preset time window (the current time is within the preset time window) is extracted based on the acquired detection data, and the extracted data is used as target detection data corresponding to the target space.

[0033] The window length of the preset time window can be fixedly written into the program of the present application, or can be set in advance by the user.

[0034] S2: Performing a live behavior analysis based on the target detection data to obtain a target result; Optionally, based on a preset living body behavior analysis method, a living body behavior analysis is performed on the target detection data. If the result of the analysis is that all objects (that is, living bodies) in the target space have left, the target result is set to all leaving; if the result of the analysis is that there are still objects (that is, living bodies) in the target space, the target result is set to not all leaving.

[0035] Optionally, a model obtained based on artificial intelligence training can also be used to perform live behavior analysis on the target detection data, and the data obtained from the analysis can be used as the target result. For example, a classification model obtained based on artificial intelligence training (e.g., a binary classification model, a multi-classification model) can be used to classify and predict live behavior on the target detection data, and the vector element with the largest value can be obtained from the predicted vector, and the classification category corresponding to the obtained vector element can be used as the target result. It can be understood that the model structure and model training method of the classification model obtained based on artificial intelligence training can be selected from the existing technology and will not be elaborated here.

[0036] S3: If the target result is that all devices leave, then obtain the energy-saving control configuration corresponding to the target space, and perform energy-saving control on each smart device in the target space according to the energy-saving control configuration.

[0037] The energy-saving control configuration describes the associated data between the device identification and the first control configuration. The first control configuration includes: standard duration data and the device identification.

[0038] Specifically, if the target result is to leave altogether, it means that energy-saving control needs to be performed on each smart device in the target space. Therefore, the energy-saving control configuration corresponding to the target space is obtained from the preset storage space, and energy-saving control based on immediate shutdown and / or delayed shutdown is performed on each smart device in the target space according to the energy-saving control configuration.

[0039] It is understandable that if the target result is not all leaving, it means that there is no need to perform energy-saving control on each smart device in the target space. Therefore, the remaining process of the device energy-saving control method based on multimodal data triggered by the target detection data is terminated.

[0040] This embodiment first obtains the target detection data corresponding to the target space. This step can integrate information from many aspects, unlike a single sensor that provides one-sided information. Then, the target result is obtained by analyzing the behavior of the living body based on the target detection data. This analysis is based on multimodal data (that is, target detection data), which can avoid misjudgment caused by a single factor. When the target result is that all people have left, the energy-saving control configuration corresponding to the target space is obtained and energy-saving control is performed on each smart device in the target space. Through such a process, the state of the living body in the target space can be accurately judged, avoiding the situation where the equipment is shut down prematurely due to misjudgment, affecting normal use, or the equipment is idle and wastes energy when there is no living body, thereby effectively achieving energy saving and improving energy utilization efficiency, while ensuring a reasonable match between equipment operation and living body use needs.

[0041] In one embodiment, the step of performing a live behavior analysis based on the target detection data to obtain a target result includes: S21: performing signal strength change trend analysis on target networking data in the target detection data to obtain a trend analysis result, wherein the target networking data includes: Bluetooth connection data and / or WiFi connection data; Specifically, a signal strength curve is generated for the target networking data in the target detection data, and a signal strength change trend analysis (for example, slope analysis) is performed on the signal strength curve. If the data obtained by the analysis is within the change trend range corresponding to departure, and the current target networking data does not have data on device networking of living organisms (this means that all living organisms connected to the network through the device have left the site), then the trend analysis result is determined to be the departure of all living organisms. If the data obtained by the analysis is within the change trend range corresponding to signal fluctuation, then the trend analysis result is determined to be signal fluctuation.

[0042] S22: If the trend analysis result is that all the living bodies have left, determining the target result as all have left; Specifically, if the trend analysis result is that all living bodies have left, this means that there are no living bodies in the target space and energy-saving control is required. Therefore, the target result is determined as all living bodies have left.

[0043] S23: If the trend analysis result is a signal fluctuation, the target result is determined as not all leaving.

[0044] Specifically, if the trend analysis result is a signal fluctuation, it means that there may be living bodies in the target space and energy-saving control is not required. Therefore, the target result is determined as not all living bodies have left.

[0045] It is understandable that the analysis logic of step S21 to step S23 is the preset living body behavior analysis method.

[0046] This embodiment performs a signal strength change trend analysis on the target networking data in the target detection data to obtain a trend analysis result, and the target networking data includes: Bluetooth connection data and / or WiFi connection data. When the trend analysis result is that all living things have left the scene, the target result is determined as all have left, which enables the state of living things in the venue to be accurately judged in the monitoring scenario, avoiding the waste of energy caused by idling of equipment when there are no living things due to misjudgment; and when the trend analysis result is a signal fluctuation, the target result is determined as not all have left. This judgment method improves the accuracy of the judgment of the presence of living things, avoiding premature shutdown of equipment due to misjudgment that affects normal use.

[0047] In one embodiment, if the trend analysis result is that all the living bodies have left, the step of determining the target result as all leaving includes: S221: If the trend analysis result is that all living bodies have left the scene, performing liveness detection according to the liveness detection data in the target detection data to obtain a liveness detection result, wherein the liveness detection data includes: millimeter wave detection data and / or infrared detection data; Specifically, if the trend analysis result is that all living things have left the scene, all living things connected to the network through devices have left the target space, but there are other living things (people, animals) in the target space that are not connected to the network through devices, if energy-saving control is directly performed on each smart device in the target space, it may affect the normal use of smart devices by other living things in the target space that are not connected to the network through devices. Therefore, the liveness detection data in the target detection data is input into the pre-trained liveness detection model for classification prediction of the liveness, and the vector element with the largest value is obtained from the predicted vector, and the classification category corresponding to the obtained vector element is used as the liveness detection result.

[0048] The liveness detection result describes the type of liveness, such as human, cat, or dog.

[0049] S222: If the liveness detection result is within the target range, determining the target result as not completely leaving; The target range includes one or more of human body, cat and dog.

[0050] Specifically, if the liveness detection result is within the target range, it means that there are still living bodies in the target space that are not connected to the Internet through devices and need to continue using smart devices. Therefore, the target result is determined as not all leaving.

[0051] S223: If the liveness detection result is outside the target range, the target result is determined as all out.

[0052] Specifically, if the liveness detection result is outside the target range, it means that there are no living beings in the target space who are connected to the Internet and need to continue using smart devices, and there are no living beings who are not connected to the Internet through devices and need to continue using smart devices. Therefore, the target result is determined as not all leaving.

[0053] In many scenarios, there may be misjudgments based solely on the trend analysis of the signal strength of network data, and adding a liveness detection link can provide a second confirmation. If the liveness detection result is within the target range, the target result is determined as not all left, avoiding misjudgments caused by factors such as signal interference, and avoiding premature shutdown of equipment due to misjudgments that affect normal use; if the liveness detection result is outside the target range, the target result is determined as all left, avoiding idling of equipment and wasting energy when there is no liveness due to misjudgments, which helps to reasonably perform energy-saving control.

[0054] In one embodiment, the target networking data includes: registration data and visitor data, the registration data includes: Bluetooth connection data and / or WiFi connection data, the visitor data includes: Bluetooth connection data and / or WiFi connection data, the type of object corresponding to the registration data is a registered user, the type of object corresponding to the visitor data is an unregistered user, and the object corresponding to the visitor data currently stays in the target space for a period of time that exceeds a first period of time.

[0055] In this embodiment, for registered users, incorporating Bluetooth connection data and / or WiFi connection data into target networking data helps to quickly obtain relevant information, so that energy-saving control can be implemented on registered users in a more timely manner. In terms of visitor management, different permissions are granted based on the length of visitor stay, which can not only handle the impact of visitors who stay for more than the first time on energy-saving control, but also prevent visitors who stay for less than the first time from causing misjudgment of energy-saving control, thereby ensuring the accuracy and effectiveness of energy-saving control.

[0056] In one embodiment, the step of performing energy-saving control on each smart device in the target space according to the energy-saving control configuration includes: S31: Playing a delayed reminder voice according to the real-time status of each smart device in the target space; Specifically, the real-time status feedback of each smart device in the target space is obtained, and a delayed reminder voice can be generated for each smart device whose real-time status is turned on; when the number of all smart devices whose real-time status is turned on is greater than 1, one or more delayed reminder voices can also be generated for all smart devices whose real-time status is turned on.

[0057] The real-time status is: open or closed.

[0058] S32: acquiring a reply voice within a second time period after playing the delayed reminder voice, and obtaining an acquisition result; Specifically, a reply voice is obtained within a second time period after playing the delayed reminder voice. If the reply voice is obtained, the acquisition result is determined to be successful; if the reply voice is not obtained, the acquisition result is determined to be a failure.

[0059] S33: If the acquisition result is failure, then performing energy-saving control of delayed shutdown of each of the smart devices in the target space according to the standard duration data in the energy-saving control configuration; Specifically, if the acquisition result is a failure, it means that the reply voice corresponding to the delayed reminder voice played is not obtained, and energy-saving control can be performed according to the preset mode. Therefore, according to the standard duration data in the energy-saving control configuration, energy-saving control of delayed shutdown is performed on each of the smart devices in the target space.

[0060] S34: If the acquisition result is successful, performing delay duration recognition according to the reply voice to obtain a recognition result; Specifically, if the acquisition result is successful, it means that the reply voice corresponding to the delayed reminder voice played is obtained, and energy saving control can be performed according to the user's needs (the demand can be extracted through the reply voice). Therefore, the delay duration is recognized according to the reply voice to obtain the recognition result.

[0061] Among them, the reply voice is converted into text, and intention recognition and time recognition are performed based on the converted text. If the intention recognition is delayed, the conclusion of the recognition result is determined to be delayed; if the intention recognition is not delayed, the conclusion of the recognition result is determined to be not delayed; if the intention recognition fails (that is, the reply voice does not mention delay or no delay), the conclusion of the recognition result is determined to be delayed.

[0062] Speech-to-text conversion can use a model trained based on Automatic Speech Recognition (ASR) technology.

[0063] The intent recognition method can select an intent recognition model from the prior art. The model structure and training method of the intent recognition model can be selected from the prior art, which will not be described in detail here.

[0064] When performing time recognition based on the text obtained by converting the reply voice, recognition can be performed based on time keywords in the text. For example, if clear time expressions such as "point", "hour", "minute", and "point" appear in the text, these keywords can be directly extracted to determine the time information.

[0065] S35: If the conclusion of the recognition result is delay and the delay duration of the recognition result is not empty, then, according to the delay duration of the recognition result, energy-saving control of delay shutdown is performed on each of the smart devices in the target space; Specifically, if the conclusion of the recognition result is delay and the delay duration of the recognition result is not empty, it means that the user has a clear delay duration. Therefore, according to the delay duration of the recognition result, energy-saving control of delayed shutdown is performed on each of the smart devices in the target space.

[0066] S36: If the conclusion of the recognition result is delay and the delay duration of the recognition result is empty, then, according to the standard duration data in the energy-saving control configuration, perform energy-saving control of delaying shutdown of each of the smart devices in the target space; Specifically, if the conclusion of the recognition result is delay and the delay duration of the recognition result is empty, it means that the user needs a delay but there is no clear delay duration. Energy-saving control can be performed according to the preset mode. Therefore, according to the standard duration data in the energy-saving control configuration, energy-saving control of delayed shutdown is performed on each of the smart devices in the target space.

[0067] S37: If the conclusion of the recognition result is no delay, then according to the energy-saving control configuration, energy-saving control is performed on each of the smart devices in the target space to immediately shut down.

[0068] Specifically, if the conclusion of the recognition result is no delay, it means that the user does not need a delay and wants to turn off the smart device immediately. Therefore, according to the energy-saving control configuration, energy-saving control is performed on each of the smart devices in the target space to shut down immediately.

[0069] This embodiment first plays the delayed reminder voice and then obtains the reply voice mechanism, so that the application can interact with the user and make decisions based on the user's feedback, which reflects intelligence and humanity. When the acquisition result fails, the smart device is delayed and closed according to the standard duration data in the energy-saving control configuration to ensure that the device can operate according to the established energy-saving strategy without user feedback, avoiding unnecessary long-term opening of the device and causing energy waste. When the acquisition result is successful, the reply voice is further identified for the delay duration, and different control measures are taken according to the different conclusions of the recognition result. For example, when the conclusion is delayed and the delay duration is not empty, the smart device is delayed and closed according to the duration. When the conclusion is delayed but the delay duration is empty, it is controlled according to the standard duration data. If the conclusion is no delay, the smart device is immediately turned off. This series of operations accurately controls the energy saving of smart devices according to the user's intentions and actual conditions, improves the energy utilization efficiency, and also improves the user's operating experience of smart devices, achieving a good balance between energy saving and user needs.

[0070] In one embodiment, the step of acquiring a reply voice within a second time period after playing the delayed reminder voice to obtain an acquisition result includes: S321: obtaining a reply voice within a second time period after playing the delayed reminder voice, and if the reply voice is not obtained, determining the acquisition result as a failure; if the reply voice is obtained, using the reply voice as the voice to be analyzed; Specifically, the time when the delayed reminder voice is played is taken as the start time, and the reply voice is obtained within the second time period.

[0071] S322: extracting voiceprint from the speech to be analyzed to obtain the voiceprint to be analyzed; There are mainly the following specific ways to extract voiceprints from the speech to be analyzed. First, the speech signal needs to be preprocessed, including removing interference factors such as noise. This can be done by using technical means such as filtering to make the speech signal purer and more conducive to subsequent analysis. Then, the speech signal is segmented into appropriate frames, usually according to a certain time length, so that it can be analyzed frame by frame. Next, the acoustic features of each frame are extracted. These features may include Mel-frequency cepstral coefficients (MFCC), etc. These features can effectively characterize the characteristics of the speech. After extracting enough acoustic features, these features are combined through a specific algorithm to form a voiceprint model. This algorithm may involve statistical analysis, machine learning and other technologies, such as using algorithms such as Gaussian mixture models (GMM) or deep neural networks (DNN) to build voiceprint models. Finally, representative voiceprint feature vectors are extracted from the constructed voiceprint model. These vectors constitute the voiceprint of the speech to be analyzed, thereby realizing voiceprint extraction.

[0072] S323: Obtaining a standard voiceprint corresponding to the target result as a target voiceprint; Specifically, the standard voiceprint corresponding to the target result is a preset voiceprint. The standard voiceprint corresponding to the target result can be obtained from a preset storage space, and the obtained standard voiceprint is used as the target voiceprint.

[0073] S324: Calculate the similarity between the voiceprint to be analyzed and the target voiceprint to obtain a target similarity; Optionally, a cosine similarity calculation method is used to calculate the cosine similarity between the voiceprint to be analyzed and the target voiceprint, and the calculated cosine similarity is used as the target similarity.

[0074] S325: If the target similarity is within a preset similarity range, the to-be-analyzed voice is used as the reply voice corresponding to the delayed reminder voice, and the acquisition result is determined as successful; Specifically, if the target similarity is within the preset similarity range, it means that the reply voice meets the requirements. Therefore, the voice to be analyzed is used as the reply voice corresponding to the delayed reminder voice, and the acquisition result is determined to be successful.

[0075] S326: If the target similarity is outside the preset similarity range, the acquisition result is determined as a failure.

[0076] Specifically, if the target similarity is outside the preset similarity range, it means that the reply voice is not a voice that meets the requirements, and therefore, the acquisition result is determined to be a failure.

[0077] In another embodiment of the application, step S326 can be replaced by: if the target similarity is outside the preset similarity range, if the interval between the current time and the time when the delayed reminder voice is played is less than the second duration, then jump to the step of obtaining the reply voice within the second duration after playing the delayed reminder voice and continue to execute, if the interval between the current time and the time when the delayed reminder voice is played is greater than or equal to the second duration, then the acquisition result is determined to be a failure. If the interval duration is less than the second duration, jump back to the step of obtaining the reply voice and continue to execute, which makes the present application have a certain fault tolerance, giving the opportunity to obtain the valid reply voice again within the specified time, avoiding misjudgment of the result due to occasional interference or inaccurate initial judgment. When the interval is greater than or equal to the second duration, the acquisition result is determined to be a failure, which ensures the timeliness of the entire process, and gives a clear result judgment after a reasonable waiting time, thereby improving the accuracy and reliability of the present application in the interactive process.

[0078] This embodiment first ensures the timeliness and certainty of the operation of this application by setting the reply voice to be obtained within the second time period and clarifying the judgment criteria for the acquisition results. Then, the voiceprint to be analyzed is extracted and the similarity with the target voiceprint is calculated. This process realizes the accurate verification of the source of the reply voice. If the target similarity is within the preset range, the acquisition result is determined to be successful, and if it is outside the range, it is a failure. This mechanism effectively avoids the situation of misidentification of the reply voice and improves the accuracy of the interaction of the entire application. Overall, these operations can accurately judge the validity of the reply voice, thereby providing a reliable basis for subsequent energy-saving control of smart devices in the target space based on the acquisition results. It helps to improve the stability, reliability and accuracy of intelligent decision-making of this application, and ensure that the entire application runs efficiently according to the expected logic.

[0079] In one embodiment, the method further comprises: S41: Obtaining a configuration generation signal; Specifically, the configuration generation signal input by the user can be obtained, the configuration generation signal input by a third-party application can be obtained, and the configuration generation signal can be actively triggered by the program of the present application according to preset conditions.

[0080] Optionally, the preset condition is: a time list, that is, a configuration generation signal is triggered once at each time point in the time list.

[0081] Optionally, the preset condition is: time interval, that is, triggering a configuration generation signal once at each start time of the time interval.

[0082] S42: In response to the configuration generation signal, obtaining historical average delay data, current ambient temperature data and device type data of the target space; Specifically, when the configuration generation signal is obtained, the historical average delay data, current ambient temperature data and device type data of the target space are obtained from the preset storage space.

[0083] Optionally, the historical average delay data is the average value of all actual delay durations of each smart device in history.

[0084] Optionally, the historical average delay data is the average value of the actual delay duration of each smart device in each time period in history.

[0085] The current ambient temperature data is the real-time temperature outside the target space. A temperature sensor can be installed on the outside of the wall of the target space to obtain the temperature detected by the temperature sensor in real time as the current ambient temperature data.

[0086] The device type data describes the type of smart devices in the target space. For example, the device type data includes one or more of the air conditioner and lamp types.

[0087] S43: Generate the energy-saving control configuration corresponding to the target space according to the current time, the historical average delay data, the current ambient temperature data and the equipment type data.

[0088] First, the current time is a key factor. Different time periods may correspond to different energy demand patterns. For example, the frequency of use and energy consumption requirements for equipment during the day and at night may be very different. Based on the historical average delay data, we can understand the response delay of the equipment in the target space under similar circumstances in the past, which helps to predict the time required for the equipment to switch its operating state in the current environment, so as to reasonably arrange the rhythm of energy-saving control. The current ambient temperature data also has an important impact on the energy-saving control configuration. The device type data clarifies the type of equipment in the target space. Different types of equipment have different energy-saving strategies. For example, the energy-saving control methods of lighting equipment and large appliances are very different. Taking these four aspects of data into consideration, through specific algorithms or rules, the most suitable energy-saving control configuration for the target space can be generated to achieve the goal of saving energy to the greatest extent while meeting the needs.

[0089] Optionally, the current time is obtained, and the energy-saving control configuration corresponding to the target space is determined by a table lookup method based on the current time, the historical average delay data, the current ambient temperature data and the equipment type data.

[0090] Optionally, the current time, the historical average delay data, the current ambient temperature data and the equipment type data are concatenated and input into a preset solution prediction model for classification prediction, and the vector element with the largest value is obtained from the predicted vector. The control solution corresponding to the classification category (describing the solution identifier) ​​corresponding to the obtained vector element is used as the energy-saving control configuration corresponding to the target space.

[0091] The scheme identifier may be data that uniquely identifies a control scheme, such as a scheme name and a scheme ID.

[0092] The scenario prediction model is a pre-trained multi-classification model.

[0093] This embodiment obtains a configuration generation signal as the starting point of the entire process, so that the application can respond to the demand in a timely manner. After receiving the signal, the historical average delay data, current ambient temperature data and device type data of the target space are obtained. This process integrates many aspects of key information related to the target space. Among them, the historical average delay data reflects the historical laws of the operation of the target space equipment, the current ambient temperature data reflects the potential impact of the external environment on the equipment, and the equipment type data clarifies the characteristics of the equipment. Based on these data and combined with the current time, the energy-saving control configuration is generated, which can make the energy-saving control more accurate and intelligent.

[0094] In one embodiment, the energy-saving control configuration includes: air conditioning energy-saving configuration and linkage closing configuration, and the linkage closing configuration includes: one or more of window closing configuration, door closing configuration, lamp closing configuration and curtain closing configuration.

[0095] The air conditioner energy saving configuration describes the first control configuration for energy saving control of the air conditioner.

[0096] The linkage shutdown configuration describes the first control configuration for energy-saving control of smart devices other than air conditioners.

[0097] The window closing configuration describes a first control configuration for closing a window. The door closing configuration describes a first control configuration for closing a door. The lamp closing configuration describes a first control configuration for closing a lamp. The curtain closing configuration describes a first control configuration for closing a curtain.

[0098] The air conditioning energy-saving configuration of this embodiment directly acts on the air conditioning equipment, and can perform energy-saving control on the air conditioning to avoid unnecessary energy waste. Among the window closing configuration, door closing configuration, lamp closing configuration and curtain closing configuration in the linkage closing configuration, one or more of the lamp closing configuration can avoid unnecessary power waste, the curtain closing configuration and window closing configuration can not only effectively block the sunlight, but also avoid outside vision, and the window closing configuration and door closing configuration can prevent rain from drifting into the target space.

[0099] See also Figure 3 As shown, in one embodiment, after the step of performing the live behavior analysis according to the target detection data to obtain the target result, the step further includes: S5: If the target result is the first entry, obtain the start control configuration corresponding to the target space, and perform start control on each smart device in the target space according to the start control configuration.

[0100] A signal strength curve is generated for the target networking data in the target detection data, and a signal strength change trend analysis (for example, slope analysis) is performed on the signal strength curve. If the data obtained by the analysis is within the change trend range corresponding to the entry, and the target networking data is data of only one user (that is, only one user's device is currently connected to the Internet), then the target result is determined to be the first entry.

[0101] The start control configuration describes the associated data between the device identification and the second control configuration. The second control configuration includes: the device identification and the start control parameter. For example, when the device type corresponding to the device identification is air conditioning, the start control parameter is temperature.

[0102] Specifically, if the target result is the first entry, it means that each smart device in the target space needs to be turned on. Therefore, the turn-on control configuration corresponding to the target space is obtained from the preset storage space, and the turn-on control of each smart device in the target space is performed according to the turn-on control configuration.

[0103] This embodiment can intelligently and accurately control the opening of each smart device in the target space, which not only avoids the accidental opening of the device, but also creates a suitable space environment for entrants according to specific needs and settings, greatly improving the intelligence level of smart device management and the rationality of energy utilization, improving the user experience while also contributing to the efficiency of space equipment management.

[0104] In one embodiment, the opening control configuration includes: ventilation opening configuration and linkage opening configuration, the ventilation opening configuration includes: one or more of air conditioning opening configuration, circulation fan control configuration, curtain control configuration and window control configuration, the linkage opening configuration includes: lighting opening configuration, humidifier opening configuration and dryer opening configuration.

[0105] The air conditioner on configuration describes a second control configuration for turning on the air conditioner.

[0106] The circulation fan control configuration describes a second control configuration for turning the circulation fan on or on first and then off. The curtain control configuration describes a second control configuration for turning the curtains on sequentially or on first and then off. The window control configuration describes a second control configuration for turning the windows on sequentially or on first and then off.

[0107] The lamp on configuration describes a second control configuration for turning on the lamp. The humidifier on configuration describes a second control configuration for turning on the humidifier. The dryer on configuration describes a second control configuration for turning on the dryer.

[0108] The ventilation opening configuration of this embodiment covers one or more of the air conditioner, circulating fan, curtain and window control configuration, which enables effective coordinated control of the ventilation-related equipment in the space when the target result is the first entry. For example, according to the preset ventilation opening configuration, it can be intelligently decided whether to turn on the air conditioner to adjust the temperature and air circulation, or to open the windows and curtains for natural ventilation, or to start the circulating fan to assist air flow, which helps to create a comfortable indoor air environment and optimize energy utilization. The linkage opening configuration includes the lamp, humidifier and dryer opening configuration, which can realize the coordinated work of related equipment according to actual needs. When a living body enters the space, according to the linkage opening configuration, the lamp can provide appropriate lighting, and the humidifier or dryer automatically adjusts according to the humidity conditions of the space to ensure the comfort of the environment. This classification and configuration method improves the refinement of intelligent device management, improves the coordination of device control, and creates a more intelligent, comfortable and energy-saving space use experience for users.

[0109] See also Figure 4 As shown, in one embodiment, a device energy-saving control device based on multimodal data is provided, and the device includes: The data acquisition module 801 is used to acquire target detection data corresponding to the target space; A behavior analysis module 802 is used to perform a live behavior analysis based on the target detection data to obtain a target result; The control module 803 is used to obtain the energy-saving control configuration corresponding to the target space if the target result is that all smart devices leave, and perform energy-saving control on each smart device in the target space according to the energy-saving control configuration.

[0110] This embodiment first obtains the target detection data corresponding to the target space. This step can integrate information from many aspects, unlike a single sensor that provides one-sided information. Then, the target result is obtained by analyzing the behavior of the living body based on the target detection data. This analysis is based on multimodal data (that is, target detection data), which can avoid misjudgment caused by a single factor. When the target result is that all people have left, the energy-saving control configuration corresponding to the target space is obtained and energy-saving control is performed on each smart device in the target space. Through such a process, the state of the living body in the target space can be accurately judged, avoiding the situation where the equipment is shut down prematurely due to misjudgment, affecting normal use, or the equipment is idle and wastes energy when there is no living body, thereby effectively achieving energy saving and improving energy utilization efficiency, while ensuring a reasonable match between equipment operation and living body use needs.

[0111] In one embodiment, the step of performing live behavior analysis according to the target detection data to obtain a target result in the behavior analysis module 802 includes: Performing a signal strength change trend analysis on target networking data in the target detection data to obtain a trend analysis result, wherein the target networking data includes: Bluetooth connection data and / or WiFi connection data; If the trend analysis result is that all the living bodies have left the scene, then the target result is determined as all have left; If the trend analysis result is a signal fluctuation, the target result is determined as not all leaving.

[0112] In one embodiment, if the trend analysis result is that all the living bodies have left the scene, the step of determining the target result as all leaving in the behavior analysis module 802 includes: If the trend analysis result is that all living bodies have left the scene, liveness detection is performed according to the liveness detection data in the target detection data to obtain a liveness detection result, wherein the liveness detection data includes: millimeter wave detection data and / or infrared detection data; If the liveness detection result is within the target range, determining the target result as not completely leaving; If the liveness detection result is outside the target range, the target result is determined to be completely out of the target range.

[0113] In one embodiment, the target networking data includes: registration data and visitor data, the registration data includes: Bluetooth connection data and / or WiFi connection data, the visitor data includes: Bluetooth connection data and / or WiFi connection data, the type of object corresponding to the registration data is a registered user, the type of object corresponding to the visitor data is an unregistered user, and the object corresponding to the visitor data currently stays in the target space for a period of time that exceeds a first period of time.

[0114] In one embodiment, the step of performing energy-saving control on each smart device in the target space according to the energy-saving control configuration in the control module 803 includes: Play a delayed reminder voice according to the real-time status of each smart device in the target space; Acquire a reply voice within a second time period after playing the delayed reminder voice, and obtain an acquisition result; If the acquisition result is failure, then performing energy-saving control of delayed shutdown of each of the smart devices in the target space according to the standard duration data in the energy-saving control configuration; If the acquisition result is successful, then performing delay duration recognition according to the reply voice to obtain a recognition result; If the conclusion of the recognition result is delay and the delay duration of the recognition result is not empty, then, according to the delay duration of the recognition result, energy-saving control of delay shutdown is performed on each of the smart devices in the target space; If the conclusion of the recognition result is delay and the delay duration of the recognition result is empty, then the energy-saving control of delaying shutdown of each of the smart devices in the target space is performed according to the standard duration data in the energy-saving control configuration; If the conclusion of the recognition result is no delay, then according to the energy-saving control configuration, energy-saving control is performed on each of the smart devices in the target space to immediately shut down.

[0115] In one embodiment, the step of acquiring the reply voice within the second time period after playing the delayed reminder voice and obtaining the acquisition result in the control module 803 includes: Acquire a reply voice within a second time period after playing the delayed reminder voice, if the reply voice is not acquired, determine the acquisition result as failure, if the reply voice is acquired, use the reply voice as the voice to be analyzed; Extracting voiceprints from the speech to be analyzed to obtain voiceprints to be analyzed; Obtaining a standard voiceprint corresponding to the target result as the target voiceprint; Calculating the similarity between the voiceprint to be analyzed and the target voiceprint to obtain a target similarity; If the target similarity is within a preset similarity range, the speech to be analyzed is used as the reply speech corresponding to the delayed reminder speech, and the acquisition result is determined as successful; If the target similarity is outside the preset similarity range, the acquisition result is determined to be a failure.

[0116] In one embodiment, the apparatus further comprises: a configuration generation module, wherein the configuration generation module is configured to: Get the configuration generation signal; In response to the configuration generation signal, historical average delay data, current ambient temperature data and device type data of the target space are obtained; The energy-saving control configuration corresponding to the target space is generated according to the current time, the historical average delay data, the current ambient temperature data and the equipment type data.

[0117] In one embodiment, the energy-saving control configuration includes: air conditioning energy-saving configuration and linkage closing configuration, and the linkage closing configuration includes: one or more of window closing configuration, door closing configuration, lamp closing configuration and curtain closing configuration.

[0118] In one embodiment, the device further comprises: an opening module, wherein the opening module is configured to: If the target result is the first entry, the start control configuration corresponding to the target space is obtained, and the start control of each smart device in the target space is performed according to the start control configuration.

[0119] In one embodiment, the opening control configuration includes: ventilation opening configuration and linkage opening configuration, the ventilation opening configuration includes: one or more of air conditioning opening configuration, circulation fan control configuration, curtain control configuration and window control configuration, the linkage opening configuration includes: lighting opening configuration, humidifier opening configuration and dryer opening configuration.

[0120] See also Figure 5 As shown, in one embodiment, a target device 1 is provided, the target device 1 includes a communication component 13, a memory 11, a processor 12, and a computer program stored in the memory 11 and executable on the processor 12, the processor 12 is electrically connected to the communication component 13, and the processor 12 implements the following steps when executing the computer program: Obtain target detection data corresponding to the target space; Performing a live behavior analysis based on the target detection data to obtain a target result; If the target result is that all devices leave, the energy-saving control configuration corresponding to the target space is obtained, and energy-saving control is performed on each smart device in the target space according to the energy-saving control configuration.

[0121] This embodiment first obtains the target detection data corresponding to the target space. This step can integrate information from many aspects, unlike a single sensor that provides one-sided information. Then, the target result is obtained by analyzing the behavior of the living body based on the target detection data. This analysis is based on multimodal data (that is, target detection data), which can avoid misjudgment caused by a single factor. When the target result is that all people have left, the energy-saving control configuration corresponding to the target space is obtained and energy-saving control is performed on each smart device in the target space. Through such a process, the state of the living body in the target space can be accurately judged, avoiding the situation where the equipment is shut down prematurely due to misjudgment, affecting normal use, or the equipment is idle and wastes energy when there is no living body, thereby effectively achieving energy saving and improving energy utilization efficiency, while ensuring a reasonable match between equipment operation and living body use needs.

[0122] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented: Obtain target detection data corresponding to the target space; Performing a live behavior analysis based on the target detection data to obtain a target result; If the target result is that all devices leave, the energy-saving control configuration corresponding to the target space is obtained, and energy-saving control is performed on each smart device in the target space according to the energy-saving control configuration.

[0123] This embodiment first obtains the target detection data corresponding to the target space. This step can integrate information from many aspects, unlike a single sensor that provides one-sided information. Then, the target result is obtained by analyzing the behavior of the living body based on the target detection data. This analysis is based on multimodal data (that is, target detection data), which can avoid misjudgment caused by a single factor. When the target result is that all people have left, the energy-saving control configuration corresponding to the target space is obtained and energy-saving control is performed on each smart device in the target space. Through such a process, the state of the living body in the target space can be accurately judged, avoiding the situation where the equipment is shut down prematurely due to misjudgment, affecting normal use, or the equipment is idle and wastes energy when there is no living body, thereby effectively achieving energy saving and improving energy utilization efficiency, while ensuring a reasonable match between equipment operation and living body use needs.

[0124] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or the target device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0125] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0126] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0127] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A device energy-saving control method based on multimodal data, characterized in that: The method comprises: Obtain target detection data corresponding to the target space; Performing a live behavior analysis based on the target detection data to obtain a target result; If the target result is that all devices leave, the energy-saving control configuration corresponding to the target space is obtained, and energy-saving control is performed on each smart device in the target space according to the energy-saving control configuration.

2. The device energy-saving control method based on multimodal data according to claim 1, characterized in that: The step of performing a live behavior analysis based on the target detection data to obtain a target result includes: Performing a signal strength change trend analysis on target networking data in the target detection data to obtain a trend analysis result, wherein the target networking data includes: Bluetooth connection data and / or WiFi connection data; If the trend analysis result is that all the living bodies have left the scene, then the target result is determined as all have left; If the trend analysis result is a signal fluctuation, the target result is determined as not all leaving.

3. The device energy-saving control method based on multimodal data according to claim 2, characterized in that: If the trend analysis result is that all the living bodies have left the scene, the step of determining the target result as all the living bodies have left the scene comprises: If the trend analysis result is that all living bodies have left the scene, liveness detection is performed according to the liveness detection data in the target detection data to obtain a liveness detection result, wherein the liveness detection data includes: millimeter wave detection data and / or infrared detection data; If the liveness detection result is within the target range, determining the target result as not completely leaving; If the liveness detection result is outside the target range, the target result is determined to be completely out of the target range.

4. The device energy-saving control method based on multimodal data according to claim 2, characterized in that: The target networking data includes: registration data and visitor data, the registration data includes: Bluetooth connection data and / or WiFi connection data, the visitor data includes: Bluetooth connection data and / or WiFi connection data, the type of object corresponding to the registration data is a registered user, the type of object corresponding to the visitor data is an unregistered user, and the object corresponding to the visitor data currently stays in the target space for a period of time that exceeds a first period of time.

5. The device energy-saving control method based on multimodal data according to claim 1, characterized in that: The step of performing energy-saving control on each smart device in the target space according to the energy-saving control configuration includes: Play a delayed reminder voice according to the real-time status of each smart device in the target space; Acquire a reply voice within a second time period after playing the delayed reminder voice, and obtain an acquisition result; If the acquisition result is failure, then performing energy-saving control of delayed shutdown of each of the smart devices in the target space according to the standard duration data in the energy-saving control configuration; If the acquisition result is successful, then performing delay duration recognition according to the reply voice to obtain a recognition result; If the conclusion of the recognition result is delay and the delay duration of the recognition result is not empty, then, according to the delay duration of the recognition result, energy-saving control of delay shutdown is performed on each of the smart devices in the target space; If the conclusion of the recognition result is delay and the delay duration of the recognition result is empty, then the energy-saving control of delaying shutdown of each of the smart devices in the target space is performed according to the standard duration data in the energy-saving control configuration; If the conclusion of the recognition result is no delay, then according to the energy-saving control configuration, energy-saving control is performed on each of the smart devices in the target space to immediately shut down.

6. The device energy-saving control method based on multimodal data according to claim 5, characterized in that: The step of acquiring a reply voice within a second time period after playing the delayed reminder voice and obtaining an acquisition result comprises: Acquire a reply voice within a second time period after playing the delayed reminder voice, if the reply voice is not acquired, determine the acquisition result as failure, if the reply voice is acquired, use the reply voice as the voice to be analyzed; Extracting voiceprints from the speech to be analyzed to obtain voiceprints to be analyzed; Obtaining a standard voiceprint corresponding to the target result as the target voiceprint; Calculating the similarity between the voiceprint to be analyzed and the target voiceprint to obtain a target similarity; If the target similarity is within a preset similarity range, the speech to be analyzed is used as the reply speech corresponding to the delayed reminder speech, and the acquisition result is determined as successful; If the target similarity is outside the preset similarity range, the acquisition result is determined to be a failure.

7. The device energy-saving control method based on multimodal data according to claim 1, characterized in that: The method further comprises: Get the configuration generation signal; In response to the configuration generation signal, historical average delay data, current ambient temperature data and device type data of the target space are obtained; The energy-saving control configuration corresponding to the target space is generated according to the current time, the historical average delay data, the current ambient temperature data and the equipment type data.

8. The device energy-saving control method based on multimodal data according to claim 1, characterized in that: The energy-saving control configuration includes: air-conditioning energy-saving configuration and linkage closing configuration, and the linkage closing configuration includes: one or more of window closing configuration, door closing configuration, lamp closing configuration and curtain closing configuration.

9. The device energy-saving control method based on multimodal data according to claim 1, characterized in that: After the step of performing the live behavior analysis according to the target detection data to obtain the target result, the method further includes: If the target result is the first entry, the start control configuration corresponding to the target space is obtained, and the start control of each smart device in the target space is performed according to the start control configuration.

10. The device energy-saving control method based on multimodal data according to claim 9, characterized in that: The opening control configuration includes: ventilation opening configuration and linkage opening configuration, the ventilation opening configuration includes: one or more of air conditioning opening configuration, circulation fan control configuration, curtain control configuration and window control configuration, the linkage opening configuration includes: lighting opening configuration, humidifier opening configuration and dryer opening configuration.

11. A device energy-saving control device based on multimodal data, characterized in that: The device comprises: A data acquisition module, used to acquire target detection data corresponding to the target space; A behavior analysis module, used to perform a live behavior analysis based on the target detection data to obtain a target result; The control module is used to obtain the energy-saving control configuration corresponding to the target space if the target result is that all smart devices leave, and perform energy-saving control on each smart device in the target space according to the energy-saving control configuration.

12. A target device, characterized in that: The target device includes a communication component, a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is electrically connected to the communication component. When the processor executes the computer program, the steps of the device energy-saving control method based on multimodal data as described in any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the device energy-saving control method based on multimodal data as claimed in any one of claims 1 to 10 are implemented.

Images