Information prompting method and device, storage medium and electronic equipment
Patent Information
- Application Number
- CN202411407778.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-14
Smart Images
Figure CN119440326A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of smart home technology, and in particular to an information prompting method, device, storage medium and electronic device. Background Art
[0002] With the improvement of people's living standards and the change of consumption concepts, smart home products have gradually become an important choice for consumers to pursue high-quality life, and their market demand continues to grow. Existing smart home devices are not limited to the traditional single function realization, but rely on advanced technical means, such as fingerprint recognition, password input and biometric recognition and other diversified interaction methods to improve the convenience and security of user experience, effectively solve the inconvenience of using traditional home appliances, and fully meet the user's use needs in a variety of scenarios. However, in recent years, with the continuous maturity of the smart home market, consumers' functional requirements for smart home devices are no longer satisfied with the realization of basic functions. They also expect them to provide more diversified and intelligent services to further integrate into and optimize daily life. Therefore, an information prompt method is needed to support the breakthrough upgrade and personalized development of smart home devices. Summary of the invention
[0003] The embodiments of the present application provide an information prompt method, device, storage medium and electronic device to solve the technical problem that the information prompt effect of smart home devices is not good.
[0004] In a first aspect, an embodiment of the present application provides an information prompting method, the method comprising:
[0005] Obtaining door front event data recorded by smart home devices for door front events;
[0006] Input the door-front event data into the event analysis big model, and determine the event type of the door-front event output by the event analysis big model based on the door-front event data;
[0007] The target prompting method corresponding to the door front event is determined according to the event type, and the prompting information corresponding to the door front event is displayed on the smart home device according to the target prompting method.
[0008] In a second aspect, an embodiment of the present application provides an information prompting device, the device comprising:
[0009] A video acquisition module is used to acquire door front event data recorded by smart home devices for door front events;
[0010] A type determination module, used for inputting the door front event data into the event analysis big model, and determining the event type of the door front event output by the event analysis big model based on the door front event data;
[0011] The information prompt module is used to determine the target prompt method corresponding to the door front event according to the event type, and display the prompt information corresponding to the door front event on the smart home device according to the target prompt method.
[0012] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of the above-mentioned method.
[0013] In a fourth aspect, an embodiment of the present application provides an electronic device, characterized in that it includes: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method of the first aspect above.
[0014] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:
[0015] The embodiment of the present application provides an information prompt method, which obtains the door event data recorded by the smart home device for the door event; inputs the door event data into the event analysis model, and determines the event type of the door event output by the event analysis model based on the door event data; determines the target prompt method corresponding to the door event according to the event type, and displays the prompt information corresponding to the door event on the smart home device according to the target prompt method. Since the smart home device can monitor the door event and obtain the corresponding door event data, and input the data into the event analysis model to obtain the corresponding event type, the intelligent analysis of the door event data is realized, so as to accurately identify and classify the door events in various complex scenarios (such as family members returning home, express delivery, strangers staying, etc.). Based on the event type output by the event analysis model, the most appropriate information prompt method can be selected for each event, so as to realize more intelligent information prompts for different types of events on the smart home device, so that the information prompts are more in line with the personalized needs of users. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1 An exemplary system architecture diagram of an information prompt method provided in an embodiment of the present application;
[0018] Figure 2 A flowchart of an information prompting method provided in an embodiment of the present application;
[0019] Figure 3 A schematic diagram of an interface for displaying prompt information on a display screen of a smart home device provided in an embodiment of the present application;
[0020] Figure 4 A schematic diagram of an interface for displaying prompt information on a display screen of a smart home device provided in an embodiment of the present application;
[0021] Figure 5 A flowchart of an information prompting method provided in an embodiment of the present application;
[0022] Figure 6 A schematic diagram of an interface for setting animation prompts for different events through a terminal provided in an embodiment of the present application;
[0023] Figure 7 A schematic diagram of an interface for setting animation prompts for different events through a terminal provided in an embodiment of the present application;
[0024] Figure 8 A schematic diagram of an interface for setting voice prompts for different events through a terminal provided in an embodiment of the present application;
[0025] Fig. 9 A schematic diagram of an interface for setting up real-time care of a package through a terminal provided in an embodiment of the present application;
[0026] Fig.10 A schematic diagram of an interface for setting a stay alarm through a terminal provided in an embodiment of the present application;
[0027] Fig.11 A flowchart of a model training method for a large event analysis model provided in an embodiment of the present application;
[0028] Fig.12 A structural block diagram of an information prompting device provided in an embodiment of the present application;
[0029] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] In order to make the features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0031] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.
[0032] With the rapid development of science and technology and the improvement of people's quality of life, smart home devices have become an indispensable part of the family. In addition to the basic functions, existing smart home devices are gradually integrating more intelligent elements through advanced technologies such as fingerprint recognition, password verification, and facial recognition, providing users with a more comprehensive and considerate service experience. For example, smart door locks provide more convenient unlocking methods such as fingerprint, password and face recognition. Users can choose the appropriate unlocking method according to actual conditions, which improves the convenience of daily entry and exit. In addition to the basic unlocking function, smart door locks have gradually integrated more intelligent functions, which can visually monitor the scene in front of the door and prompt simple events in front of the door, so that users can keep abreast of the dynamics in front of the door, providing users with basic security protection.
[0033] However, in the widespread application of smart home devices, how to further improve their intelligence level and provide more accurate and personalized information prompt services has become an important direction of current technological development. Existing smart home devices are difficult to accurately distinguish the types of complex events occurring in front of the door, such as family members returning home, express delivery, strangers staying, etc., resulting in insufficient accuracy and pertinence of information prompts. Moreover, most of the existing information prompt methods are limited to simple forms such as sound or light, lacking personalization and intelligence, and it is difficult to meet users' needs for diversified information prompts. Therefore, smart home devices still have a lot of room for improvement in user experience. It is particularly important to more accurately identify user identities and provide information prompt methods that are more in line with user needs.
[0034] Therefore, an embodiment of the present application provides an information prompt method to solve the technical problem that the information prompt effect of the above-mentioned smart home devices is not good.
[0035] See also Figure 1 , Figure 1 An exemplary system architecture diagram of an information prompt method provided in an embodiment of the present application.
[0036] like Figure 1As shown, the system architecture may include an electronic device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the electronic device 101 and the server 103. The network 102 may include various types of wired communication links or wireless communication links, for example, the wired communication link includes an optical fiber, a twisted pair, or a coaxial cable, and the wireless communication link includes a Bluetooth communication link, a Wireless-Fidelity (Wi-Fi) communication link, or a microwave communication link.
[0037] The electronic device 101 can interact with the server 103 through the network 102 to receive messages from the server 103 or send messages to the server 103, or the electronic device 101 can interact with the server 103 through the network 102 to receive messages or data sent by other users to the server 103. The electronic device 101 can be hardware or software. When the electronic device 101 is hardware, it includes but is not limited to smart home devices (such as smart door locks, smart cameras, etc.), smart phones, tablet computers, laptop portable computers, and desktop computers. When the electronic device 101 is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules (for example: used to provide distributed services), or it can be implemented as a single software or software module, which is not specifically limited here.
[0038] In the embodiment of the present application, the electronic device 101 first obtains the door event data recorded by the smart home device for the door event; then the electronic device 101 inputs the door event data into the event analysis model, and determines the event type of the door event output by the event analysis model based on the door event data; finally, the electronic device 101 determines the target prompt method corresponding to the door event according to the event type, and displays the prompt information corresponding to the door event on the smart home device according to the target prompt method. It should be noted that the electronic device 101 can be the smart home device itself mentioned in the above steps, or it can be other electronic devices (such as user terminals, etc.) that can connect and communicate with the above smart home devices.
[0039] The server 103 may be a business server that provides various services. It should be noted that the server 103 may be hardware or software. When the server 103 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or it may be implemented as a single server. When the server 103 is software, it may be implemented as multiple software or software modules (for example, for providing distributed services), or it may be implemented as a single software or software module, which is not specifically limited here.
[0040] Alternatively, the system architecture may not include the server 103. In other words, the server 103 may be an optional device in the embodiments of this specification. That is, the method provided in the embodiments of this specification may be applied to a system structure that only includes the electronic device 101, and the embodiments of this application do not limit this.
[0041] It should be understood that Figure 1 The number of electronic devices, networks and servers in the figure is only illustrative, and any number of electronic devices, networks and servers may be used according to implementation requirements.
[0042] See also Figure 2 , Figure 2 A flowchart of an information prompt method provided for an embodiment of the present application. The execution subject of the embodiment of the present application can be an electronic device that executes the information prompt, or a processor in an electronic device that executes the information prompt method, or an information prompt service in an electronic device that executes the information prompt method. For the convenience of description, the specific execution process of the information prompt method is introduced below by taking the execution subject being a processor in an electronic device as an example.
[0043] like Figure 2 As shown, the information prompt method may at least include:
[0044] S202: Obtain door event data recorded by smart home devices for door events.
[0045] Optionally, in the smart home system, smart home devices such as smart door locks installed on or in front of the door, as the first line of defense for family security, are increasingly becoming important nodes for family security and smart services. With the increasing diversification of consumer demand, personalized services have become an important development trend of smart home devices. In order to further improve the intelligence level and service quality of smart home devices and bring users a more convenient, safe and comfortable home life experience, how to provide more accurate and personalized information prompt services for door front events through such smart home devices has become the key. Based on this, in order to realize personalized information display for different door front events, it is first necessary to record and distinguish various door front events. Therefore, obtaining the data recorded by smart home devices for various door front events has become the basis for distinguishing door front events. By obtaining rich door front event data, such as the switch status of the door, the activities in front of the door, etc., the specific situation in front of the door can be more comprehensively understood, thereby providing more accurate data input for the classification of subsequent door front events, which is crucial for subsequent event analysis and the selection of information prompt methods.
[0046] Optionally, in order to achieve comprehensive and accurate data acquisition of door-front events, a variety of sensors can be directly integrated into smart home devices, including but not limited to cameras, infrared sensors, sound sensors, etc. These sensors can capture the door's open and close status, doorbell button pressing, images in front of the door, face recognition results, sound recognition results and other information, providing a rich data basis for event analysis. When any event occurs in front of the door, the relevant sensors will immediately capture and record the relevant data, and transmit it to the central processing unit of the electronic device or the cloud server in real time through a wireless network or wired connection for further processing to ensure the timeliness and accuracy of the data. In addition, since the sensor may capture a large amount of redundant or invalid data, the acquired data can be preprocessed and screened before the specific analysis of the door-front event data. This includes steps such as removing noise data, filling missing values, and data compression to improve the quality and transmission efficiency of the data. It should be noted that these sensors that obtain door event data can be integrated into smart door locks or other smart home devices, such as smart peepholes, smart cameras, etc., as long as they can realize data transmission between the sensors and the information prompt system; at the same time, multiple sensors can be selected at the same time to receive data. For example, two cameras can be integrated at the upper and lower positions of the smart home device respectively to collect video or images in a larger range. The embodiments of the present application do not limit the specific setting method and setting quantity of the sensors.
[0047] S204: Input the door event data into the event analysis model, and determine the event type of the door event output by the event analysis model based on the door event data.
[0048] Optionally, different users have different needs for information prompts, and different types of door-front events often require different prompt methods. By accurately determining the event types corresponding to different door-front events and combining the user's personal preferences and living habits, more personalized information prompt services can be provided to users to improve user experience. Figure 3 , 4 The interface diagram of displaying prompt information on the display screen of a smart home device is shown. When a family member is identified as returning home, a welcome message can be displayed on the display screen of the smart home device, and a welcome message can be broadcasted by voice. When a stranger is identified as staying for too long, a sound alarm or a warning message displayed on the screen may be used to warn the stranger. Therefore, after obtaining the door event data recorded for the door event and performing certain data preprocessing, the door event needs to be identified and classified so that the smart home device can provide personalized information prompts based on different door events.
[0049] Optionally, in order to accurately identify the event type, an event analysis model can be trained with the help of artificial intelligence technology, such as deep learning and machine learning technology. This large model is used to process and analyze the door-front event data, thereby realizing intelligent identification and classification of door-front events. In order to accurately identify the event type of the door-front event, the model needs to be fully trained during the construction phase. The training data should contain samples of various door-front events, such as family face images, express delivery scenes, and abnormal stays of strangers. Each event type should have a sufficient number of instances and be labeled with accurate event type labels. The model can learn how to accurately predict the event type of the door-front event based on these samples. When the preprocessed data is input into the event analysis large model, the model will infer and analyze these features based on its internally learned knowledge and rules, and finally determine the most likely event type. In order to improve accuracy, technical means such as multi-model fusion can also be used to make a final judgment by combining the prediction results of multiple models.
[0050] S206: Determine a target prompt method corresponding to the door event according to the event type, and display prompt information corresponding to the door event on the smart home device according to the target prompt method.
[0051] Optionally, after the event type of the door front event is determined, it is necessary to perform differentiated prompting methods according to the different types of door front events, which requires an in-depth understanding of common door front events, the ability to accurately grasp the characteristics of each event type, such as the frequency and importance, and the ability to quickly perform corresponding personalized prompts after identification. Based on this, a set of mapping rules can be preset for common door front event types, which defines the direct correspondence between each event type and the prompting method, so that each time a door front event occurs, the smart home device can quickly identify the event type and take the corresponding prompting method without the need for a complex decision-making process, greatly improving the response speed and efficiency of the information prompt service. For example, the event type of a family member returning home may trigger the display screen of the smart home device to display a welcome message and play a welcome sound effect; the event type of a stranger staying for too long may trigger alarm information, such as a sound alarm, flashing lights, animation warnings, and sending emergency notifications to the user's mobile phone; when it is recognized that a family member goes out in the rain, the door lock will prompt the current weather conditions and remind the user to bring an umbrella; when the camera captures the courier delivering the goods to the door and monitors the courier ringing the doorbell, it prompts an automatic signing response voice, etc. It should be noted that various event types and their corresponding prompt methods can be set by the user, and the embodiments of the present application do not limit this.
[0052] Optionally, in order to improve the effectiveness of prompt information and user experience, smart home devices also support multimodal prompt methods. In addition to basic visual (such as display screen display) and auditory (such as voice reminder) prompts, other modes of prompts such as tactile (such as vibration reminders) and lighting (such as flashing lights of specific colors) can be introduced according to event types and user needs. This fusion of multimodal prompt methods can ensure that users can receive prompt information in a timely and effective manner in various situations. In addition, the number of devices for information prompts on smart home devices can also be multiple, for example, two display screens can be set to prompt different information. The embodiments of the present application do not limit the specific settings of the prompt method and prompt device.
[0053] Optionally, for frequently occurring doorstep events, intelligent filtering and priority sorting functions can also be set. By analyzing historical data and user behavior patterns, smart home devices can automatically identify and filter out unimportant or repeated prompts, while ensuring that important or urgent prompts are displayed to users first. Similarly, the priority sorting of doorstep event types can also be customized by the user according to their own needs. If the user sets the priority sorting of various events, the smart home device can perform intelligent filtering of events according to the user settings, which helps reduce user information interference and improve the efficiency of information processing.
[0054] In an embodiment of the present application, an information prompt method is provided to obtain the door event data recorded by the smart home device for the door event; input the door event data into the event analysis model, determine the event type of the door event output by the event analysis model based on the door event data; determine the target prompt method corresponding to the door event according to the event type, and display the prompt information corresponding to the door event on the smart home device according to the target prompt method. Since the smart home device can monitor the door event and obtain the corresponding door event data, and input the data into the event analysis model to obtain the corresponding event type, the intelligent analysis of the door event data is realized, so as to accurately identify and classify the door events in various complex scenarios (such as family members returning home, express delivery, strangers staying, etc.). Based on the event type output by the event analysis model, the most appropriate information prompt method can be selected for each event, so as to realize more intelligent information prompts for different types of events on the smart home device, so that the information prompts are more in line with the personalized needs of users.
[0055] See also Figure 5 , Figure 5 A flowchart of an information prompting method provided in an embodiment of the present application.
[0056] like Figure 5 As shown, the information prompt method may at least include:
[0057] S502: Acquire prompt mode setting information sent by the user terminal, and determine the user identity information, specific event information, and prompt mode information in the prompt mode setting information.
[0058] Optionally, considering that different users may have different preferences and requirements for information prompts, the function of user customization and dynamic adjustment of mapping rules can be introduced to increase the flexibility and adaptability of the system. Figure 6 , 7 The interface diagram of setting animation prompts for different events through the terminal is shown, and Figure 8 The interface diagram for setting voice prompts for different events through the terminal is shown. Users can set or modify the prompts corresponding to specific event types according to their actual conditions and preferences through the control panel of the smart home device or the mobile phone APP and other terminals. For example, people who are more concerned about weather conditions can set the weather conditions to be broadcast every time the door is unlocked; while people who pay more attention to the festive atmosphere can set different blessings for different festivals. When the user returns home during the festival, the smart home device automatically broadcasts the blessing voice and displays the blessing animation. For people who are busy at work, you can choose "mobile phone APP push notification" as the main information prompt method, and turn on "voice broadcast" as an alternative method, so that you can receive prompts even when your mobile phone is silent or not with you; for people who often stay at home, you can choose to turn off the "voice broadcast" prompt method to reduce interference with your rest.
[0059] Optionally, the smart home device can receive requests and data packets from the user's smart terminal. When a data packet containing prompt mode setting information is detected, it is immediately received and stored in the corresponding configuration database. The received setting information is usually a structured data packet containing multiple fields. The system needs to parse the data packet according to a preset format to extract user identity information, specific event information and prompt mode information. Among them, the user identity information includes information such as user ID, user name, user face image, etc., which is used to identify which user initiated the setting request or for which user the personalized setting is made, so as to ensure the personalization of subsequent prompt services. The specific event information specifies which types of door-front events require special setting of prompt modes, such as express delivery, family members returning home, visitors, strangers staying, etc. The prompt mode information is the prompt mode selected by the user for the above specific events, such as displaying a preset animation (such as a smiley face for welcome, an exclamation mark for warning), playing a preset voice (such as "Please put the express at the door"), a vibration prompt, or a combination of multiple modes. These prompt mode setting information can be selected by the user based on the prompt options pre-set by the system, or uploaded by the user. In addition, the user is also allowed to flexibly set the application time of the information prompt function. For example, users who frequently receive express deliveries may wish to set the delivery reminder to sound only during a specific time period to avoid disturbing other family members.
[0060] Optionally, in addition to receiving information prompts on smart home devices, users can also choose to synchronize corresponding information push on mobile phones and other terminals, so that when users are not at home, they can receive real-time information prompts of door-front events, understand the door-front conditions and take timely response measures. Fig. 9 Taking the interface diagram for setting up real-time care of packages through the terminal as an example, users can set rules such as the area and time range for real-time care of packages outside the door on terminals such as mobile phones. When the package is delivered and automatically signed for within the preset time, the smart home device can perform real-time care of the package according to the preset rules, and send corresponding package abnormality reminders to the receiving terminal set by the user under abnormal conditions.
[0061] S504. Determine a prompt method set corresponding to the specific event information, and add a prompt method corresponding to the user identity information to the prompt method set based on the prompt method information. The prompt method is at least one of displaying a preset animation and playing a preset voice on a prompt device in the smart home device.
[0062] Optionally, each specific event (such as express delivery, visitor visit, stranger stay, etc.) predefines one or more available prompt method sets, which contain feasible prompt methods in smart home devices, such as displaying preset animations, playing preset voices, etc. After parsing the received prompt method setting information and clarifying which events the user wants to set or modify the prompt method for, it is necessary to save the effective setting information to the prompt method set corresponding to the specific event, so that the corresponding information prompt method can be quickly executed when the event is triggered. Specifically, in the prompt method setting information provided by the user, a personalized prompt method for a specific specific event is specified. The system will first query the corresponding predefined prompt method set based on the parsed specific event information, and then add the specified prompt method information to the corresponding prompt method set and associate it with the user identity information. If the user selects a new prompt method and the method is not in the predefined set, the system may first evaluate its feasibility and then decide whether to add it to the set. For example, when a user adds a prompt method setting information about a "holiday greeting" event, the system will first parse the prompt method setting information, determine that the user identity information in the information is a child at home, and the specific event information is a Children's Day greeting. The prompt method information is to display the text "Happy Children's Day" on the screen and play the voice "Welcome home" on the speaker. Then the system will find the prompt method set corresponding to the "holiday greeting" event according to the setting information, and save the above information to the set, and at the same time associate the information with the identity information of the children at home, so that the smart home device can provide corresponding information prompts when it recognizes that the children have returned home on Children's Day.
[0063] Optionally, the user who adds the prompt mode setting information and the user corresponding to the user identity information in the setting information can be the same person or different people, that is, the user is allowed to set personalized information prompt modes for other users. For example, a mother can set a prompt mode corresponding to weather condition reminders for herself, or a prompt mode corresponding to holiday greetings for her children. In addition, the prompt mode setting information may not include user identity information. When specific user identity information is not included, the corresponding prompt mode is triggered by default when the corresponding event occurs, without distinguishing and displaying according to the user's identity.
[0064] S506: Obtain door front event data recorded by the smart home device for the door front event.
[0065] Regarding step S506, please refer to the detailed description in step S202, which will not be repeated here.
[0066] S508, input the door event data into the event analysis model, determine the event type of the door event output by the event analysis model based on the door event data, and the event type includes the target user identity information in the door event and the target specific event information of the door event.
[0067] Optionally, after obtaining the door event data recorded for the door event and performing certain data preprocessing, the door event needs to be identified and classified so that the smart home device can provide personalized information prompts according to different door events. Therefore, the door event data can be processed and analyzed with the help of the event analysis model, so as to realize intelligent identification and classification of the door event types.
[0068] Optionally, the event analysis model uses face recognition, voice recognition or other biometrics to extract target user identity information from the door event data. This information includes but is not limited to the user's facial features, voice features, preset ID or access rights, etc. By comparing this information with the user information stored in the database, the model can accurately identify the target user who participated in the door event. In addition to user identity information, the event analysis model can also deeply analyze the specific details of the door event, such as the time, location, duration, accompanying actions or behaviors of the event, etc., which together constitute the target specific event information of the door event. For example, in a visitor visit event, the model may identify details such as the visitor's stay time, whether he or she carries items, and whether he or she tries to communicate with family members. Based on the above analysis, the event analysis model combines user identity information and specific event information to output a complete event type description. This description not only details the participants of the event (target user identity information), but also records the specific context (target specific event information), providing a comprehensive and accurate basis for the subsequent prompt method selection and display.
[0069] S510: Determine a target prompting method set corresponding to the door front event according to the target specific event information, wherein the target prompting method set includes at least one prompting method.
[0070] Optionally, since a corresponding prompt method set is pre-defined for each specific event, when the event analysis model parses the target specific event information, the target prompt method set corresponding to this door-front event can be determined according to the pre-set rules. For example, if the door-front event "family goes home" occurs, the target specific event information parsed by the event analysis model may be "children go home on Children's Day". At this time, the corresponding "family goes home" target prompt method set will be found according to this target specific event information. This set pre-sets all the information prompt methods that may be used for the door-front event "family goes home".
[0071] S512: Determine a target prompt method corresponding to the target user identity information from the target prompt method set, and display prompt information corresponding to the door front event on the smart home device according to the target prompt method.
[0072] Optionally, the target prompting method set includes all possible information prompting methods for this type of doorstep event, so it is also necessary to find the target prompting method for the user of this doorstep event in the set in combination with the target user identity information, and perform corresponding information prompts according to the method. For example, the target prompting method set of "family going home" may include information prompting methods for multiple doorstep events such as "dad going home", "mom going home", and "children going home on Children's Day", while the event of "children going home on Children's Day" only needs to find its corresponding information prompting method, and there is no need to display other prompting methods.
[0073] S514: The event type is a regular event or an abnormal event. When the event type is an abnormal event, the abnormal level of the door front event is determined according to the event type.
[0074] Optionally, door-front events can be divided into regular events and abnormal events based on factors such as urgency, importance, and safety index. Abnormal events are often associated with potential risks and dangers, and have high urgency and danger. They may directly threaten the user's safety, property, or quality of life, and require immediate and drastic information prompts. For example, fires, long-term stays of strangers, or the removal of express deliveries are all high-urgency events that require immediate notification to the user and may trigger corresponding safety measures. In contrast, regular events are relatively mild and are often related to the user's daily quality of life. For example, a "Welcome Home" prompt after arriving home will not affect the user's daily safety, but can enhance the user's sense of happiness in life. By distinguishing between regular events and abnormal events, it helps the system prioritize urgent and important matters to ensure that the user's safety and interests are not lost; at the same time, for abnormal events, more resources can be invested and more eye-catching and effective prompts can be used to ensure the timely and accurate communication of prompt information and the effective implementation of subsequent processing measures. Fig.10Taking the interface diagram of setting the stay alarm through the terminal as an example, the user can set that when a stranger stays outside the door for a certain period of time within a certain range, the smart home device automatically plays the stay and expulsion warning and sends an alarm notification to the preset receiving terminal. When the smart home device detects that a stranger has stayed outside the door for a long time according to the preset rules, it will immediately issue a stay alarm, which can scare off the intruder and buy precious response time for the user. In addition, for abnormal unlocking situations inside the door such as when the adults go out and the children at home leave the house without permission, the user can also set the arming mode. After the arming mode is turned on, when the smart home device detects the abnormal unlocking situation inside the door, it will send an arming alarm notification to the preset receiving terminal, so that the adults can also understand the safety of the children at home at any time when they go out. In addition, timely notifying the user when a fire hazard is found can also reduce the losses and injuries caused by the fire. The type of abnormal event and the information prompt method can be set by the user, and the specific setting method of the abnormal event is not limited in the embodiment of the present application.
[0075] Optionally, considering that the urgency and danger of various abnormal events are different, the abnormal events can be graded to ensure that the system can give priority to those more urgent and important events, and adopt different prompting methods and processing strategies for abnormal events of different levels, so as to optimize resource allocation and ensure that abnormal events can be handled promptly and effectively. For example, the danger of a stranger staying is higher than that of a courier being removed, so a higher level should be set, and a more eye-catching and intense reminder and warning method should be used.
[0076] S516. Determine a target prompt method corresponding to the door front event based on the abnormality level, and display prompt information corresponding to the door front event on the smart home device according to the target prompt method, wherein the abnormality level is positively correlated with the intensity of the prompt method.
[0077] Optionally, when the current door front event is identified as an abnormal event, the corresponding abnormal level is first determined, and the target prompt method is determined according to the abnormal level, and then the prompt information is prompted according to the target prompt method. Among them, the higher the level of the abnormal event, the more intense the corresponding prompt method. This prompt mechanism in which the prompt method is gradually upgraded as the level increases meets the user's information acquisition needs in different situations and improves the user-friendliness of smart home devices.
[0078] In an embodiment of the present application, an information prompt method is provided, which allows users to customize information prompt methods through terminals, including prompt methods for specific users and specific events. This customized prompt method enables smart home devices to provide more personalized and accurate prompt services, ensuring that users obtain information that matches their needs. When a door front event is identified, the event analysis model can analyze the door front event, parse out the target user identity information and target specific event information, and determine the corresponding target prompt method accordingly. Through the introduction of the event analysis model, the system can independently determine the event type and take corresponding processing measures without manual intervention by the user. This intelligent processing method improves the processing efficiency of the system, and comprehensively considers the user identity information and specific event information during the processing process, ensuring the accuracy and reliability of the processing results. In addition, by distinguishing between regular events and abnormal events, and then dividing the abnormal events into abnormal levels, the smart door lock can select the most appropriate prompt method according to the urgency and importance of the event, so as to avoid unnecessary interference to the user or ignoring truly important events.
[0079] See also Fig.11 , Fig.11 A flowchart of a model training method for a large event analysis model provided in an embodiment of the present application.
[0080] like Fig.11 As shown, the model training method of the event analysis big model may at least include:
[0081] S1102. Construct an initial event analysis big model for event prompt scenarios based on the basic multimodal big model.
[0082] Optionally, the event analysis big model needs to classify the door-front event data based on the multi-faceted features of the door-front event data. In this case, a trained basic multimodal big model can be obtained. The basic multimodal big model can output prediction results for the prediction object based on various types of features of the prediction object. Therefore, an initial event analysis big model for event prompt scenarios is constructed based on the basic multimodal big model, so that the initial event analysis big model can also obtain the ability to "output prediction results for the prediction object based on various types of features of the prediction object".
[0083] Optionally, when the basic multimodal big model is directly used in a specific scenario, the unadjusted big model is difficult to adapt to the new scenario. Based on this, after building the initial event analysis big model for the event prompt scenario, the initial event analysis big model needs to be trained specifically.
[0084] S1104. Acquire multiple sample door-front event data, where the multiple sample door-front event data are all sample data with standard event type labels.
[0085] Optionally, multiple sample door-front event data are obtained for training the initial event analysis model, and the multiple sample door-front event data are all sample data with standard event type labels. The standard event type label is the correct event type corresponding to each sample door-front event, so it can be used as a standard label for the sample door-front event data.
[0086] S1106. Input multiple sample door-front event data into the initial event analysis large model to train the initial event analysis large model.
[0087] S1108. During the training process of the initial event analysis big model, the initial event analysis big model is controlled to output predicted event type labels for multiple sample door-front event data, and the parameters of the initial event analysis big model are adjusted according to the predicted event type labels and the standard event type labels of multiple sample door-front event data until the initial event analysis big model converges to obtain the trained event analysis big model.
[0088] Optionally, during the training process of the initial event analysis large model, the initial event analysis large model is controlled to output predicted event type labels for multiple sample door-front event data. These predicted event type labels are the event type prediction results of the initial event analysis large model for multiple sample door-front event data. Then, the gap between the predicted event type labels and the standard event type labels is the gap between the current state of the initial event analysis large model and the expected performance. Furthermore, the training loss value of the model can be calculated based on the predicted event type labels and the standard event type labels of multiple sample door-front event data, and the parameters in the initial event analysis large model are adjusted based on the training loss value until the initial event analysis large model converges to obtain the trained event analysis large model.
[0089] Optionally, the model training end condition of the model may include, for example, the value of the loss function meets the target value condition, the number of iterations reaches a preset number threshold, etc. The specific model training end condition can be determined based on actual conditions and is not specifically limited here.
[0090] In an embodiment of the present application, an information prompting method is provided, which includes a training method for an event analysis big model. The event analysis big model is constructed through the basic multimodal big model, and the event analysis big model is trained in a targeted manner in the event prompting scenario, so that the event analysis big model can perform in-depth analysis of the door event data, and through learning the sample door event data, it can obtain information that comprehensively considers both users and events in the event prompting scenario, thereby outputting more accurate and reliable prediction results.
[0091] See also Fig.12 , Fig.12 A structural block diagram of an information prompting device provided in an embodiment of the present application.
[0092] like Fig.12 As shown, the information prompting device 1200 includes:
[0093] The video acquisition module 1210 is used to acquire the door front event data recorded by the smart home device for the door front event;
[0094] A type determination module 1220 is used to input the door front event data into the event analysis big model, and determine the event type of the door front event output by the event analysis big model based on the door front event data;
[0095] The information prompt module 1230 is used to determine the target prompt method corresponding to the door front event according to the event type, and display the prompt information corresponding to the door front event on the smart home device according to the target prompt method.
[0096] Optionally, the event type includes the target user identity information in the door-front event and the target specific event information of the door-front event; the information prompt module 1230 is also used to determine the target prompt method set corresponding to the door-front event based on the target specific event information, the target prompt method set including at least one prompt method; determine the target prompt method corresponding to the target user identity information from the target prompt method set.
[0097] Optionally, the information prompt device 1200 also includes: a prompt method setting module, which is used to obtain the prompt method setting information sent by the user terminal, determine the user identity information, specific event information and prompt method information in the prompt method setting information; determine the prompt method set corresponding to the specific event information, and add the prompt method corresponding to the user identity information in the prompt method set based on the prompt method information.
[0098] Optionally, the prompting method is to display at least one of a preset animation and a preset voice on a prompting device in the smart home device.
[0099] Optionally, the event type is a regular event or an abnormal event, and the information prompt module 1230 is also used to determine the abnormal level of the door front event according to the event type when the event type is an abnormal event; determine the target prompt method corresponding to the door front event based on the abnormal level, wherein the size of the abnormal level is positively correlated with the intensity of the prompt method.
[0100] Optionally, the information prompting device 1200 also includes: a model training module, which is used to build an initial event analysis big model for the event prompting scenario based on the basic multimodal big model; obtain multiple sample door-front event data, and the multiple sample door-front event data are all sample data with standard event type labels; input the multiple sample door-front event data into the initial event analysis big model, and train the initial event analysis big model; during the training process of the initial event analysis big model, control the initial event analysis big model to output predicted event type labels for the multiple sample door-front event data, and adjust the parameters of the initial event analysis big model according to the predicted event type labels and the standard event type labels of the multiple sample door-front event data until the initial event analysis big model converges, so as to obtain the trained event analysis big model.
[0101] Optionally, the model training module is also used to calculate the training loss value based on the predicted event type label and the standard event type label of multiple sample door event data, and adjust the parameters of the initial event analysis large model based on the training loss value until the training loss value of the initial event analysis large model meets the target convergence condition, thereby determining that the initial event analysis large model converges.
[0102] In an embodiment of the present application, an information prompt device is provided, wherein a video acquisition module is used to acquire the door event data recorded by the smart home device for the door event; a type determination module is used to input the door event data into the event analysis model, and determine the event type of the door event output by the event analysis model based on the door event data; an information prompt module is used to determine the target prompt method corresponding to the door event according to the event type, and display the prompt information corresponding to the door event on the smart home device according to the target prompt method. Since the event analysis model can perform intelligent analysis based on the door event data, determine the door event reflected by the door event data, and analyze its specific type from the nature of the event, thereby obtaining the event type of the door event; then based on the event type output by the event analysis model, different information prompt methods can be adopted for each event according to the type of the event, so that a more reasonable information prompt method can be adopted for various events, and flexible notification of different types of events can be achieved, thereby improving the user experience.
[0103] An embodiment of the present application further provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of any method in the above embodiments.
[0104] See also Fig.13 , Fig.13 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.
[0105] like Fig.13As shown, the electronic device 1300 may include: at least one processor 1301 , at least one network interface 1304 , a user interface 1303 , a memory 1305 , a touch screen 1306 , and at least one communication bus 1302 .
[0106] The communication bus 1302 may be used to realize the connection and communication among the above-mentioned components.
[0107] The user interface 1303 may include buttons, a display screen, and a camera. The optional user interface may also include a standard wired interface and a wireless interface.
[0108] The network interface 1304 may optionally include a Bluetooth module, an NFC module, a Wi-Fi module, etc.
[0109] Among them, the processor 1301 may include one or more processing cores. The processor 1301 uses various interfaces and lines to connect the various parts of the entire electronic device 1300, and executes various functions and processes data of the electronic device 1300 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1305, and calling data stored in the memory 1305. Optionally, the processor 1301 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 1301 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1301, and it can be implemented separately through a chip.
[0110] Among them, the memory 1305 may include RAM and ROM. Optionally, the memory 1305 includes a non-transitory computer-readable medium. The memory 1305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1305 may optionally be at least one storage device located away from the aforementioned processor 1301. As Fig.13 As shown, the memory 1305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an information prompt program.
[0111] Specifically, the processor 1301 may be used to call the information prompt program stored in the memory 1305, and specifically perform the following operations:
[0112] Obtaining door front event data recorded by smart home devices for door front events;
[0113] Input the door-front event data into the event analysis big model, and determine the event type of the door-front event output by the event analysis big model based on the door-front event data;
[0114] The target prompting method corresponding to the door front event is determined according to the event type, and the prompting information corresponding to the door front event is displayed on the smart home device according to the target prompting method.
[0115] In some possible embodiments, the event type includes the target user identity information in the door front event and the target specific event information of the door front event; when the processor 1301 determines the target prompt method corresponding to the door front event according to the event type, it specifically performs the following steps: determine the target prompt method set corresponding to the door front event according to the target specific event information, and the target prompt method set includes at least one prompt method; determine the target prompt method corresponding to the target user identity information from the target prompt method set.
[0116] In some possible embodiments, the processor 1301 also specifically performs the following steps: obtaining the prompt mode setting information sent by the user terminal, determining the user identity information, specific event information and prompt mode information in the prompt mode setting information; determining the prompt mode set corresponding to the specific event information, and adding the prompt mode corresponding to the user identity information in the prompt mode set based on the prompt mode information.
[0117] In some possible embodiments, the prompting method is to display at least one of a preset animation and a preset voice on a prompting device in the smart home device.
[0118] In some possible embodiments, the event type is a regular event or an abnormal event; when the processor 1301 determines the target prompt method corresponding to the door front event according to the event type, it specifically performs the following steps: when the event type is an abnormal event, determine the abnormal level of the door front event according to the event type; determine the target prompt method corresponding to the door front event based on the abnormal level, wherein the size of the abnormal level is positively correlated with the intensity of the prompt method.
[0119] In some possible embodiments, the processor 1301 also specifically performs the following steps: constructing an initial event analysis big model for event prompt scenarios based on the basic multimodal big model; obtaining multiple sample door-front event data, wherein the multiple sample door-front event data are all sample data with standard event type labels; inputting the multiple sample door-front event data into the initial event analysis big model, and training the initial event analysis big model; during the training process of the initial event analysis big model, controlling the initial event analysis big model to output predicted event type labels for the multiple sample door-front event data, and adjusting the parameters of the initial event analysis big model according to the predicted event type labels and the standard event type labels of the multiple sample door-front event data until the initial event analysis big model converges, thereby obtaining the trained event analysis big model.
[0120] In some possible embodiments, when the processor 1301 adjusts the parameters of the initial event analysis large model according to the predicted event type label and the standard event type label of multiple sample gate event data until the initial event analysis large model converges, it also specifically performs the following steps: calculating the training loss value according to the predicted event type label and the standard event type label of multiple sample gate event data, adjusting the parameters of the initial event analysis large model based on the training loss value until the training loss value of the initial event analysis large model meets the target convergence condition, and determining that the initial event analysis large model converges.
[0121] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0122] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0123] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above computer program product includes one or more computer instructions. When the above computer program instructions are loaded and executed on a computer, the above process or function according to the embodiment of this specification is generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium or transmitted by the above computer-readable storage medium. The above computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVD)), or semiconductor media (for example, solid state disks (SSD)), etc.
[0124] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0125] In addition, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the user identity information and user face information involved in this application are all obtained with full authorization.
[0126] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0127] The above is a description of an information prompt method, device, storage medium and electronic device provided by the present application. For technicians in this field, according to the ideas of the embodiments of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. An information prompting method, characterized in that: The method comprises: Obtaining door front event data recorded by smart home devices for door front events; Inputting the door event data into an event analysis model, and determining the event type of the door event output by the event analysis model based on the door event data; A target prompting method corresponding to the door event is determined according to the event type, and prompting information corresponding to the door event is displayed on the smart home device according to the target prompting method.
2. The method according to claim 1, characterized in that The event type includes the target user identity information in the door-front event and the target specific event information of the door-front event; The determining, according to the event type, a target prompting method corresponding to the door front event includes: Determine a target prompt mode set corresponding to the door front event according to the target specific event information, wherein the target prompt mode set includes at least one prompt mode; The target prompting method corresponding to the target user identity information is determined from the target prompting method set.
3. The method according to claim 2, characterized in that The method further comprises: Acquire the prompt mode setting information sent by the user terminal, and determine the user identity information, specific event information, and prompt mode information in the prompt mode setting information; A prompt method set corresponding to the specific event information is determined, and a prompt method corresponding to the user identity information is added to the prompt method set based on the prompt method information.
4. The method according to claim 3, characterized in that The prompting method is at least one of displaying a preset animation and playing a preset voice on a prompting device in the smart home device.
5. The method according to claim 1, characterized in that The event type is a regular event or an abnormal event, and the determining, according to the event type, a target prompting method corresponding to the door front event includes: When the event type is the abnormal event, determining the abnormal level of the door front event according to the event type; A target prompting method corresponding to the door event is determined based on the abnormality level, wherein the magnitude of the abnormality level is positively correlated with the intensity of the prompting method.
6. The method according to claim 1, characterized in that The method further comprises: Construct an initial event analysis model for event prompt scenarios based on the basic multimodal model; Acquire multiple sample door-front event data, where the multiple sample door-front event data are all sample data with standard event type labels; Inputting the plurality of sample door-front event data into the initial event analysis large model to train the initial event analysis large model; During the training process of the initial event analysis big model, the initial event analysis big model is controlled to output predicted event type labels for the multiple sample door-front event data, and the parameters of the initial event analysis big model are adjusted according to the predicted event type labels and the standard event type labels of the multiple sample door-front event data until the initial event analysis big model converges, thereby obtaining the trained event analysis big model.
7. The method according to claim 6, characterized in that The step of adjusting the parameters of the initial event analysis model according to the predicted event type label and the standard event type labels of the plurality of sample door-front event data until the initial event analysis model converges includes: A training loss value is calculated according to the predicted event type label and the standard event type label of the multiple sample door event data, and the parameters of the initial event analysis large model are adjusted based on the training loss value until the training loss value of the initial event analysis large model meets the target convergence condition, thereby determining that the initial event analysis large model converges.
8. An information prompting device, characterized in that: The device comprises: A data acquisition module, used to acquire door front event data recorded by smart home devices for door front events; A type determination module, used for inputting the door event data into an event analysis model, and determining the event type of the door event output by the event analysis model based on the door event data; An information prompt module is used to determine a target prompt method corresponding to the door event according to the event type, and display prompt information corresponding to the door event on the smart home device according to the target prompt method.
9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.
10. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 7.