Article forgetting reminding method and device, electronic equipment and storage medium

Abstract

This invention provides a method, device, electronic device, and storage medium for reminding users to remember forgotten items, relating to the fields of smart home and Internet of Things (IoT) technology. The method includes: identifying the user's current activity context; determining the target item associated with the current activity context; acquiring the target item's status data, including location information and existence information; calculating a forgetting risk value corresponding to the current activity context based on the location and existence information; and outputting a reminder message if the forgetting risk value meets a condition. This allows for rapid location of the target item associated with the user's current activity context after identification, enabling passive, real-time tracking of the spatial location and existence information of items associated with the activity context without requiring the user to wear any tags. This reduces reliance on active tags and associated costs, while increasing the coverage of item tracking, significantly reducing the user's cognitive burden and improving convenience.

Description

Technical Field

[0001] This invention relates to the fields of smart home and Internet of Things technology, and in particular to a method for reminding users to forget items, a device for reminding users to forget items, an electronic device, and a readable storage medium. Background Technology

[0002] In daily life, people often face the frustration of forgetting to bring important items (such as keys, wallets, and medications) or spending a lot of time searching for them. On the one hand, item tracking solutions, represented by Bluetooth or UWB (Ultra-Wideband) active tags, require each item to be equipped with a dedicated electronic tag. This not only increases the user's economic costs and usage burden (such as the need for regular charging), but also makes it difficult to cover all items that may require management, such as glasses, remote controls, and medicine bottles. More importantly, these solutions are passive response modes, only searching after the user discovers the item is lost, and cannot proactively warn users when they are about to forget something. On the other hand, while smart reminder systems based on fixed rules (such as timed reminders) can proactively issue reminders, their reminders are often relatively blind and lack factual basis. For example, the system may still issue an invalid "Please take your keys" reminder when the user already has them, or it may be unable to confirm if the user has forgotten to take medication, causing annoyance or security oversight. Summary of the Invention

[0003] The present invention provides a method, device, electronic device, and readable storage medium for reminding users to forget items, in order to solve or partially solve the problems of high cost and limited coverage caused by relying on active tags in item tracking and management, as well as the blind reminder methods and lack of fact-based proactive intervention and task closure confirmation capabilities.

[0004] This invention discloses a method for reminding users to remember forgotten items, comprising: Identify the user's current activity context and determine the target item associated with the current activity context; Acquire the status data corresponding to the target item, wherein the status data includes at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception; Based on the location information and the existence information, calculate the forgetting risk value corresponding to the current activity context. If the forgetting risk value meets the preset reminder conditions, output the reminder information for the target item.

[0005] In some feasible implementations, calculating the forgetting risk value corresponding to the current activity context based on the location information and the existence information includes: The location information and the existence information are used to perform state recognition to obtain the current state of the target item and to obtain the activation probability corresponding to the current activity context; Based on the current state, calculate the probability that the target item is missing because it is not carried by the user; The forgetting risk value corresponding to the current activity context is calculated based at least on the activation probability and the missing probability.

[0006] In some feasible implementations, the step of using the location information and the existence information to perform state recognition and obtain the current state of the target item includes: If the location information is valid and the existence information indicates that the target item exists, then it is determined that the target item is currently in a stationary or moving state. If the location information is invalid but the existence information indicates that the target item exists, then the target item is determined to be currently in an obscured state. If the location information is invalid and the existence information indicates that the target item does not exist, then the historical interaction record of the target item is obtained. If the historical interaction record indicates that the target item has been operated by the user, then the target item is determined to be in a carried state.

[0007] In some feasible implementations, calculating the probability of the target item not being carried by the user based on the current state includes: Obtain the probability baseline value corresponding to the current state; Obtain the real-time spatial distance between the target item and the user; The probability baseline value is dynamically adjusted according to the real-time spatial distance to obtain the probability of the target item not being carried by the user.

[0008] In some feasible implementations, calculating the forgetting risk value corresponding to the current activity context based at least on the activation probability and the missing probability includes: Obtain the weight information corresponding to the target item; The total missing probability corresponding to the current activity scenario is obtained by weighting all the target items associated with the current activity scenario and calculating their missing probabilities. The forgetting risk value corresponding to the current activity context is obtained by calculating the activation probability and the total missing probability.

[0009] In some feasible implementations, if the forgetting risk value meets the preset reminder conditions, then a reminder message for the target item is output, including: If the risk value of forgetting is greater than or equal to the preset reminder threshold, then a reminder message for the target item is output.

[0010] In some feasible implementations, the output of reminder information for the target item includes: The forgetting risk value is matched with a preset risk level range to locate the target risk level range where the forgetting risk value is located, and the reminder strategy corresponding to the target risk level range is obtained. Execute the reminder strategy for the target item; The reminder strategy includes at least one of the following: controlling a designated light fixture to generate a preset light prompt, controlling a voice device to broadcast a preset voice prompt, and sending a notification message to a designated terminal device.

[0011] In some feasible implementations, identifying the user's current activity context includes: Acquire trigger data collected by the sensing sensors; The trigger data is matched with multiple preset activity scenarios. Based on the matching results, the user's current activity scenario is identified, and the activation probability corresponding to the current activity scenario is calculated.

[0012] In some feasible implementations, the step of matching the trigger data with multiple preset activity scenarios, identifying the user's current activity scenario based on the matching results, and calculating the activation probability corresponding to the current activity scenario includes: The trigger data is matched with expected sensor signal sequences in multiple preset activity scenarios to obtain the pattern matching degree corresponding to the trigger data; Obtain the current time information and match the current time information with the time context corresponding to the activity scenario to obtain the corresponding time matching degree; The pattern matching degree and the time matching degree are weighted and fused to obtain the activation probability corresponding to the current activity scenario.

[0013] In some feasible implementations, obtaining the status data corresponding to the target item includes: Obtain the last known location of the target item; Visual sensors deployed in the area associated with the last known location are scheduled to perceive the last known location, obtain corresponding image data, and perform target detection on the image data to obtain the location information of the target item; The millimeter-wave radar sensors deployed in the associated area are scheduled to analyze the last known location, obtain the corresponding reflected signal, and analyze the reflected signal to obtain the existence information of the target object; The location information and the existence information are fused to generate the status data of the target item.

[0014] This invention also discloses a device for reminding users to remember forgotten items, comprising: The identification module is used to identify the user's current activity context and determine the target item associated with the current activity context; A state recognition module is used to acquire state data corresponding to the target item, the state data including at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception. The reminder module is used to calculate the forgetting risk value corresponding to the current activity context based on the location information and the existence information. If the forgetting risk value meets the preset reminder conditions, the reminder information for the target item is output.

[0015] In some feasible implementations, the reminder module is specifically used for: The location information and the existence information are used to perform state recognition to obtain the current state of the target item and to obtain the activation probability corresponding to the current activity context; Based on the current state, calculate the probability that the target item is missing because it is not carried by the user; The forgetting risk value corresponding to the current activity context is calculated based at least on the activation probability and the missing probability.

[0016] In some feasible implementations, the reminder module is specifically used for: If the location information is valid and the existence information indicates that the target item exists, then it is determined that the target item is currently in a stationary or moving state. If the location information is invalid but the existence information indicates that the target item exists, then the target item is determined to be currently in an obscured state. If the location information is invalid and the existence information indicates that the target item does not exist, then the historical interaction record of the target item is obtained. If the historical interaction record indicates that the target item has been operated by the user, then the target item is determined to be in a carried state.

[0017] In some feasible implementations, the reminder module is specifically used for: Obtain the probability baseline value corresponding to the current state; Obtain the real-time spatial distance between the target item and the user; The probability baseline value is dynamically adjusted according to the real-time spatial distance to obtain the probability of the target item not being carried by the user.

[0018] In some feasible implementations, the reminder module is specifically used for: Obtain the weight information corresponding to the target item; The total missing probability corresponding to the current activity scenario is obtained by weighting all the target items associated with the current activity scenario and calculating their missing probabilities. The forgetting risk value corresponding to the current activity context is obtained by calculating the activation probability and the total missing probability.

[0019] In some feasible implementations, the reminder module is specifically used for: If the risk value of forgetting is greater than or equal to the preset reminder threshold, then a reminder message for the target item is output.

[0020] In some feasible implementations, the reminder module is specifically used for: The forgetting risk value is matched with a preset risk level range to locate the target risk level range where the forgetting risk value is located, and the reminder strategy corresponding to the target risk level range is obtained. Execute the reminder strategy for the target item; The reminder strategy includes at least one of the following: controlling a designated light fixture to generate a preset light prompt, controlling a voice device to broadcast a preset voice prompt, and sending a notification message to a designated terminal device.

[0021] In some feasible implementations, the identification module is specifically used for: Acquire trigger data collected by the sensing sensors; The trigger data is matched with multiple preset activity scenarios. Based on the matching results, the user's current activity scenario is identified, and the activation probability corresponding to the current activity scenario is calculated.

[0022] In some feasible implementations, the identification module is specifically used for: The trigger data is matched with expected sensor signal sequences in multiple preset activity scenarios to obtain the pattern matching degree corresponding to the trigger data; Obtain the current time information and match the current time information with the time context corresponding to the activity scenario to obtain the corresponding time matching degree; The pattern matching degree and the time matching degree are weighted and fused to obtain the activation probability corresponding to the current activity scenario.

[0023] In some feasible implementations, the state recognition module is specifically used for: Obtain the last known location of the target item; Visual sensors deployed in the area associated with the last known location are scheduled to perceive the last known location, obtain corresponding image data, and perform target detection on the image data to obtain the location information of the target item; The millimeter-wave radar sensors deployed in the associated area are scheduled to analyze the last known location, obtain the corresponding reflected signal, and analyze the reflected signal to obtain the existence information of the target object; The location information and the existence information are fused to generate the status data of the target item.

[0024] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0025] This invention also discloses a readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the method described in this invention.

[0026] The embodiments of the present invention have the following advantages: In this embodiment of the invention, by identifying the user's current activity context and determining the target item associated with the current activity context, the status data corresponding to the target item is then acquired. The status data includes at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception. Then, based on the location information and existence information, a forgetting risk value corresponding to the current activity context is calculated. If the forgetting risk value meets the preset reminder conditions, a reminder message for the target item is output. Thus, after identifying the user's current activity context, the target item associated with the activity context can be quickly located. By integrating multimodal perception methods such as computer vision and millimeter-wave radar, the status data corresponding to the target item can be acquired. This allows for passive, real-time tracking of the spatial location and existence information of items associated with the activity context without requiring the user to wear any tags. This reduces the dependence on active tags and costs, while improving the coverage of item tracking. Furthermore, based on the tracking status of the item, the corresponding forgetting risk value is calculated, which can trigger accurate proactive reminders. This achieves a task loop between "item tracking" and "proactive reminders," significantly reducing the user's cognitive burden and improving the convenience of life. Attached Figure Description

[0027] Figure 1This is a flowchart of the steps of a method for reminding people to forget items provided in an embodiment of the present invention; Figure 2 This is a system architecture diagram of the forgetting reminder provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of a forgotten item reminder device provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] As an example, in the process of tracking related items, on the one hand, item tracking solutions represented by Bluetooth or UWB active tags require each item to be equipped with a dedicated electronic tag. This not only increases the user's economic costs and usage burden (such as the need for regular charging), but also makes it difficult to cover all items that may need management, such as glasses, remote controls, and medicine bottles. More importantly, these solutions are passive response modes, only searching after the user discovers the item is lost, and cannot proactively warn when the user is about to forget an item. On the other hand, intelligent reminder systems based on fixed rules (such as timed reminders) can proactively issue reminders, but their reminders are often relatively blind and lack factual basis. For example, the system may still issue an invalid reminder of "Please take your keys" when the user already has them, or it cannot confirm when the user has forgotten to take medication, causing annoyance or security oversight.

[0030] In response to this, this invention provides a complete solution for passive and precise item status tracking and intelligent forgetting intervention without requiring users to actively wear or attach any electronic tags (such as Bluetooth trackers). By constructing a "passive item-activity binding" intelligent framework, after identifying the user's current activity context, it quickly locates the target item associated with that activity context. It integrates multimodal perception methods, including computer vision and millimeter-wave radar, to acquire the corresponding status data of the target item. This allows for passive, real-time tracking of the spatial location and existence information of items associated with the activity context without requiring the user to wear any tags. This reduces reliance on active tags and associated costs, while increasing the coverage of item tracking. Furthermore, based on the tracking status of items, it calculates the corresponding forgetting risk value, triggering precise proactive reminders. This achieves a closed-loop task between "item tracking" and "proactive reminders," significantly reducing the user's cognitive burden and improving convenience.

[0031] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, some technical features involved in the embodiments of the present invention are explained and described below: Activity contexts can be fragments of daily behavior with clear intent and fixed patterns, such as "leaving home," "preparing to take medication," or "going to bed." This can be a semantically abstracted process, allowing the system to aggregate a series of raw sensor signals (such as walking towards the entrance or picking up a coat) into a system-understandable "user goal." For a user's activity context, the system can quantify the confidence that the activity is happening or about to happen using activation probability, a value between 0 and 1. Activation probability is the triggering factor and key weight for calculating the risk of forgetting; only when the activation probability is sufficiently high will the system initiate subsequent in-depth checks on related items.

[0032] For example, if the system detects a user walking towards the entrance and picking up a briefcase at 8:15 AM on Monday, the user's activity highly matches the preset "leaving for work" scenario pattern. Based on historical patterns for this time period, the system may calculate the activation probability P(R). LeavingHome The value of 0.95 indicates that there is a 95% probability that the user's current activity intention is to go to work.

[0033] The last known location of a tracked item can be a dynamically maintained spatial coordinate anchor point for each tracked item. It is not fixed, but rather the physical location (coordinates in the world coordinate system) where the item was most recently reliably sensed and confirmed by the system. For example, if a key was last seen or detected at (x=2.1, y=3.5, z=0.8) on the kitchen counter, this coordinate is its "last known location." When the system needs to inspect the item, it will first direct the sensors to focus on this area for sensing.

[0034] Optionally, a "container following" mechanism can be introduced during the tracking of items. This mechanism can include "static occlusion" and "dynamic inclusion." "Static occlusion" refers to the situation where the tracked item is visually lost, but the millimeter-wave radar features at its location coordinates have not disappeared. In this case, the system can maintain the "last known location" and mark the tracked item's state as "occluded," such as a scenario where a key is obscured by a newspaper. "Dynamic inclusion" can be triggered when the system detects that the bounding box of the "tracked item" overlaps with the bounding box of the "user (or clothing)" at the Intersection over Union (IOU), and the visual signal of the item subsequently disappears. In this case, the system can transfer the coordinate reference system of the "tracked item" from the "world coordinate system" to the "human / clothing coordinate system." After the coordinate system transfer, as long as the user moves, the virtual position of the "tracked item" can be locked on the user. By designing a "container following" mechanism, the "last known position" is no longer a fixed coordinate point, but a dynamic pointer. For example, when the item is on the table, it points to the coordinates of the table; when the item is in the pocket, it points to the coordinates of the user, and so on.

[0035] The location information of an object refers to its coordinates in three-dimensional space calculated through visual perception (such as a camera combined with target detection algorithms), which can answer questions like "Where is the object's precise location?". For example, through image analysis, the system calculates the visual coordinates of a key as (x=2.12, y=3.48, z=0.79), which basically matches the last known location. However, pure visual perception fails under occlusion or low light conditions, at which point the location information may become null. Based on this, this invention also introduces existence information, which can be derived from millimeter-wave radar perception. The existence information can be a probability value (such as 0.98) used to confirm whether the target object still physically exists at a specific location (especially the "last known location"). Specifically, millimeter-wave radar can penetrate non-metallic obstructions and detect micro-movements to answer the question "Is the object still there?".

[0036] Location information and existence information serve as the two pillars of multimodal perception fusion: the former provides precise coordinates but is susceptible to interference, while the latter provides reliable existence verification but lacks precise semantics. Combining the two allows the system to be certain that an object is still in its original location ('last known location') even when it is covered by a newspaper (visual 'location information' is invalid), thanks to the high probability value of radar's 'existence information', thus constructing a robust object state perception. The last known location, location information, and existence information can collectively serve to provide a precise digital description of the object's physical state, enabling the system to accurately identify the object's state.

[0037] Missing probability is a concept specific to a particular item, quantifying the likelihood that the item is "not within the user's carrying range," with a value ranging from 0 to 1. The calculation of missing probability can rely on the aforementioned item status data (location information, existence information, and the merged status, etc.). For example, if the item status is determined to be "carried," its missing probability is close to 0; if it is determined to be "stationary," the missing probability is close to 1.

[0038] The forgetting risk value serves as a comprehensive risk assessment indicator at the system level. It quantifies the overall risk that a user's activity may be hindered or suffer adverse consequences due to the omission of a key item in the currently identified activity context. Specifically, the forgetting risk value calculation integrates the activation probability (the likelihood of the situation occurring), the probability of one or more related items being missing (the likelihood that the item is not present), and the preset importance weight of each item in that context. The forgetting risk value can be a fusion result based on a probabilistic model, essentially answering the following question: "Based on the current user's behavioral intention (high activation probability) and the fact that he / she may not have brought an important item (high missing probability), what is the overall risk of a 'forgotten incident' occurring at this moment?" Only when this calculated forgetting risk value exceeds a preset threshold will the system determine that intervention is necessary, thereby triggering a reminder.

[0039] Regarding the aforementioned technical features, a clear decision-making logic chain is formed in this embodiment of the invention: the system infers the user's "activity context" through sensor data and calculates its "activation probability"; for items associated with this context, the system uses its "last known location" as a reference point and determines the real-time status of the items by fusing "location information" and "existence information"; based on this status, the "missing probability" of each item is calculated; finally, the "activation probability" and "missing probability" are combined to obtain a global "forgetting risk value," which is used as the basis for deciding whether to execute proactive intelligent reminders.

[0040] Specifically, refer to Figure 1 The diagram illustrates a flowchart of a method for reminding users to forget items, as provided in an embodiment of the present invention. Specifically, it may include the following steps: Step 101: Identify the user's current activity context and determine the target item associated with the current activity context; Optionally, the physical deployment of the entire system can include wide-angle vision sensors (such as 1080p HD cameras), millimeter-wave radar sensors (such as TIIWR 6843), environmental sensors (such as door magnets and infrared human body sensors), and edge computing gateways serving as the system's "brain," deployed in key indoor areas (such as entryways, living rooms, bedrooms, and kitchens). Correspondingly, at the software level, the system runs on top of the gateway and can include: a context recognition engine, a multimodal perception fusion module, a knowledge graph management module, a forgetting risk calculation engine, and a multimodal reminder actuator. During the process of reminding users to remember forgotten items, all data processing, model reasoning, and decision-making can be completed locally on the gateway to fully protect user privacy and ensure real-time response.

[0041] In this embodiment of the invention, with user authorization, the system can identify the user's current activity context through trigger data collected by environmental sensors, so as to determine the target item associated with the current activity context and perform subsequent processing on the target item.

[0042] In some feasible implementations, after acquiring trigger data collected by sensing sensors, the system can match the trigger data with multiple preset activity scenarios, identify the user's current activity scenario based on the matching results, and calculate the activation probability corresponding to the current activity scenario. Specifically, the activation probability calculation process involves first matching the trigger data with expected sensor signal sequences in multiple preset activity scenarios to obtain the pattern matching degree corresponding to the trigger data. Simultaneously, it acquires the current time information and matches it with the time context corresponding to the activity scenario to obtain the corresponding time matching degree. Then, it performs a weighted fusion of the pattern matching degree and the time matching degree to obtain the activation probability corresponding to the current activity scenario.

[0043] Furthermore, after identifying the user's current activity context, the system can quickly filter out target items associated with the current activity context through methods such as table lookup, so as to perform subsequent process processing on the target items.

[0044] In its implementation, the system continuously collects asynchronous and heterogeneous trigger data streams from deployed environmental sensors, including but not limited to spatial movement and orientation data, object interaction data, environmental state data, and temporal context data. Specifically, for spatial movement and orientation data, the system can use visual sensors (cameras) combined with human skeletal keypoint detection algorithms (such as OpenPose or the lightweight MoveNet) to track the user's movement trajectory, speed, and orientation in real time indoors (e.g., detecting a user moving from the bedroom to the entryway). For object interaction data, the system can use visual sensors to identify the user's interaction actions with specific objects (e.g., "picking up a medicine bottle" or "putting on a coat"). For environmental state data, the system can use door magnetic sensors to obtain the open / closed status of doors (especially the front door); and use infrared sensors to sense the user's continuous lingering in a specific area (such as the entryway). For temporal context data, the system can maintain an internal clock and associate it with a database of the user's historical behavior patterns.

[0045] Accordingly, the system can predefine a set of key "daily activity scenarios" templates, which can include all the activity scenarios that a user might perform on a daily basis, for example, R LeavingHome (Away from home), R GoingToBed (go to bed), R TakingMedicine_AM (Morning medication), R WorkingAtDesk (Start working), etc. Each scenario template can be associated with a expected sensor signal sequence and a time context model, so as to determine whether the user's current activity matches the corresponding activity scenario based on the expected sensor signal sequence and the time context model.

[0046] After collecting the corresponding trigger data stream, the system can perform dynamic time warping (DTW) or neural network-based sequence similarity calculation on the real-time collected trigger data stream (such as the visual event sequence of "user walks towards the entrance" + "door sensor closes" + "picks up coat") and the expected sequence of each scenario template to obtain a pattern matching score S between 0 and 1. pattern For example, the above sequence and R LeavingHome The template's match score can be as high as 0.95, while the match score with R_GoingToBed is close to 0.

[0047] And, compare the current time with the typical occurrence time window of each scenario template (such as R). LeavingHome This usually occurs between 7:00-9:00 on weekdays and 9:00-12:00 on weekends; R TakingMedicine_AM The comparison is performed daily at 10:00 AM. A time-match score S_time is calculated using a Gaussian distribution or an empirical probability model. For example, at 8:30 AM on Monday, S_time... time (R LeavingHomeThe value might be 0.9, while S time (R TakingMedicine_AM The value is 0.1.

[0048] Ultimately, the activation probability P(R) of the current activity context R is... active P(R) can be obtained by weighted fusion of pattern matching degree and time matching degree: active )=α*S pattern +β*S time The weights α and β can be optimized according to the characteristics of different scenarios (e.g., for strictly time-related medication, β has a higher weight; for behavior-driven outings, α has a higher weight). The system can choose P(R active The scenario that exceeds a certain confidence threshold (e.g., 0.7) and has the highest score is designated as the "current activity scenario".

[0049] Furthermore, once the current activity context is identified, the system can further associate target items with that context. Optionally, the system can determine the target items associated with the current activity context by querying a pre-built and continuously evolving "context-item association knowledge graph." This knowledge graph can be structured with "activity context" and "item entity" as nodes, "associations" (such as requires, suggestions) as edges, and each edge is assigned a weight w, representing the importance of the item in completing the activity.

[0050] For example, there are edges in the graph: (R LeavingHome )-[requires, w=0.9]->(Keys), (R LeavingHome )-[suggests, w=0.6]->(Sunglasses).

[0051] For the "context-item association knowledge graph," in the initial version, it can be preset by the system or manually configured by the user through the application (e.g., selecting keys, wallet, and phone in the "leaving home" scenario). In subsequent versions, it can continuously evolve through dynamic learning. In some examples, the system can adjust the weights of corresponding items through reinforcement learning. If the system repeatedly reminds the user to bring sunglasses (w=0.6), but the user always ignores it and leaves directly, the system can automatically lower w through negative feedback. sunglasses (e.g., dropping to 0.2); conversely, if the system observes that the user actively searches for headphones before leaving the house, it can automatically add the "Bluetooth Headphones" node to R. LeavingHomeThe system generates an association list and assigns it a high initial weight (e.g., 0.8). In other examples, the system can analyze historical item interaction data to automatically discover patterns. For instance, assuming that for a week on weekday mornings, the probability of a user walking to the entrance within 5 minutes of picking up their work card exceeds 90%, the system can automatically create an association list (R). LeavingHome The association between )-[requires, w=0.85]->(WorkIDCard).

[0052] During the process of retrieving target items associated with the current activity context, the system can retrieve a set of all item entities that are directly associated with the current activity context and have a weight greater than or equal to a preset threshold (such as 0.3) from the knowledge graph based on the current activity context. This set is then used as the target items associated with the current activity context for subsequent steps such as state tracking and risk calculation.

[0053] Step 102: Obtain the status data corresponding to the target item. The status data includes at least the location information obtained based on visual perception and the existence information obtained based on millimeter-wave radar perception. After identifying the target item associated with the current activity context, the system can further acquire the state data corresponding to the target item. This state data may include location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception. Optionally, the location information can be a visual positioning vector, and the existence information can be an existence probability; this invention does not impose limitations on these aspects.

[0054] In some feasible implementations, the system can first obtain the last known location of the target item, then schedule visual sensors deployed in the area associated with the last known location to perceive the last known location, obtain the corresponding image data, and perform target detection on the image data to obtain the location information of the target item. Additionally, it can schedule millimeter-wave radar sensors deployed in the associated area to analyze the last known location, obtain the corresponding reflection signal, and analyze the reflection signal to obtain the existence information of the target item. Finally, the location information and existence information are fused to generate the state data of the target item.

[0055] In its implementation, the system does not need to monitor the entire space indiscriminately. Instead, it can obtain the last known location of the target item and then use that location as a reference for precise and on-demand sensing. Specifically, the system can maintain a dynamic database of item locations. For each target item Oi, the system can query the coordinates (x, y, y) of its latest known location. 已知 y 已知 z 已知The coordinates can be based on a world coordinate system (e.g., with a fixed corner of the room as the origin). The system can send corresponding collaborative acquisition commands to visual sensors and millimeter-wave radar sensors deployed in the area associated with the last known location. Through these collaborative acquisition commands, the system schedules the designated camera to capture image data of the area associated with the last known location. For the captured image data, a lightweight target detection model (such as YOLOv8-M or NanoDet) running on edge nodes (such as gateways or camera built-in chips) can perform real-time analysis of the image and output the 2D bounding box of the target object. Then, the system can use the pre-calibrated camera intrinsic parameter matrix K and extrinsic parameter matrix [R|T], and combine the "primary support surface assumption" (assuming that the object is usually located on a plane of known height, such as a tabletop or the ground), to back-project the 2D pixel coordinates (u, v) into 3D space through projective geometry to obtain the initial visual positioning vector L. vis =[x vis y vis , z vis ]^T.

[0056] Furthermore, the system can also schedule millimeter-wave radar to perform high-precision scanning of the last known location through collaborative acquisition commands. The millimeter-wave radar sensor transmits frequency-modulated continuous waves and receives reflected signals. The received reflected signals can be processed to generate range-Doppler heatmaps and point cloud data. Optionally, millimeter-wave radar can penetrate non-metallic obstructions (such as paper and fabric) and is sensitive to minute movements / micro-Doppler effects.

[0057] Furthermore, based on the raw sensory data collected in the above process, the system can generate unified item state data through a multimodal fusion engine. Optionally, the multimodal fusion engine can be based on Kalman filters or deep neural networks, etc., and this invention is not limited thereto.

[0058] Specifically, the data collected by the millimeter-wave radar sensor can be used to calculate the probability of the existence of a target object at its last known location. The system can analyze the intensity of the reflected signal and micro-Doppler characteristics at the last known location of the target object, and match them with the previously learned signal "fingerprint" of the target object. A continuous probability value P is output through a sigmoid function. mmW ∈[0, 1]. For example, even if a metal key is covered by a newspaper, its strong reflective properties will still cause P to... mmW The value remains above 0.95; if the item is removed, the signal characteristic at that location disappears, and P... mmW It will drop to close to 0.1 (background noise level), etc.

[0059] Accordingly, for location information, precise ranging information from millimeter-wave radar sensors can be used for calibration to improve the accuracy of location perception. For example, when vision fails due to insufficient light or complete occlusion (i.e., the visual positioning vector is "NULL," and the existence of the target object cannot be perceived visually), as long as the existence information P... mmW If the height is high enough, the system can still be sure that the item is in its original location and retain the last valid location information as its location reference.

[0060] After obtaining the corresponding visual positioning vector and existence probability, the system can determine the current state of the item based on a logical combination of the two, using a decision tree or finite state machine, as shown in Table 1 below. The corresponding judgment logic may include:

[0061] Table 1 Ultimately, each target item O i The state data at time t can be encapsulated into a state vector: O t =(ID,L vis P mmW S state This vector can provide a precise, structured, and digital description of the physical world state of an object.

[0062] Step 103: Calculate the forgetting risk value corresponding to the current activity context based on the location information and the existence information. If the forgetting risk value meets the preset reminder conditions, output the reminder information for the target item.

[0063] Through the process described in the aforementioned embodiments, the system perceives "where" and "in what state" the target item is, and understands "what the user is going to do" based on the context. The system can then conduct a risk assessment and trigger intelligent actions. Based on the tracking of the item, the system calculates the corresponding risk value of forgetting. If the risk value of forgetting meets the preset reminder conditions, the system can trigger a precise proactive reminder, thus realizing a closed loop between "item tracking" and "proactive reminders." This significantly reduces the cognitive burden on users and improves the convenience of life.

[0064] In some feasible implementations, after obtaining the location information and existence information of the target item, the system can use the location information and existence information to perform state recognition, obtain the current state of the target item, obtain the activation probability corresponding to the current activity context, and calculate the missing probability of the target item not being carried by the user based on the current state. Then, based at least on the activation probability and the missing probability, the system can calculate the forgetting risk value corresponding to the current activity context.

[0065] In identifying the current state of a target item, the system can accurately determine its current state based on factors such as the validity of location information (i.e., whether the target item remains at its "last known location" in visual perception) and the likelihood that existence information indicates the target item remains at its "last known location." Specifically, if the location information is valid and existence information indicates the target item exists, the system determines that the target item is currently stationary or moving. If the location information is invalid but existence information indicates the target item exists, the system determines that the target item is currently occluded. If the location information is invalid and existence information indicates the target item does not exist, the system retrieves the target item's historical interaction records. If the historical interaction records indicate that the target item has been operated by the user, the system determines that the target item is being carried.

[0066] Furthermore, after identifying the current state of the target item, the system can obtain the probability baseline value corresponding to the current state, then obtain the real-time spatial distance between the target item and the user, and then dynamically adjust the probability baseline value according to the real-time spatial distance to obtain the missing probability that the target item is not carried by the user. Then, the system can at least calculate the forgetting risk value corresponding to the current activity scenario based on the activation probability and the missing probability.

[0067] Optionally, the calculation of the forgetting risk value can be a multi-factor probability model, and its core calculation formula can be:

[0068] in, This can be the probability of a target "key daily activity" being activated, i.e., the activation probability, which can be inferred by the daily activity recognition engine based on sensors (such as door sensors, cameras). For example, when the system detects that a user walks towards the entrance + picks up a coat + the entrance door sensor remains closed, the system infers P(R). LeavingHome =0.95.

[0069] I can be the i-th key item bound to the activity R, stored in the knowledge graph.

[0070] You can assign a weight to the importance of item i for the activity (defined by the user or learned by the system). For example, for "leaving home", the key's w_keys = 0.9, but the sunglasses' w_keys = 0.9. sunglasses =0.2.

[0071] It represents an item The probability that the item is not in the user's "personal area" (such as a pocket or hand), i.e., the missing probability, can be calculated based on the results obtained in the aforementioned embodiments. For example, if Display items If it remains "still at the bar", then If the item has already been picked up ( If the probability is close to 1, then the probability is close to 1. .

[0072] In its implementation, after calculating the missing probability of the target item, the system can further obtain the weight information corresponding to the target item. Then, it uses the weight information of all target items associated with the current activity context and their missing probabilities to perform a weighted calculation to obtain the total missing probability for the current activity context. Next, it uses the activation probability and the total missing probability to calculate the forgetting risk value for the current activity context. Optionally, during the calculation of this forgetting risk value, the system can trigger the system to check "high-weight" items associated with the current activity context if the activation probability meets a condition (e.g., greater than or equal to a preset threshold), and then calculate the corresponding forgetting risk value. For example, the system will only check items with "high weight" for that activity (e.g., keys) when a "high-probability critical activity" (e.g., about to leave) occurs. If these items are "highly unlikely to be on the user" (i.e., still on the table), F... int The value will surge, thereby triggering a precise reminder; this invention does not limit this.

[0073] After calculating the corresponding forgetting risk value through the aforementioned embodiments, the system can compare the forgetting risk value with a preset reminder threshold. If the forgetting risk value is greater than or equal to the preset reminder threshold, a reminder message for the target item is output. Furthermore, during the output of the reminder message, the system can match the forgetting risk value with a preset risk level range, locate the target risk level range where the forgetting risk value is located, obtain the reminder strategy corresponding to the target risk level range, and then execute the reminder strategy for the target item. The reminder strategy includes at least one of the following: controlling a designated light fixture to generate a preset light prompt, controlling a voice device to broadcast a preset voice reminder, and sending a notification message to a designated terminal device. Thus, through a hierarchical multimodal reminder strategy, reminders are given specifically according to the actual scenario. On the one hand, this effectively saves system resource consumption; on the other hand, it improves the targeting and effectiveness of "forgotten reminders" through differentiated reminder methods.

[0074] In some instances, when calculating the forgetting risk value F... int Then, the system can compare it with a preset alert threshold to make a final intervention decision. Specifically, the decision logic adopted by the system can be: if F intIf the threshold is greater than or equal to a preset threshold (e.g., the threshold could be set to 0.75), the system is deemed to have met the alert condition and will trigger the corresponding intervention. It should be noted that this threshold is not fixed but can be flexibly adjusted according to the sensitivity of different activity scenarios. For example, for health and safety-related "medication" scenarios, the system can use a lower threshold (e.g., 0.7) to ensure earlier and more cautious alerts; while for everyday "going out" scenarios, a relatively higher threshold can be used to balance the timeliness of the alert with the potential for user disruption.

[0075] Furthermore, to optimize user experience while ensuring the effectiveness of reminders, the system can adopt a tiered strategy, implementing a tiered multimodal reminder mechanism. Specifically, this mechanism can be based on F... int The numerical range, or in combination with other risk indicators in the calculation process (such as the weight w of a related item). i (Abnormally high), the alerts are divided into different levels from light to heavy, thereby achieving accurate and appropriate human-computer interaction.

[0076] Specifically, when the system determines a risk to be mild (e.g., 0.75 ≤ F), int When the risk level is <0.85, or the user is only likely to forget non-critical items, a non-intrusive and gentle reminder will be used. For example, the smart lights in the entryway may flash a warm yellow light slowly, or a gentle notification icon may be displayed on the screen of a smart refrigerator or other interface the user passes by, conveying information without interrupting the user's current activity. When the risk escalates to a moderate risk level (e.g., 0.85 ≤ F...), a more subtle and gentle reminder will be used. int If the value is less than 0.95, or involves the loss of key items such as keys, the system will activate a clear voice reminder, broadcasting specific information such as "Friendly reminder, your keys seem to still be on the kitchen counter" through the smart speaker closest to the user, to clearly attract the user's attention. For high-risk situations (such as F...), int For alerts ≥0.95, or in scenarios involving health and safety such as missed medication doses, the system will implement a strong intervention strategy: not only will it play a more urgent voice alert indoors (e.g., "Attention! Your 10 AM medication has not been taken!"), but it will also send push notifications to the user's linked mobile app and forward the alert to designated emergency contacts (e.g., children) according to preset rules, thus forming a multi-layered protection network to ensure that critical safety needs are not overlooked. Through this tiered, multi-channel alert system, the system can provide information at the appropriate time and in the appropriate manner, effectively improving the acceptance of warnings and the success rate of intervention.

[0077] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that those skilled in the art can make further settings according to actual needs under the guidance of the ideas in the embodiments of the present invention, and the present invention does not limit such settings.

[0078] In this embodiment of the invention, by identifying the user's current activity context and determining the target item associated with the current activity context, the status data corresponding to the target item is then acquired. The status data includes at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception. Then, based on the location information and existence information, a forgetting risk value corresponding to the current activity context is calculated. If the forgetting risk value meets the preset reminder conditions, a reminder message for the target item is output. Thus, after identifying the user's current activity context, the target item associated with the activity context can be quickly located. By integrating multimodal perception methods such as computer vision and millimeter-wave radar, the status data corresponding to the target item can be acquired. This allows for passive, real-time tracking of the spatial location and existence information of items associated with the activity context without requiring the user to wear any tags. This reduces the dependence on active tags and costs, while improving the coverage of item tracking. Furthermore, based on the tracking status of the item, the corresponding forgetting risk value is calculated, which can trigger accurate proactive reminders. This achieves a task loop between "item tracking" and "proactive reminders," significantly reducing the user's cognitive burden and improving the convenience of life.

[0079] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following examples are provided for illustrative purposes: As an example, refer to Figure 2 This paper illustrates a system architecture diagram for forgetting reminders provided in an embodiment of the present invention. The system comprises an end-to-end process from physical perception to intelligent decision-making and final execution. The system design follows the concept of "cloud-edge-device" collaboration to ensure real-time performance, accuracy, and privacy. The following provides a complete description of each layer, module, and interaction relationship within the system: I. Perception Layer The perception layer is the "sensory nerve ending" of the entire system, responsible for collecting multi-dimensional and heterogeneous raw data from the physical world.

[0080] 101a microphone array: Deployed in key indoor areas to capture user voice commands. It primarily responds to direct voice queries, such as when a user asks, "Where are my keys?" The system picks up the audio and uploads the audio stream, providing input for voice interaction functions.

[0081] The 101b wide-angle camera: also deployed in key areas, is used to capture video streams within its coverage area. Its core task is to provide visual information, providing the image data foundation for subsequent object recognition and user behavior analysis.

[0082] 101c millimeter-wave radar: Deployed in conjunction with cameras, it transmits and receives millimeter-wave signals. It can penetrate non-metallic obstructions to detect the presence, subtle movements, and precise distances of objects, providing a perception dimension beyond vision and specifically addressing the challenges of obstruction and insufficient light.

[0083] 101d door magnets / sensors: These generally refer to contact or environmental sensors (such as infrared human body sensors) installed on doors, windows, drawers, etc. They provide discrete "activity signals," such as the "opening / closing" of the entrance door, or someone entering a specific area, and are important triggers for inferring users' daily activities.

[0084] II. Edge Computing Nodes Edge computing nodes are the "local reflection hubs" of the system. They are deployed at the network edge close to the sensors and are responsible for handling sensing tasks with extremely high real-time requirements, relieving the pressure on the cloud and reducing the upload of raw data.

[0085] 200 edge computing nodes: These are typically embedded devices or home gateways with a certain computing power. They receive raw streaming data from the perception layer.

[0086] Edge Object Recognition (YOLOv8): A lightweight deep learning model (such as YOLOv8) runs on edge nodes. It processes video streams from a 101b wide-angle camera in real time, performs object detection tasks, identifies pre-registered key items (such as keys and medicine bottles), and outputs the item's ID and its coordinates in the image coordinate system. This pre-processed structured data (rather than the raw video) is then uploaded to the cloud, greatly saving bandwidth and protecting privacy.

[0087] III. Cloud Processing Center The cloud processing center is the "brain" of the system, possessing powerful computing and storage resources, and is responsible for complex data fusion, contextual understanding, knowledge reasoning, and decision-making.

[0088] 300 Cloud Processing Center: Cloud Server Cluster.

[0089] 301 Multimodal Fusion Engine: This is one of the core processing modules. It receives item coordinates / IDs from edge nodes, as well as raw or pre-processed radar presence data from the 101c millimeter-wave radar. The engine spatiotemporally aligns and fuses visual positioning information with radar presence and ranging information to generate a high-confidence, unified state vector (containing precise 3D position, presence probability, motion state, etc.) for each item.

[0090] 302 Daily Activity Recognition Engine: Another core processing module. It continuously receives activity signals from the 101d door sensor / thermometer and analyzes them in conjunction with time patterns. For example, after continuously receiving signals such as "user walks towards the entrance," "picks up a coat," and "door sensor remains closed," the engine can infer the user's intention to "leave home" and output the activity status (R). active The confidence level that the activity is happening is usually expressed as a probability value.

[0091] 303 Item-Contextual Knowledge Graph: This is the system's "memory" and "common sense base." It is a graph database that stores two core relationships: Item-State Association: Records the real-time and historical states of each tracked item (updated by the 301 multimodal fusion engine).

[0092] Activity-Item Association: Defines which key items (such as keys, wallet) a user typically needs to associate with a specific activity state (such as "going out"). It responds to queries from the 302 engine and also accepts voice queries from users (such as "What should I take with me when I go out?").

[0093] 304 Forgetting Intervention Strategy Engine: This is the system's "decision center." It receives the activity status (R) from the 302 engine. active It then queries the 303 knowledge graph for items associated with the activity and their latest status. Based on this information, it runs a risk calculation model to calculate the final intervention instruction (F). int ). F int A quantifiable risk value can be used to determine whether intervention is needed and the urgency of such intervention.

[0094] IV. Execution Layer The execution layer is the system's "hands and feet," responsible for translating intelligent decisions from the cloud into actions in the physical world that users can perceive.

[0095] 400 Execution Layer: Composed of various controllable terminal devices within the home.

[0096] 401 Smart Lighting: Receives intervention commands from the 304 engine. Depending on the level of the command, it executes different lighting responses; for example, it slowly flashes yellow light when the risk is low, and rapidly flashes red light when the risk is high.

[0097] The 402 Smart Speaker receives intervention commands and executes voice broadcasts. For example, it can broadcast "Your keys are on the table" in response to a location query, or broadcast "Don't forget your medicine bottle" as a proactive reminder.

[0098] 403 Mobile App Notifications: Push important reminders, especially high-risk health and safety alerts, to users' smartphones to ensure no information is missed.

[0099] 404 Display / AR Glasses: Provides visual feedback. For example, on a smart home control screen or augmented reality glasses, the location of the object being searched can be highlighted graphically.

[0100] Based on the aforementioned system architecture, the data flow and workflow of the entire system can include: Perception and Edge Processing: Cameras and radar continuously perceive the environment, and edge nodes identify objects in real time and upload their coordinates. Cloud Fusion and Understanding: The cloud fusion engine integrates visual and radar data to accurately determine the status of objects; the activity recognition engine analyzes sensor signals to determine the user's current context. Knowledge Graph Query and Decision-Making: Based on the identified activity, the strategy engine queries the knowledge graph to obtain a list of related items and their real-time status, and calculates the forgetting risk value F. int Multimodal execution: based on F int Based on the value, the strategy engine generates intervention commands at different levels, driving devices such as smart speakers, lights, and mobile apps to provide tiered interventions ranging from gentle prompts to strong alarms. Interaction loop: Users can also directly query the system via voice. After the system finds the item's status through a knowledge graph, it provides a direct response via a smart speaker or display screen.

[0101] Through the above process, after identifying the user's current activity context, the system quickly locates the target items associated with that context. By integrating multimodal perception methods such as computer vision and millimeter-wave radar, it acquires the corresponding status data of the target items. This allows for passive, real-time tracking of the spatial location and existence information of items associated with the activity context without requiring the user to wear any tags. This reduces reliance on active tags and associated costs, while also increasing the coverage of item tracking. Furthermore, based on the tracking status of the items, a corresponding forgetting risk value is calculated, which can trigger precise proactive reminders. This achieves a closed loop between "item tracking" and "proactive reminders," significantly reducing the user's cognitive burden and improving convenience.

[0102] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0103] Reference Figure 3 The diagram shows a structural block diagram of a forgotten item reminder device provided in an embodiment of the present invention, which may specifically include the following modules: The identification module 301 is used to identify the user's current activity context and determine the target item associated with the current activity context; The state recognition module 302 is used to acquire state data corresponding to the target item, and the state data includes at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception. The reminder module 303 is used to calculate the forgetting risk value corresponding to the current activity context based on the location information and the existence information. If the forgetting risk value meets the preset reminder conditions, the reminder information for the target item is output.

[0104] In some feasible implementations, the reminder module is specifically used for: The location information and the existence information are used to perform state recognition to obtain the current state of the target item and to obtain the activation probability corresponding to the current activity context; Based on the current state, calculate the probability that the target item is missing because it is not carried by the user; The forgetting risk value corresponding to the current activity context is calculated based at least on the activation probability and the missing probability.

[0105] In some feasible implementations, the reminder module is specifically used for: If the location information is valid and the existence information indicates that the target item exists, then it is determined that the target item is currently in a stationary or moving state. If the location information is invalid but the existence information indicates that the target item exists, then the target item is determined to be currently in an obscured state. If the location information is invalid and the existence information indicates that the target item does not exist, then the historical interaction record of the target item is obtained. If the historical interaction record indicates that the target item has been operated by the user, then the target item is determined to be in a carried state.

[0106] In some feasible implementations, the reminder module is specifically used for: Obtain the probability baseline value corresponding to the current state; Obtain the real-time spatial distance between the target item and the user; The probability baseline value is dynamically adjusted according to the real-time spatial distance to obtain the probability of the target item not being carried by the user.

[0107] In some feasible implementations, the reminder module is specifically used for: Obtain the weight information corresponding to the target item; The total missing probability corresponding to the current activity scenario is obtained by weighting all the target items associated with the current activity scenario and calculating their missing probabilities. The forgetting risk value corresponding to the current activity context is obtained by calculating the activation probability and the total missing probability.

[0108] In some feasible implementations, the reminder module is specifically used for: If the risk value of forgetting is greater than or equal to the preset reminder threshold, then a reminder message for the target item is output.

[0109] In some feasible implementations, the reminder module is specifically used for: The forgetting risk value is matched with a preset risk level range to locate the target risk level range where the forgetting risk value is located, and the reminder strategy corresponding to the target risk level range is obtained. Execute the reminder strategy for the target item; The reminder strategy includes at least one of the following: controlling a designated light fixture to generate a preset light prompt, controlling a voice device to broadcast a preset voice prompt, and sending a notification message to a designated terminal device.

[0110] In some feasible implementations, the identification module is specifically used for: Acquire trigger data collected by the sensing sensors; The trigger data is matched with multiple preset activity scenarios. Based on the matching results, the user's current activity scenario is identified, and the activation probability corresponding to the current activity scenario is calculated.

[0111] In some feasible implementations, the identification module is specifically used for: The trigger data is matched with expected sensor signal sequences in multiple preset activity scenarios to obtain the pattern matching degree corresponding to the trigger data; Obtain the current time information and match the current time information with the time context corresponding to the activity scenario to obtain the corresponding time matching degree; The pattern matching degree and the time matching degree are weighted and fused to obtain the activation probability corresponding to the current activity scenario.

[0112] In some feasible implementations, the state recognition module is specifically used for: Obtain the last known location of the target item; Visual sensors deployed in the area associated with the last known location are scheduled to perceive the last known location, obtain corresponding image data, and perform target detection on the image data to obtain the location information of the target item; The millimeter-wave radar sensors deployed in the associated area are scheduled to analyze the last known location, obtain the corresponding reflected signal, and analyze the reflected signal to obtain the existence information of the target object; The location information and the existence information are fused to generate the status data of the target item.

[0113] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0114] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described method for reminding forgotten items and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0115] This invention also provides a readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the aforementioned method for reminding forgotten items, achieving the same technical effect. To avoid repetition, these processes will not be described again here. The readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0116] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0117] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, EEPROM, Flash, and eMMC, etc.) containing computer-usable program code.

[0118] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0122] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0123] The foregoing has provided a detailed description of a method and device for reminding people to forget items. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for reminding users to remember forgotten items, characterized in that, include: Identify the user's current activity context and determine the target item associated with the current activity context; Acquire the status data corresponding to the target item, wherein the status data includes at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception; Based on the location information and the existence information, calculate the forgetting risk value corresponding to the current activity context. If the forgetting risk value meets the preset reminder conditions, output the reminder information for the target item.

2. The method according to claim 1, characterized in that, The step of calculating the forgetting risk value corresponding to the current activity context based on the location information and the existence information includes: The location information and the existence information are used to perform state recognition to obtain the current state of the target item and to obtain the activation probability corresponding to the current activity context; Based on the current state, calculate the probability that the target item is missing because it is not carried by the user; The forgetting risk value corresponding to the current activity context is calculated based at least on the activation probability and the missing probability.

3. The method according to claim 2, characterized in that, The step of using the location information and the existence information to perform state recognition and obtain the current state of the target item includes: If the location information is valid and the existence information indicates that the target item exists, then it is determined that the target item is currently in a stationary or moving state. If the location information is invalid but the existence information indicates that the target item exists, then the target item is determined to be currently in an obscured state. If the location information is invalid and the existence information indicates that the target item does not exist, then the historical interaction record of the target item is obtained. If the historical interaction record indicates that the target item has been operated by the user, then the target item is determined to be in a carried state.

4. The method according to claim 2, characterized in that, The step of calculating the probability of the target item not being carried by the user based on the current state includes: Obtain the probability baseline value corresponding to the current state; Obtain the real-time spatial distance between the target item and the user; The probability baseline value is dynamically adjusted according to the real-time spatial distance to obtain the probability of the target item not being carried by the user.

5. The method according to claim 2, characterized in that, The calculation of the forgetting risk value corresponding to the current activity context, based at least on the activation probability and the missing probability, includes: Obtain the weight information corresponding to the target item; The total missing probability corresponding to the current activity scenario is obtained by weighting all the target items associated with the current activity scenario and calculating their missing probabilities. The forgetting risk value corresponding to the current activity context is obtained by calculating the activation probability and the total missing probability.

6. The method according to any one of claims 1 to 5, characterized in that, If the forgotten risk value meets the preset reminder conditions, then a reminder message for the target item is output, including: If the risk value of forgetting is greater than or equal to the preset reminder threshold, then a reminder message for the target item is output.

7. The method according to any one of claims 1 to 5, characterized in that, The output of the reminder information for the target item includes: The forgetting risk value is matched with a preset risk level range to locate the target risk level range where the forgetting risk value is located, and the reminder strategy corresponding to the target risk level range is obtained. Execute the reminder strategy for the target item; The reminder strategy includes at least one of the following: controlling a designated light fixture to generate a preset light prompt, controlling a voice device to broadcast a preset voice prompt, and sending a notification message to a designated terminal device.

8. The method according to claim 1, characterized in that, The process of identifying the user's current activity context includes: Acquire trigger data collected by the sensing sensors; The trigger data is matched with multiple preset activity scenarios. Based on the matching results, the user's current activity scenario is identified, and the activation probability corresponding to the current activity scenario is calculated.

9. The method according to claim 8, characterized in that, The step of matching the trigger data with multiple preset activity scenarios, identifying the user's current activity scenario based on the matching results, and calculating the activation probability corresponding to the current activity scenario includes: The trigger data is matched with expected sensor signal sequences in multiple preset activity scenarios to obtain the pattern matching degree corresponding to the trigger data; Obtain the current time information and match the current time information with the time context corresponding to the activity scenario to obtain the corresponding time matching degree; The pattern matching degree and the time matching degree are weighted and fused to obtain the activation probability corresponding to the current activity scenario.

10. The method according to claim 1, characterized in that, The step of obtaining the status data corresponding to the target item includes: Obtain the last known location of the target item; Visual sensors deployed in the area associated with the last known location are scheduled to perceive the last known location, obtain corresponding image data, and perform target detection on the image data to obtain the location information of the target item; The millimeter-wave radar sensors deployed in the associated area are scheduled to analyze the last known location, obtain the corresponding reflected signal, and analyze the reflected signal to obtain the existence information of the target object; The location information and the existence information are fused to generate the status data of the target item.

11. A device for reminding users to remember an item they have forgotten, characterized in that, include: The identification module is used to identify the user's current activity context and determine the target item associated with the current activity context; A state recognition module is used to acquire state data corresponding to the target item, the state data including at least location information obtained based on visual perception and existence information obtained based on millimeter-wave radar perception. The reminder module is used to calculate the forgetting risk value corresponding to the current activity context based on the location information and the existence information. If the forgetting risk value meets the preset reminder conditions, the reminder information for the target item is output.

12. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-10.

13. A readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-10.

Images