Intelligent terminal user identity recognition method and system based on multi-modal perception and terminal

Abstract

The application discloses a multi-modal perception-based intelligent terminal user identity identification method and system and a terminal, relates to the technical field of intelligent terminal user identity identification, and comprises the following steps: when it is detected that a mobile terminal meets the target local area network candidate user list and the current short-distance wireless signal strength meets the predetermined strength condition, it is determined that the mobile terminal owner meeting the condition is the current target user of the intelligent terminal; when it is detected that there is no mobile terminal meeting the target local area network candidate user list and the current short-distance wireless signal strength does not meet the predetermined strength condition, a secondary confirmation mechanism including non-inductive historical identity and active inquiry is controlled to be used to determine the current target user of the intelligent terminal; based on the determination of the current target user of the intelligent terminal, the entire interface of the intelligent terminal is seamlessly switched to the service interface and the recommended content corresponding to the current target user portrait. The application improves the confidence and reliability of intelligent terminal user identity identification.

Description

Technical Field

[0001] This invention relates to the field of smart terminal user identification technology, and in particular to a smart terminal user identification method, system, smart terminal, and storage medium based on multimodal perception. Background Technology

[0002] Currently, smart TVs, serving as the center of home entertainment, rely heavily on user profiles for their content recommendation systems. However, traditional TVs have a fundamental flaw: they cannot automatically identify the specific user currently watching. This leads to the following problems: 1) Profile confusion: TV recommendations are often a mixture of viewing histories from multiple family members, failing to provide precise personalized services for any one individual. For example, the awkward situation of "Dad's recommendations are all cartoons" might occur. 2) Rigid interaction: TVs cannot adjust the interface, language preferences, or auxiliary functions based on the current user.

[0003] Therefore, existing technologies cannot reliably and seamlessly identify the current TV viewer, which is not conducive to personalized recommendations.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method, system, smart terminal and storage medium for user identification of smart terminals based on multimodal perception, which addresses the problems and defects of the prior art. The present invention provides stable proof of device presence through the connection status of local area networks such as Wi-Fi, and provides accurate distance perception through Bluetooth signal strength. The two work together to make decisions, which overcomes the defects of single technologies and improves the confidence and reliability of identification.

[0006] The technical solution adopted by this invention to solve the problem is as follows: A method for identifying users of intelligent terminals based on multimodal perception, comprising: Upon detecting an identity verification request, the system retrieves a list of online devices in the target local area network and generates a list of candidate users for the target local area network; it also scans for nearby short-range wireless communication devices and obtains the short-range wireless signal strength of these devices. When a mobile terminal is detected to simultaneously meet the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets the predetermined strength condition, the owner of the mobile terminal that meets the conditions is determined to be the current target user of the smart terminal. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively asking questions to determine the current target user of the smart terminal. Based on identifying the current target user of the smart terminal, the system seamlessly switches the entire interface of the smart terminal to the service interface and recommended content corresponding to the current target user profile.

[0007] The method for identifying the identity of a smart terminal user based on multimodal perception, wherein the step of controlling a secondary confirmation mechanism including seamlessly retaining historical identity and active inquiry to determine the current target user of the smart terminal when a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength meets a predetermined strength condition includes: When multiple mobile terminals are detected to have equal wireless signal strength, or a wireless signal strength difference less than a predetermined value, or when it cannot be determined, the system enters a secondary confirmation mechanism; wherein, the secondary confirmation mechanism includes: The control system directly uses the user profile that was successfully identified last time to determine the current target user of the smart terminal; Alternatively, the control system may proactively inquire about the user's identity to determine the current target user of the smart terminal. This proactive inquiry may include voice inquiry, sending a push notification to the user's mobile terminal to request confirmation, or displaying a user selection interface on the smart terminal.

[0008] The aforementioned method for identifying user identities on intelligent terminals based on multimodal perception, wherein the step of seamlessly switching the entire interface of the intelligent terminal to a service interface and recommended content corresponding to the current target user profile, based on determining the current target user of the intelligent terminal, includes: Once the identity of the current target user on the smart terminal is confirmed, the control will synchronize the confirmed target user identity to the cloud. Based on identifying the current target user of the smart terminal, the system seamlessly switches the entire interface of the smart terminal to the service interface and recommended content corresponding to the current target user profile. The identification data will be recorded for subsequent user identification.

[0009] The method for identifying intelligent terminal users based on multimodal perception, wherein the target local area network includes: a Wi-Fi network, a ZigBee network, or a Z-Wave network; The wireless signals include Bluetooth signals, UWB ultra-wideband wireless signals, or NFC signals.

[0010] The method for identifying the identity of a smart terminal user based on multimodal perception, wherein the step of controlling a secondary confirmation mechanism including seamlessly retaining historical identity and active inquiry to determine the current target user of the smart terminal when a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, further includes: If a mobile terminal is not detected in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength condition, the current target user of the smart terminal is determined by combining the mobile terminal device's past time habit data and the motion sensor data of the mobile terminal after the last successful identification.

[0011] The method for identifying intelligent terminal users based on multimodal perception, wherein the steps of detecting an identification request, obtaining a list of online devices in the target local area network, and generating a candidate user list for the target local area network; and scanning surrounding short-range wireless communication devices and obtaining the short-range wireless signal strength of the short-range wireless communication devices include: Upon detecting an identity verification request, the system retrieves a list of online devices in the target local area network and generates a list of candidate users for the target local area network. It also scans surrounding short-range wireless communication devices and obtains the short-range wireless signal strength of these devices. And periodically collect and acquire activity data and historical location association data of mobile terminals in the candidate user list of the target local area network; The activity data includes: screen on status data, application usage data, charging status data and / or sensor data; The screen on status data includes the time and duration of the device screen's most recent on-screen event. Application usage data records the recent usage frequency and duration of frequently used apps on the device; Charging status data records whether the device is charging and the duration of charging. Sensor data is recorded from the accelerometer and gyroscope sensors of the device to determine whether the device is stationary or moving.

[0012] The method for identifying smart terminal users based on multimodal perception, wherein the step of determining the owner of a mobile terminal as the current target user of the smart terminal when a mobile terminal simultaneously meets the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets a predetermined strength condition includes: When a target mobile terminal is detected to simultaneously meet the following conditions: it is in the candidate user list of the target local area network, its current short-range wireless signal strength meets the predetermined strength condition, its activity data meets the preset activity condition, and its historical location association data matches the area where the smart terminal is located, then the control determines the owner of the target mobile terminal that meets the conditions with high confidence as the current target user of the smart terminal. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively querying to determine the current target user of the smart terminal. The steps include: If a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, or if the short-range wireless signal strength of a mobile terminal meets the predetermined strength condition but its activity data does not meet the preset activity condition, or its historical location association data does not match the area where the smart terminal is located, then the control enters the secondary confirmation mechanism. When entering the secondary confirmation mechanism, priority is given to target mobile terminals with high activity data and high historical location correlation, and a secondary confirmation is actively requested to determine the current target user of the smart terminal.

[0013] A smart terminal user identification device based on multimodal perception, wherein the device comprises: The online device acquisition and wireless scanning module is used to detect identity recognition requests, acquire a list of online devices in the target local area network, and generate a list of candidate users in the target local area network; as well as scan surrounding short-range wireless communication devices and acquire the short-range wireless signal strength of the short-range wireless communication devices. The high-confidence identification module is used to determine the owner of the mobile terminal that meets the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets a predetermined strength condition. The secondary confirmation and identification module is used to determine the current target user of the smart terminal when a mobile terminal is detected that is not in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength conditions. The control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively asking questions. The interface matching and display module is used to control the entire interface of the smart terminal to seamlessly switch to the service interface and recommended content corresponding to the current target user profile, based on the current target user of the smart terminal.

[0014] A smart terminal includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs comprising the method for performing any one of the methods.

[0015] A computer-readable storage medium, wherein, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the methods described above.

[0016] The present invention provides a method, system, smart terminal and storage medium for user identification of smart terminals based on multimodal perception. The present invention utilizes Wi-Fi connection status perception and Bluetooth signal strength (RSSI) perception for decision-making and includes an uncertainty handling mechanism.

[0017] This invention provides stable proof of device presence through Wi-Fi connection status and precise distance perception through Bluetooth signal strength. The collaborative decision-making of these two technologies overcomes the limitations of single technologies, improving the confidence and reliability of identification. When confidence is insufficient, a secondary confirmation mechanism ensures the system can still provide available service, enhancing robustness. The entire process requires no user intervention and does not collect biometric data, fully respecting privacy. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the intelligent terminal user identification method based on multimodal perception provided in Embodiment 1 of the present invention.

[0020] Figure 2 This is a flowchart illustrating the intelligent terminal user identity recognition method based on multimodal perception provided in Embodiment 2 of the present invention.

[0021] Figure 3 The present invention provides a schematic diagram of an embodiment of a smart terminal user identification device based on multimodal perception.

[0022] Figure 4 This is a block diagram illustrating the internal structure of a smart terminal provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0024] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0025] Smart TVs, as current smart devices, have become the center of home entertainment, and their content recommendation systems heavily rely on user profiles. However, existing smart TVs have a fundamental flaw: they cannot automatically identify the specific user currently watching the TV. This leads to a series of problems: 1) User profile confusion: TV recommendations are often a mixture of the viewing history of multiple family members, failing to provide precise personalized services for any one individual, sometimes resulting in awkward situations like "Dad is watching TV and the recommendations are all cartoons." 2) Rigid interaction: TVs cannot adjust the interface, language preferences, or auxiliary functions based on the current user.

[0026] 3) The shortcomings of existing technical solutions include: A) Manual account switching requires active user operation, which is cumbersome and easily overlooked; B) Pure Bluetooth solution relies on continuous background broadcasting by the mobile APP. Once the APP is terminated by the system or the user turns off Bluetooth, the identification function immediately fails, resulting in poor reliability; C) Pure Wi-Fi solution can only determine whether the device is at home, but cannot accurately determine whether it is in front of the TV (e.g., it cannot distinguish between the bedroom and the living room); D) Camera visual recognition involves highly sensitive privacy issues, has low user acceptance, and is ineffective in poor lighting or when the user is facing away from the camera.

[0027] Therefore, there is an urgent need for a user identification solution that can accurately identify users, fully respect privacy, and is stable and reliable.

[0028] like Figure 1 As shown in Embodiment 1 of the present invention, a method for identifying the identity of a smart terminal user based on multimodal perception includes the following steps: Step S100: Detect an identity verification request, obtain a list of online devices in the target local area network, and generate a list of candidate users in the target local area network; and scan surrounding short-range wireless communication devices and obtain the short-range wireless signal strength of the short-range wireless communication devices; In this specific implementation, user identification is performed while a user is watching TV in front of a smart TV. The implementation includes a user's mobile terminal (such as a mobile phone), a smart TV terminal, a home router, and a cloud server. The target local area network in this embodiment is a Wi-Fi network, and the short-range wireless communication device is a Bluetooth device. This embodiment employs a collaborative decision-making process combining Wi-Fi status awareness and Bluetooth signal strength (RSSI) awareness.

[0029] Specifically, such as Figure 2As shown, when a user is about to watch a TV program on a smart TV, they initiate a user identification request through the smart TV terminal. Then, they can control the cloud server to obtain the list of online devices in the home Wi-Fi network and generate a candidate user list. If user A, user B, and user C are all in the home Wi-Fi network at this time, a candidate user list will be generated.

[0030] In this implementation, the smart TV terminal will also scan for nearby BLE (Bluetooth Low Energy) devices and obtain the Bluetooth signal strength (RSSI) of these Bluetooth devices, which will be used in subsequent steps to determine which Bluetooth device has the strongest Bluetooth signal.

[0031] Step S200: When it is detected that a mobile terminal simultaneously meets the conditions of being in the candidate user list of the target local area network and the current short-range wireless signal strength meets the predetermined strength condition, the owner of the mobile terminal that meets the conditions is determined to be the current target user of the smart terminal. In this embodiment of the invention, when a mobile terminal is detected to simultaneously meet the following conditions: it is in the target local area network candidate user list, and its current short-range wireless signal strength meets a predetermined strength condition; that is, when a mobile terminal is detected to simultaneously meet the two conditions of "being in the Wi-Fi list" and "having the strongest Bluetooth (BLE) signal" (condition one), the strongest Bluetooth signal indicates that it is closest to the smart TV, then the system of this embodiment of the invention determines with high confidence that the owner of the mobile terminal meeting the conditions is the current user. In other words, it is determined that the current user is carrying a mobile phone, has joined the home Wi-Fi local area network, has turned on Bluetooth, and is closest to the smart TV while watching television.

[0032] Step S300: When a mobile terminal is detected that is not in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism including seamlessly retaining the historical identity and actively asking questions to determine the current target user of the smart terminal. In this embodiment of the invention, when it is detected that the user's mobile phone is not in the target local area network candidate user list, that is, the user's mobile phone may be using mobile data and not connected to the home local area network, or it is connected to the home local area network but multiple users are at approximately the same distance from the smart TV, and there are multiple users with basically equal short-range wireless signal strengths, which do not meet the predetermined strength conditions, such as the mobile terminal with the highest invalid signal, then the control adopts a secondary confirmation mechanism including seamlessly retaining the historical identity and actively asking questions to determine the current target user of the smart terminal.

[0033] Specifically, in this embodiment of the invention, if multiple devices are detected in the candidate user list of the target local area network and the detected short-range wireless signal strength, such as Bluetooth signal strength, is close or cannot be determined (condition two), the system determines that the confidence level is low and enters the secondary confirmation mechanism.

[0034] The secondary confirmation mechanism in this embodiment may include: Option A (seamless reuse of historical identity) means that the smart TV system uses the user profile that was successfully identified last time to achieve seamless switching.

[0035] Option B (Proactive Voice Inquiry): The smart TV terminal proactively inquires about the user's identity by asking "Is this XXX?"

[0036] Confirm the identity of the end user, either through Option A or Option B.

[0037] Step S400: Based on the current target user of the smart terminal, control the entire interface of the smart terminal to seamlessly switch to the service interface and recommended content corresponding to the current target user profile.

[0038] In this embodiment, a smart terminal refers to a device with intelligent processing capabilities that can interact with users, such as a tablet computer or a smart TV, and serves as a carrier for service interfaces and recommended content.

[0039] The current target user refers to a user currently using a smart terminal, or a specific user identified by the terminal through identification (such as account login or biometrics) who requires targeted services.

[0040] The user profile is a three-dimensional user model built based on data such as the user's basic information (age, gender), behavioral habits (usage preferences, operation records), and demand characteristics (core needs, potential needs), used to accurately match services and content.

[0041] The service interface in this embodiment refers to the operation interface presented to the user on the smart terminal, including function entry points, interactive elements, etc. Different user profiles correspond to different interface layouts and function priorities.

[0042] The recommended content in this embodiment is personalized content pushed based on user profiles, such as news, applications, products, and services, which are tailored to the user's interests and needs.

[0043] Seamless switching in this embodiment refers to a smooth and natural switching process between the interface and content, without obvious lag or interruptions, so that the user is unaware of or barely aware of it, and it does not affect the user experience.

[0044] In this embodiment, after determining the target user currently using a smart terminal such as a smart TV through the above steps, the system controls the entire terminal interface (including functional layout and interaction design) in a smooth and lag-free manner to switch to a dedicated interface that highly matches the user's profile (user information, behavioral habits, core needs, etc.). At the same time, personalized content that fits the user's interests and needs is pushed to achieve precise service of "one interface and one content per person".

[0045] In this embodiment of the invention, after confirming the user's identity, the system synchronizes the user's identity to the cloud and updates the TV service interface and recommended content, realizing instant switching between device-user binding and user profile. Furthermore, the smart TV system records the identification data for subsequent optimization of the identification model.

[0046] This invention provides stable proof of device presence through the connection status of a target local area network, such as a home Wi-Fi network, while simultaneously providing accurate distance perception through the strength of short-range wireless signals, such as Bluetooth signals. The collaborative decision-making of these two technologies overcomes the limitations of single technologies, improving the confidence and reliability of identification. When confidence is insufficient, a secondary confirmation mechanism ensures the system can still provide available service, enhancing robustness. The entire process requires no user intervention and does not collect biometric data, fully respecting privacy.

[0047] In a further embodiment of the present invention, the method for identifying the identity of a smart terminal based on multimodal perception, wherein step S300 further includes: When multiple mobile terminals are detected to have equal wireless signal strength, or a wireless signal strength difference less than a predetermined value, or when it cannot be determined, the system enters a secondary confirmation mechanism; wherein, the secondary confirmation mechanism includes: The control system directly uses the user profile that was successfully identified last time to determine the current target user of the smart terminal; or the control system actively inquires about the user's identity to determine the current target user of the smart terminal. The active inquiry includes voice inquiry, sending a push notification to the user's mobile terminal to request confirmation, or displaying a user selection interface on the smart terminal.

[0048] In this embodiment of the invention, when multiple short-range wireless signals are found to be equal or similar by combining the local area network online list with the short-range wireless signals, or when it is impossible to determine which mobile terminal is closest to the smart TV, this embodiment of the invention will directly use the user profile that was successfully identified last time or actively ask the user to determine the target user identity of the current smart terminal.

[0049] In a further embodiment of the present invention, step S400 specifically includes: S401. When the identity of the current target user of the smart terminal is confirmed, the control will synchronize the confirmed target user identity to the cloud. S402. Based on the current target user of the determined smart terminal, control the entire interface of the smart terminal to seamlessly switch to the service interface corresponding to the current target user profile and display recommended content. S403, and record the identification data for subsequent user identification.

[0050] The following detailed description of the multimodal perception-based intelligent terminal user identification method of the present invention is provided through specific application examples. Specific application examples include... Figure 2 As shown, taking a family watching television as an example, if a family has three members—father, mother, and child—each owns a smartphone, and all these phones are connected to the home Wi-Fi network and have Bluetooth enabled, the specific implementation of the method embodiment of this invention includes the following steps: Step 1: The smart TV initiates an identification request. Specifically: The child comes home from school and turns on the smart TV in the living room to watch cartoons. The smart TV system detects user activity and initiates an identity verification request.

[0051] Step 2: Obtain the list of online devices on the home Wi-Fi network from the cloud. Specifically, control the cloud server to query the home Wi-Fi network and find that Dad's phone 1, Mom's phone 2, and child's phone 3 are all online. Then proceed to Step 3: Generate a "candidate user list" containing these three family members.

[0052] Step 4: The smart TV scans for nearby Bluetooth (BLE) devices and obtains the Bluetooth signal strength (RSSI). Specifically, the smart TV terminal is controlled to start scanning for nearby Bluetooth (BLE) devices. Since the child is sitting in front of the smart TV, his mobile phone 3 is closest to the smart TV. In this embodiment, the smart TV terminal detects that the Bluetooth signal strength (RSSI) emitted by the child's mobile phone 3 is the strongest, while the father and mother's mobile phones may be in other rooms (such as the study or kitchen), and their Bluetooth signal strength is relatively weak.

[0053] Step 5: Decision-making. Specifically, if condition 1 is met: that is, if a device simultaneously meets the two conditions of "in the Wi-Fi list" and "strongest Bluetooth BLE signal", then proceed to step 6. If condition two is met: the signal strengths of multiple devices are similar or cannot be determined, proceed to step 8; Step 6: High confidence level, proceed to step 7; Step 7: Confirm the device owner. Specifically, in this embodiment, the system detects that the child's phone 3 simultaneously meets the conditions of being "in the Wi-Fi list" and having "strongest Bluetooth BLE signal." Therefore, the system in this embodiment determines with high confidence that the person currently sitting in front of the smart TV is a child. Then proceed to step 10; Step 8: Low confidence level, proceed to step 9; Step 9, Secondary Confirmation Mechanism: The secondary confirmation mechanism in this embodiment includes: Option A (seamless reuse of historical identity) means that the smart TV system uses the user profile that was successfully identified last time to achieve seamless switching.

[0054] Option B (Proactive Voice Inquiry): The smart TV terminal proactively inquires about the user's identity by asking "Is this XXX?"

[0055] Verify the end-user's identity using either Option A or Option B. Then proceed to step 10.

[0056] Step 10: Synchronize user identity to the cloud and update TV service interface and recommendations. Specifically, in this embodiment, the system immediately synchronizes the identified user identity (child) to the cloud and quickly switches the TV interface to the child's exclusive user profile. The recommended content is also immediately updated to cartoons and educational programs suitable for children.

[0057] In this embodiment, the child does not need to perform any manual operation. The smart TV automatically recognizes the child's identity and provides a personalized viewing experience, achieving a seamless switching process that is "invisible to technology."

[0058] Step 11: Record the identification data for model optimization.

[0059] In another embodiment, for example, when a father and child are both in the living room, with the father sitting on the sofa and the child sitting on the carpet, and both are very close to the smart TV, the specific implementation of the method embodiment of the present invention includes the following steps: Step 11: Identification Request Initiation Step - Control the smart TV to initiate an identification request.

[0060] Step 12: Obtaining the list of Wi-Fi online devices. The cloud server detected that both the father's and child's phones were online.

[0061] Step 13: Bluetooth signal strength scanning step. The smart TV terminal scans for nearby Bluetooth (BLE) devices and finds that the Bluetooth signal strength of both the father's and child's phones is very strong and the values ​​are similar. At this time, the system has difficulty accurately determining which signal is the strongest.

[0062] Step 14, Decision-making step: In the low-confidence identification stage, since the signal strength of multiple devices is similar, the system judges the confidence level to be low, and then the control enters the secondary confirmation mechanism.

[0063] Step 15, Secondary Confirmation Step, includes: Option A (seamless reuse of historical identity) means that the smart TV system uses the user profile from the last successful identification to achieve seamless switching; In this embodiment, the steps of Solution B (active voice inquiry) involve the smart TV terminal using voice to ask a question such as, "Is Dad watching TV?". If the father answers "No, it's the child" or the child answers "It's me," then the system in this embodiment of the invention can confirm the identity of the end user.

[0064] To confirm the identity of the end user, in this embodiment, when the child answers "It is me", the system of this embodiment confirms that the current user is the child.

[0065] Step 16, Synchronization and Update Steps: In this embodiment, the system synchronizes the user's identity (child) to the cloud and updates the TV service interface and recommended content to the child's exclusive profile.

[0066] As can be seen from the above embodiments, this invention can accurately identify user identity in different scenarios (high confidence or low confidence) through a collaborative perception decision-making mechanism, and combined with a secondary confirmation mechanism, provide a seamless, reliable and privacy-respecting personalized TV experience, thus providing convenience for users.

[0067] Furthermore, in the aforementioned method for identifying the identity of intelligent terminal users based on multimodal perception, the target local area network includes: a Wi-Fi network, a ZigBee network, or a Z-Wave network; The wireless signals include Bluetooth signals, UWB ultra-wideband wireless signals, or NFC signals.

[0068] In another embodiment of the present invention, the target local area network can be a Wi-Fi local area network, or other local area network connection technologies such as ZigBee and Z-Wave. The short-range wireless communication device can be a Bluetooth device, or a more precise ultra-wideband (UWB) ranging technology device, or, on a device supporting UWB, it can be used as a higher confidence level criterion. ZigBee and Z-Wave are both wireless communication technologies used in smart homes and the Internet of Things (IoT).

[0069] For example, UWB ranging technology provides more accurate distance information compared to Bluetooth RSSI, thereby improving the confidence of recognition and reducing the frequency of secondary confirmation mechanisms in complex scenarios where multiple users are close to the TV, thus optimizing the user experience. At the same time, replacing it with other local area network (LAN) connection technologies also enhances the system's compatibility and adaptability, enabling its application in a wider range of smart home ecosystems.

[0070] For example, if home smart devices generally support UWB technology, the implementation steps of this invention are as follows: S1. Identification request initiated: When the child comes home from school, they turn on the smart TV in the living room.

[0071] S2, Wi-Fi Online Device List Acquisition: Obtains a list of online devices in the home Wi-Fi network from the cloud server, including the mobile phones of the father, mother, and children.

[0072] S3. Ultra-wideband (UWB) ranging and signal strength acquisition: Control the smart TV terminal to scan for nearby UWB devices and obtain their precise distance information. Since the child is sitting in front of the TV, his phone is closest to the TV's UWB and has the strongest signal, while the parents' phones are in other rooms and are farther away.

[0073] S4. Decision Judgment: In this embodiment, the system detects that the child's mobile phone simultaneously meets the two conditions of "being on the Wi-Fi list" and "being closest to / strongest in UWB signal". Therefore, the system of this embodiment determines with a very high degree of confidence that the person currently sitting in front of the TV is a child.

[0074] S5. Synchronization and Update: In this embodiment, the system immediately synchronizes the identified user identity (child) to the cloud and quickly switches the TV interface to the child's exclusive user profile. The recommended content is also immediately updated to cartoons and educational programs suitable for children.

[0075] It is evident that UWB technology can more accurately identify the closest user even when multiple users are close to the TV, thereby reducing the need for secondary confirmation due to similar signal strength and further improving the seamless experience and efficiency of the identification process.

[0076] In a further embodiment of the present invention, the method for identifying the identity of a smart terminal based on multimodal perception, wherein the step of controlling a secondary confirmation mechanism including seamlessly retaining historical identity and active inquiry to determine the current target user of the smart terminal when a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, further includes: If a mobile terminal is not detected in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength condition, the current target user of the smart terminal is determined by combining the mobile terminal device's past time habit data and the motion sensor data of the mobile terminal after the last successful identification.

[0077] In this embodiment of the invention, when a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, more weighted factors can be introduced, such as time habits (after 9 pm is usually the homeowner's time) and device motion sensor data since the last successful identification (to determine which device is more likely to have moved to the living room). This embodiment introduces additional contextual information and historical data on top of the two basic judgment conditions of Wi-Fi connection status and Bluetooth signal strength to improve the accuracy and intelligence of decision-making.

[0078] Specifically, this embodiment introduces multi-dimensional weighted factors to make the decision-making algorithm more intelligent and robust. For example, combining time habits can effectively solve the ambiguity in scenarios where multiple users are present but only one user is actually watching TV; while device motion sensor data can help determine whether the user has moved from another room to the TV, further improving the accuracy of recognition, especially in low-confidence scenarios, helping the system to make a more correct initial judgment, thereby reducing the frequency of secondary confirmation and improving the user experience.

[0079] A specific example combining time habits and device motion sensor data is as follows: In a family with a father and child, the father typically watches news or movies after 9 PM, while the child watches cartoons during the day. Scenario 1: After 9 PM, both the father and child are in the living room; the embodiments of this invention include the following steps: S11. Identification request initiated. For example, at 9:30 pm, the user is watching a smart TV, and the smart TV initiates an identification request.

[0080] S12. Obtaining the list of Wi-Fi online devices: In this embodiment, the cloud server detected that both the father's and child's mobile phones were in the list of Wi-Fi online devices.

[0081] S13. Bluetooth signal strength scan: Control the smart TV terminal to scan for nearby Bluetooth BLE devices. It was found that the Bluetooth signal strength of both the father's and child's mobile phones was very strong and the values ​​were similar, making it difficult for the system to accurately determine which signal was the strongest.

[0082] S14. Then, this embodiment makes a decision (introducing weighted factors). Time habit weighting: In this embodiment, the system detects that the current time is 9:30 pm. According to historical data, this time period is usually when the father watches TV.

[0083] In this embodiment, the system queries the last successful identification data. The father's mobile phone motion sensor data shows that he moved from the study to the living room, while the child's mobile phone remained stationary (possibly playing on the sofa).

[0084] S15. Based on the comprehensive decision-making steps, combined with the similarity of Bluetooth signal strength, time habits (after 9 pm is the father's viewing time), and the movement trajectory of the father's mobile phone, the system of this embodiment tends to determine that the current user is the father.

[0085] S16. If there is still uncertainty, a second confirmation will be performed. That is, in this embodiment, if the system still considers the confidence level insufficient, it may ask by voice, "Is Dad watching TV?".

[0086] S17. Confirm the identity of the end user. When the father answers "yes", the system in this embodiment of the invention confirms that the current user is the father.

[0087] S18. During synchronization and updating, the system of this embodiment synchronizes the user identity (father) to the cloud and updates the TV service interface and recommended content to a father’s exclusive profile (such as news and movies).

[0088] As can be seen, by introducing time habits and device motion sensor data as weighting factors, the embodiments of the present invention can make more intelligent judgments even in low-confidence scenarios where Bluetooth signal strength is similar, reducing the reliance on secondary confirmation mechanisms and improving the accuracy of recognition and the seamless experience.

[0089] In this embodiment of the invention, a cloud server is used to obtain the list of online devices in the home Wi-Fi network, which is a deployment mode set up in the cloud. Of course, in another embodiment of the invention, a fully localized deployment mode can also be implemented, where the entire decision-making system is deployed locally using the computing power of a TV or home gateway, without relying on the cloud, thus reducing latency and meeting the privacy requirements of data not leaving the home. Examples of local deployment include a smart TV in a home having sufficient computing power, or a smart gateway deployed in the home, capable of handling the computational task of identity recognition.

[0090] Step S21: Identification Request Initiation Step. When the child comes home from school and turns on the smart TV in the living room, the smart TV system detects the user activity and initiates an identity verification request.

[0091] Step S22: Obtain the list of online Wi-Fi devices locally. This means controlling the smart TV (or home gateway) to communicate directly with the home router through the local network interface to obtain the list of online devices in the current home Wi-Fi network and generate a "candidate user list" (including the mobile phones of dad, mom and child).

[0092] Step S23: Locally scan Bluetooth devices and obtain the RSSI Bluetooth signal strength. Specifically, control the smart TV terminal to scan for nearby Bluetooth BLE devices and obtain the Bluetooth signal strength (RSSI) of these devices. Since the child is sitting in front of the TV, his mobile phone is closest to the TV, and the smart TV terminal detects that the Bluetooth signal strength (RSSI) emitted by the child's mobile phone is the strongest.

[0093] Step S24, Local Decision-Making Step: Specifically, a high-confidence identification process is performed. The television system detects the child's mobile phone locally, which simultaneously meets the two conditions of being "in the Wi-Fi list" and having "strongest BLE signal." Therefore, the system determines locally with high confidence that the person currently sitting in front of the television is the child.

[0094] Step S25, Local Synchronization and Update: Specifically, after the system confirms the user's identity (child) locally, it immediately switches the smart TV interface to the child's exclusive user profile, and the recommended content is immediately updated to cartoons and educational programs suitable for children. All user profiles and recommendation data are stored locally without interaction with the cloud.

[0095] Step S26, Seamless Experience and Privacy Protection: Specifically, the smart TV in this implementation automatically recognizes the child's identity and provides a personalized viewing experience without any manual operation. The entire recognition process and data processing are completed locally, and the user's personal data is not uploaded to the cloud, fully protecting privacy.

[0096] As can be seen from this embodiment, in local deployment mode, the system can achieve fast, seamless, and highly privacy-protected user identification, and can work normally even without an external network connection.

[0097] As can be seen from the above, in scenarios where multiple family members share a smart TV, the multimodal perception user identification system proposed in this invention can effectively solve the problem that traditional TVs cannot automatically identify the current viewer's identity. For example, when a child comes home from school and turns on the TV, the system can identify with high confidence that the child is watching by collaboratively sensing the Wi-Fi online status and Bluetooth signal strength, and immediately switch to the child's exclusive content recommendation interface to play cartoons. Even in low-confidence scenarios, such as when the father and child are both in the living room and the Bluetooth signal strength is similar, the system can still conduct a secondary confirmation through voice inquiry, ultimately accurately identifying the current user.

[0098] However, in practical applications, especially in complex family environments with multiple generations or multiple tenants, a specific technical problem may arise: interference and misidentification of zombie devices.

[0099] "Zombie devices" refer to devices that are permanently connected to the home Wi-Fi network but are not actually in the TV viewing area, and may even be idle or forgotten in some corner, yet their Wi-Fi connection remains online, and their Bluetooth function may be occasionally activated or continuously broadcast a weak signal (e.g., old mobile phones, smartwatches, smart speakers, etc.). The presence of these devices will interfere with the "Wi-Fi online device list acquisition" and "Bluetooth signal strength scanning" steps in the present invention.

[0100] Therefore, in order to solve the problem of interference and misidentification by "zombie devices," in a further embodiment of the present invention, the technical solution of the above embodiments can be deeply optimized by introducing a device activity and location association evaluation mechanism. Specifically, based on the existing Wi-Fi connection status perception and Bluetooth signal strength (RSSI) perception, an evaluation of the device's recent activity (such as screen on, application usage, charging status, etc.) and historical location association (such as which room the device is usually in and when it is used) is added to more accurately filter out the real "candidate users" and reduce the interference of "zombie devices." Therefore, in another embodiment of the present invention, the intelligent terminal user identification method based on multimodal perception includes the following steps: S101: Detect an identity verification request, obtain a list of online devices in the target local area network, and generate a list of candidate users in the target local area network; and scan surrounding short-range wireless communication devices and obtain the short-range wireless signal strength of the short-range wireless communication devices; S102, and periodically collect and acquire activity data and historical location association data of mobile terminals in the candidate user list of the target local area network; The activity data includes: screen on status data, application usage data, charging status data and / or sensor data; The screen on status data includes the time and duration of the device screen's most recent on-screen event. Application usage data records the recent usage frequency and duration of frequently used apps on the device; Charging status data records whether the device is charging and the duration of charging. Sensor data is recorded from the accelerometer and gyroscope sensors of the device to determine whether the device is stationary or moving.

[0101] In this embodiment of the invention, when the accompanying APP of the user's mobile terminal (such as a mobile phone, tablet, etc.) is running in the background, in addition to continuously broadcasting Bluetooth signals, it should also periodically collect and upload the device's activity data to the cloud server. This data includes, but is not limited to: Screen on status: Records the time and duration of the device screen last turned on.

[0102] Application usage: Records the recent usage frequency and duration of frequently used apps (such as social, video, and game apps) on the device.

[0103] Charging status: Records whether the device is charging and the duration of charging.

[0104] Sensor data: Records data from sensors such as accelerometers and gyroscopes to determine whether the device is stationary or moving.

[0105] In this embodiment of the invention, a cloud server is responsible for receiving, storing, and analyzing this activity data, and establishing an "activity profile" for each registered device.

[0106] In this embodiment of the invention, the establishment of historical location association data refers to the process of associating and recording the current user's device with the location of the television terminal after the system successfully identifies the user for the first time, or after the user explicitly specifies their identity in the secondary confirmation mechanism. For example, if a child is identified while watching TV in the living room, the system will record that "the child's mobile phone is usually active in the living room area."

[0107] Furthermore, this invention also incorporates long-term learning, meaning the system continuously learns the historical location information of each device, such as the long-term average value of Bluetooth RSSI, the trend of Wi-Fi signal strength changes, etc., and combines user feedback to establish an "activity heat map" or "resident area" for each device in different areas of the home.

[0108] In specific implementation, when a user watches a TV program on a smart TV, the smart TV terminal initiates a user identification request. First, a list of Wi-Fi online devices is obtained and preliminarily filtered. Specifically, this list can be obtained from a cloud server. Based on this, the system performs preliminary filtering according to each device's "activity profile." For example, if a device has no screen-on or application usage records in the past 24 hours and has been inactive for a long time, it is marked as a "low-activity device." In this embodiment, its weight will be significantly reduced when generating the "candidate user list," or it may even be directly excluded.

[0109] This embodiment relates to scanning Bluetooth devices and obtaining Bluetooth signal strength (RSSI). Specifically, the smart TV terminal will scan for nearby Bluetooth BLE devices and obtain the Bluetooth signal strength (RSSI) of these devices.

[0110] S103. When it is detected that a target mobile terminal simultaneously meets the following conditions: it is in the candidate user list of the target local area network, the current short-range wireless signal strength meets the predetermined strength condition, the activity data of the target mobile terminal meets the preset activity condition, and its historical location association data matches the area where the smart terminal is located, then the control determines with high confidence that the owner of the target mobile terminal that meets the conditions is the current target user of the smart terminal. In this embodiment, if a mobile terminal device simultaneously meets the conditions of being in the "Wi-Fi list" and having the "strongest Bluetooth BLE signal", and the mobile terminal device has a high "activity score", and its "historical location association" also highly matches the area where the smart TV is located, then the system determines the device owner as the current user with high confidence.

[0111] The activity score in this embodiment is a comprehensive score of the user's mobile terminal device based on data such as screen on / off status, application usage, charging status, and / or sensor data. A high score indicates that the user's mobile terminal device has a high usage rate.

[0112] The historical location association refers to the location information of the mobile terminal device recorded by the system in the past, which highly matches the area where the smart TV is located, indicating that the device owner often moves around the area where the TV is located.

[0113] In this embodiment, when a mobile terminal device (such as a mobile phone) simultaneously meets two basic conditions: it appears in the Wi-Fi list of the smart TV (in the same network environment) and its Bluetooth BLE signal is the strongest among all devices (closest in physical distance); it also meets two auxiliary conditions: the device has a high activity score and the device's historical location highly matches the area where the smart TV is located (the owner is often active in this area), then the system will determine with a very high degree of confidence that the owner of this mobile terminal device is the user currently using the smart TV.

[0114] Thus, this invention uses the basic conditions of "network + distance" combined with the auxiliary conditions of "activity level + location" for multi-dimensional verification, significantly reducing the probability of false positives and ensuring that the current user is a genuine user. Furthermore, this invention does not require manual verification by the user (such as entering a password or scanning a code); the system automatically completes multi-condition detection without interrupting the user's TV usage process. Moreover, this invention takes into account network environment, physical distance, usage habits, and activity range, enabling quick and accurate identification whether the user is using the TV at home or is temporarily near it.

[0115] S104. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, or a mobile terminal whose short-range wireless signal strength meets the predetermined strength condition but whose activity data does not meet the preset activity condition, or whose historical location association data does not match the area where the smart terminal is located, the control enters the secondary confirmation mechanism. When entering the secondary confirmation mechanism, priority is given to target mobile terminals with high activity data and high historical location correlation, and a secondary confirmation is actively requested to determine the current target user of the smart terminal.

[0116] In this embodiment of the invention, if the signal strengths of multiple devices are similar or cannot be determined, or if a device has the strongest signal but its activity score is low or its historical location correlation is inconsistent, the system considers the confidence level low and enters a secondary confirmation mechanism. In this embodiment, for devices marked as "low-activity devices" whose Bluetooth signal strength is not significantly the strongest, even if their Wi-Fi is online, they will be assigned a very low weight by the system, thereby effectively suppressing their interference with the identification results.

[0117] In this embodiment of the invention, when entering the secondary confirmation mechanism, devices with high activity and high historical location correlation will be given priority for voice inquiry, such as: "Is [the owner of the device with high activity and historical location in the living room] watching TV?", which can improve the efficiency and accuracy of secondary confirmation.

[0118] As can be seen, the optimized embodiment of this invention, by introducing a "device activity and location association evaluation mechanism," can effectively identify and suppress interference from "zombie devices." Activity data ensures that only devices currently in use are prioritized, while historical location association further narrows the identification range, avoiding misjudgments caused by occasional signal fluctuations from old or idle devices. This makes the system more accurate in generating the "candidate user list" and more reliable in decision-making, thereby reducing unnecessary secondary confirmations, improving the seamless user experience and the accuracy of personalized recommendations, while avoiding complex artificial intelligence algorithms. It is highly rational and practical, greatly facilitating user experience.

[0119] Exemplary device like Figure 3 As shown, this embodiment of the invention provides a smart terminal user identification device based on multimodal perception, comprising: The online device acquisition and wireless scanning module 310 is used to detect an identity recognition request, acquire a list of online devices in the target local area network, and generate a list of candidate users in the target local area network; and scan surrounding short-range wireless communication devices and acquire the short-range wireless signal strength of the short-range wireless communication devices. The high confidence recognition module 320 is used to determine the owner of the mobile terminal that meets the conditions as the current target user of the smart terminal when it detects that a mobile terminal simultaneously meets the conditions of being in the candidate user list of the target local area network and the current short-range wireless signal strength meets the predetermined strength condition. The secondary confirmation and identification module 330 is used to control the current target user of the smart terminal when a mobile terminal is detected that is not in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength condition. The control adopts a secondary confirmation mechanism including seamlessly retaining the historical identity and actively asking questions. The interface matching and display module 340 is used to control the entire interface of the smart terminal to seamlessly switch to the service interface and recommended content corresponding to the current target user profile based on the current target user of the smart terminal, as described above.

[0120] Based on the above embodiments, the present invention also provides a smart terminal, the principle block diagram of which can be as follows: Figure 4 As shown, the intelligent terminal includes a processor, memory, network interface, display screen, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a multimodal perception-based intelligent terminal user identification method. The database of the intelligent terminal stores the multimodal perception-based intelligent terminal user identification program.

[0121] Those skilled in the art will understand that Figure 4 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the smart terminal to which the present invention is applied. A specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] In one embodiment, a smart terminal is provided, including a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Upon detecting an identity verification request, the system retrieves a list of online devices in the target local area network and generates a list of candidate users for the target local area network; it also scans for nearby short-range wireless communication devices and obtains the short-range wireless signal strength of these devices. When a mobile terminal is detected to simultaneously meet the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets the predetermined strength condition, the owner of the mobile terminal that meets the conditions is determined to be the current target user of the smart terminal. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively asking questions to determine the current target user of the smart terminal. Based on identifying the current target user of the smart terminal, the system seamlessly switches the entire interface of the smart terminal to the service interface and recommended content corresponding to the current target user profile, as described above.

[0123] The step of determining the current target user of the smart terminal by controlling a secondary confirmation mechanism that includes seamlessly retaining historical identity and actively querying when a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength meets the predetermined strength condition includes: When multiple mobile terminals are detected to have equal wireless signal strength, or a wireless signal strength difference less than a predetermined value, or when it cannot be determined, the system enters a secondary confirmation mechanism; wherein, the secondary confirmation mechanism includes: The control system directly uses the user profile that was successfully identified last time to determine the current target user of the smart terminal; Alternatively, the control system may proactively inquire about the user's identity to determine the current target user of the smart terminal. This proactive inquiry may include voice inquiry, sending a push notification to the user's mobile terminal to request confirmation, or displaying a user selection interface on the smart terminal.

[0124] The step of seamlessly switching the entire interface of the smart terminal to a service interface and recommended content corresponding to the current target user profile, based on determining the current target user of the smart terminal, includes: Once the identity of the current target user on the smart terminal is confirmed, the control will synchronize the confirmed target user identity to the cloud. Based on identifying the current target user of the smart terminal, the system seamlessly switches the entire interface of the smart terminal to the service interface and recommended content corresponding to the current target user profile. The identification data will be recorded for subsequent user identification.

[0125] The target local area network includes: a Wi-Fi network, a ZigBee network, or a Z-Wave network; The wireless signals include Bluetooth signals, UWB ultra-wideband wireless signals, or NFC signals.

[0126] The step of determining the current target user of the smart terminal by controlling a secondary confirmation mechanism that includes seamlessly retaining historical identity and active inquiry when a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition further includes: If a mobile terminal is not detected in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength condition, the current target user of the smart terminal is determined by combining the mobile terminal device's past time habit data and the motion sensor data of the mobile terminal after the last successful identification.

[0127] The steps of detecting an identity verification request, obtaining a list of online devices in the target local area network, generating a candidate user list for the target local area network, and scanning surrounding short-range wireless communication devices and obtaining the short-range wireless signal strength of the short-range wireless communication devices include: Upon detecting an identity verification request, the system retrieves a list of online devices in the target local area network and generates a list of candidate users for the target local area network. It also scans surrounding short-range wireless communication devices and obtains the short-range wireless signal strength of these devices. And periodically collect and acquire activity data and historical location association data of mobile terminals in the candidate user list of the target local area network; The activity data includes: screen on status data, application usage data, charging status data and / or sensor data; The screen on status data includes the time and duration of the device screen's most recent on-screen event. Application usage data records the recent usage frequency and duration of frequently used apps on the device; Charging status data records whether the device is charging and the duration of charging. Sensor data is recorded from the accelerometer and gyroscope sensors of the device to determine whether the device is stationary or moving.

[0128] The step of determining the owner of a mobile terminal that meets the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets a predetermined strength condition includes: When a target mobile terminal is detected to simultaneously meet the following conditions: it is in the candidate user list of the target local area network, its current short-range wireless signal strength meets the predetermined strength condition, its activity data meets the preset activity condition, and its historical location association data matches the area where the smart terminal is located, then the control determines the owner of the target mobile terminal that meets the conditions with high confidence as the current target user of the smart terminal. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively querying to determine the current target user of the smart terminal. The steps include: If a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, or if the short-range wireless signal strength of a mobile terminal meets the predetermined strength condition but its activity data does not meet the preset activity condition, or its historical location association data does not match the area where the smart terminal is located, then the control enters the secondary confirmation mechanism. When entering the secondary confirmation mechanism, priority is given to target mobile terminals with high activity data and high historical location correlation. The secondary confirmation is actively requested to determine the current target user of the smart terminal, as described above.

[0129] In other embodiments, this application proposes a computer-readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described method.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0131] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for identifying the identity of a smart terminal user based on multimodal perception, characterized in that, include: Upon detecting an identity verification request, the system retrieves a list of online devices in the target local area network and generates a list of candidate users for the target local area network. It also scans surrounding short-range wireless communication devices and obtains the short-range wireless signal strength of these devices. When a mobile terminal is detected to simultaneously meet the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets the predetermined strength condition, the owner of the mobile terminal that meets the conditions is determined to be the current target user of the smart terminal. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively asking questions to determine the current target user of the smart terminal. Based on identifying the current target user of the smart terminal, the system seamlessly switches the entire interface of the smart terminal to the service interface and recommended content corresponding to the current target user profile.

2. The method for identifying intelligent terminal users based on multimodal perception according to claim 1, characterized in that, When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength meets the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively querying to determine the current target user of the smart terminal. The steps include: When multiple mobile terminals are detected to have equal wireless signal strength, or a wireless signal strength difference less than a predetermined value, or when it cannot be determined, the system enters a secondary confirmation mechanism; wherein, the secondary confirmation mechanism includes: The control system directly uses the user profile that was successfully identified last time to determine the current target user of the smart terminal; Alternatively, the control system may proactively inquire about the user's identity to determine the current target user of the smart terminal. This proactive inquiry may include voice inquiry, sending a push notification to the user's mobile terminal to request confirmation, or displaying a user selection interface on the smart terminal.

3. The method for identifying intelligent terminal users based on multimodal perception according to claim 1, characterized in that, The step of seamlessly switching the entire interface of the smart terminal to a service interface and recommended content corresponding to the current target user profile, based on determining the current target user of the smart terminal, includes: Once the identity of the current target user on the smart terminal is confirmed, the control will synchronize the confirmed target user identity to the cloud. Based on identifying the current target user of the smart terminal, the system seamlessly switches the entire interface of the smart terminal to the service interface and recommended content corresponding to the current target user profile. The identification data will be recorded for subsequent user identification.

4. The method for identifying intelligent terminal users based on multimodal perception according to claim 1, characterized in that, The target local area network includes: a Wi-Fi network, a ZigBee or Z-Wave network; The wireless signals include Bluetooth signals, UWB ultra-wideband wireless signals, or NFC signals.

5. The method for identifying intelligent terminal users based on multimodal perception according to claim 1, characterized in that, The step of controlling the use of a secondary confirmation mechanism, including seamless reuse of historical identity and active inquiry, to determine the current target user of the smart terminal when a mobile terminal is not detected in the target local area network candidate user list and the current short-range wireless signal strength does not meet the predetermined strength condition, further includes: If a mobile terminal is not detected in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength condition, the current target user of the smart terminal is determined by combining the mobile terminal device's past time habit data and the motion sensor data of the mobile terminal after the last successful identification.

6. The method for identifying intelligent terminal users based on multimodal perception according to claim 1, characterized in that, The system detects an identity verification request, obtains a list of online devices in the target local area network, and generates a list of candidate users in the target local area network. The steps of scanning surrounding short-range wireless communication devices and obtaining the short-range wireless signal strength of these devices include: Upon detecting an identity verification request, the system retrieves a list of online devices in the target local area network and generates a list of candidate users for the target local area network. It also scans surrounding short-range wireless communication devices and obtains the short-range wireless signal strength of these devices. And periodically collect and acquire activity data and historical location association data of mobile terminals in the candidate user list of the target local area network; The activity data includes: screen on status data, application usage data, charging status data and / or sensor data; The screen on status data includes the time and duration of the device screen's most recent on-screen event. Application usage data records the recent usage frequency and duration of frequently used apps on the device; Charging status data records whether the device is charging and the duration of charging. Sensor data is recorded from the accelerometer and gyroscope sensors of the device to determine whether the device is stationary or moving.

7. The method for identifying intelligent terminal users based on multimodal perception according to claim 6, characterized in that, The step of determining the owner of a mobile terminal that meets the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets a predetermined strength condition includes: When a target mobile terminal is detected to simultaneously meet the following conditions: it is in the candidate user list of the target local area network, its current short-range wireless signal strength meets the predetermined strength condition, its activity data meets the preset activity condition, and its historical location association data matches the area where the smart terminal is located, then the control determines the owner of the target mobile terminal that meets the conditions with high confidence as the current target user of the smart terminal. When a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, the control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively querying to determine the current target user of the smart terminal. The steps include: If a mobile terminal is detected that is not in the candidate user list of the target local area network and whose current short-range wireless signal strength does not meet the predetermined strength condition, or if the short-range wireless signal strength of a mobile terminal meets the predetermined strength condition but its activity data does not meet the preset activity condition, or its historical location association data does not match the area where the smart terminal is located, then the control enters the secondary confirmation mechanism. When entering the secondary confirmation mechanism, priority is given to target mobile terminals with high activity data and high historical location correlation, and a secondary confirmation is actively requested to determine the current target user of the smart terminal.

8. A smart terminal user identification device based on multimodal perception, characterized in that, The device includes: The online device acquisition and wireless scanning module is used to detect identity recognition requests, acquire a list of online devices in the target local area network, and generate a list of candidate users in the target local area network; as well as scan surrounding short-range wireless communication devices and acquire the short-range wireless signal strength of the short-range wireless communication devices. The high-confidence identification module is used to determine the owner of the mobile terminal that meets the conditions of being in the candidate user list of the target local area network and having a current short-range wireless signal strength that meets a predetermined strength condition. The secondary confirmation and identification module is used to determine the current target user of the smart terminal when a mobile terminal is detected that is not in the candidate user list of the target local area network and the current short-range wireless signal strength does not meet the predetermined strength conditions. The control adopts a secondary confirmation mechanism that includes seamlessly retaining the historical identity and actively asking questions. The interface matching and display module is used to control the entire interface of the smart terminal to seamlessly switch to the service interface and recommended content corresponding to the current target user profile, based on the current target user of the smart terminal.

9. A smart terminal, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, wherein the one or more programs include methods for performing any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-7.

Images