Security control method and device, robot, storage medium and program product

Abstract

This disclosure provides a security control method, device, robot, storage medium, and computer program product. The method includes: matching facial image information collected by a sensing device with preset authorized facial image information; in the case of matching failure, controlling the robot to perform identity inquiry processing on the target person corresponding to the failed facial image information; controlling the robot to follow the target person and collecting audio data and first image data corresponding to the target person through the sensing device; determining the security risk level of the target person based on the audio data and the first image data; and performing corresponding security protection processing based on the security risk level. This disclosure realizes proactive identity verification and follow-up processing of unauthorized personnel, effectively reducing the false alarm rate of security, eliminating monitoring blind spots, and improving the initiative, risk assessment, and accuracy of security.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more particularly to a security control method, device, robot, storage medium, and computer program product. Background Technology

[0002] With the increasing demand for home security, home security equipment and robot technologies have acquired the ability to identify and warn strangers. For example, robots can capture images through cameras, identify strangers based on facial recognition technology, and trigger alarms. Currently, robots can only identify strangers through static image capture and cannot actively verify their identities, resulting in a high false alarm rate and a lack of accuracy in security risk warnings. Furthermore, robots cannot follow the movement of strangers, easily creating blind spots and posing security risks. Summary of the Invention

[0003] This disclosure provides a security control method, device, robot, storage medium, and computer program product.

[0004] According to a first aspect of this disclosure, a security control method is provided, applied to a robot, the method comprising: matching facial image information collected by a sensing device with preset authorized facial image information; in the event of a matching failure, controlling the robot to perform identity inquiry processing on a target person, wherein the target person is the person corresponding to the facial image information of the failed match; controlling the robot to follow the target person, and collecting audio data and first image data corresponding to the target person through the sensing device; determining the security risk level of the target person based on the audio data and the first image data; and performing corresponding security protection processing based on the security risk level.

[0005] Optionally, determining the security risk level of the target person based on the audio data and the first image data includes: using a security risk identification model to determine the target person's response identification information to the identity inquiry processing based on the audio data; using the security risk identification model to determine the target person's behavioral identification information based on the first image data; and determining the security risk level based on the response identification information, or based on both the response identification information and the behavioral identification information.

[0006] Optionally, determining the security risk level based on the response identification information includes: when the response identification information indicates that the identity information of the target person and the person being visited are unclear, or when there are contradictions in the response content of the target person, the security risk level is determined to be a high-risk level.

[0007] Optionally, determining the security risk level based on the response identification information and the behavior identification information includes: if the response identification information indicates that the target person's emotional state is abnormal when responding, and the behavior identification information indicates that the target person is exhibiting behavior to evade image acquisition, then the security risk level is determined to be a high-risk level.

[0008] Optionally, determining the security risk level based on the response identification information and the behavior identification information includes: if the response identification information indicates that the target person's response is an invalid response, and the behavior identification information indicates that the target person has abnormal behavior, then the security risk level is determined to be a high-risk level.

[0009] Optionally, determining the security risk level based on the response identification information and the behavior identification information includes: when the response identification information indicates that the identity information of the target person and the person being visited are clear, and the behavior identification information indicates that the target person does not exhibit any abnormal behavior, the security risk level is determined to be a low-risk level.

[0010] Optionally, the step of performing corresponding security protection processing based on the security risk level includes: when the security risk level is low, pushing the identity information of the target person and the visitor information to a preset terminal device; and performing corresponding processing according to the processing instruction information sent by the terminal device.

[0011] Optionally, the step of performing corresponding security protection processing based on the security risk level includes: when the security risk level is high, obtaining a preset alarm phone number and initiating a phone call; after the phone call is connected, playing a preset alarm prompt message; and encrypting and storing the audio data and the first image data according to preset storage location information.

[0012] Optionally, during the process of acquiring the first image data through the sensing device, a report image and the acquisition time of the report image are generated based on the first image data; the report information is sent to a preset terminal device, wherein the report information includes the encrypted report image and the acquisition time.

[0013] Optionally, upon receiving a termination command from a preset terminal device, the robot is stopped from performing the following process, and the acquisition of audio data and the first image data through the sensing device is stopped.

[0014] Optionally, if it is determined that the target person does not exist in the first image data and this continues for a preset duration, the robot is stopped from performing the following process, and the acquisition of the audio data and the first image data through the sensing device is stopped.

[0015] Optionally, the second image data corresponding to the home environment collected by the sensing device is processed to obtain the face image information.

[0016] Optionally, the process of controlling the robot to perform identity inquiry on the target person includes: determining the first location information of the target person based on the second image data; determining an inquiry stopping position based on the robot's current position and the first location information, wherein the distance between the inquiry stopping position and the first position is greater than a preset distance threshold; generating operation route information to avoid obstacles and reach the inquiry stopping position; controlling the robot to reach the inquiry stopping position according to the operation route information; and performing voice inquiry and / or displaying inquiry information according to preset inquiry information.

[0017] Optionally, if the robot is detected to have been vandalized or its external power supply is cut off, the robot's internal backup power supply is activated; and the highest level alarm message is sent to a preset alarm number.

[0018] Optionally, controlling the robot to follow the target person includes: determining the second location information and environmental information of the target person in real time based on the first image data; generating a following path information to avoid obstacles based on the second location information and the environmental information; and controlling the robot to run based on the following path information.

[0019] According to a second aspect of this disclosure, a security control device is provided for use in a robot. The device includes: a matching processing module for matching facial image information collected by a sensing device with preset authorized facial image information; an inquiry processing module for controlling the robot to perform identity inquiry processing on a target person in the event of a matching failure, wherein the target person is the person corresponding to the facial image information of the failed match; an information acquisition module for controlling the robot to follow the target person and collecting audio data and first image data corresponding to the target person through the sensing device; a risk determination module for determining the security risk level of the target person based on the audio data and the first image data; and a protection execution module for executing corresponding security protection processing based on the security risk level.

[0020] According to a third aspect of this disclosure, a security control device is provided for use in a robot, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the method described above based on instructions stored in the memory.

[0021] According to a fourth aspect of this disclosure, a robot is provided, comprising: the security control device as described above.

[0022] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the method as described above.

[0023] According to a sixth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described above.

[0024] The security control method, device, robot, storage medium, and computer program product disclosed herein, when the collected facial image information fails to match the authorized facial image information, proactively performs identity inquiry processing and follow-up processing, collects audio data and image data, determines the security risk level of the target person, and performs security protection processing based on the security risk level. This achieves proactive identity verification and follow-up processing of unauthorized personnel (strangers), effectively reducing the false alarm rate of security, eliminating monitoring blind spots, and improving the initiative, risk assessment, and accuracy of security. Attached Figure Description

[0025] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The accompanying drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other objects and advantages of this disclosure will be further described below with reference to specific embodiments and the accompanying drawings. In the drawings, the same or corresponding technical features or components will be represented by the same or corresponding reference numerals.

[0026] Figure 1 The following is a flowchart illustrating some embodiments of the security control method according to this disclosure; Figure 2 This is a schematic diagram of the process of performing identity querying in some embodiments of the security control method according to this disclosure; Figure 3 This is a schematic diagram of the tracking process in some embodiments of the security control method according to the present disclosure; Figure 4This is a flowchart illustrating the determination of security risk levels in some embodiments of the security control method according to this disclosure; Figure 5 This is a schematic diagram of modules according to some embodiments of the security control device of this disclosure; Figure 6 This is a schematic diagram of modules according to some other embodiments of the security control device according to the present disclosure. Detailed Implementation

[0027] Exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the embodiments are described in the specification. However, it should be understood that many implementation-specific settings must be made in carrying out the embodiments to achieve the developer's specific goals, such as complying with constraints related to the device and business, and these constraints may vary depending on the implementation. Furthermore, it should be understood that while the development work may be very complex and time-consuming, such development work is merely a routine task for those skilled in the art who benefit from the present disclosure.

[0028] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0029] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0030] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0031] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0032] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0033] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0034] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0035] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0036] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0037] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0038] Furthermore, to avoid obscuring this disclosure with unnecessary detail, only processing steps and / or apparatus structures closely related to at least the solutions according to this disclosure are shown in the accompanying drawings, while other details less relevant to this disclosure are omitted. It should also be noted that similar reference numerals and letters in the drawings indicate similar items, and therefore once an item is defined in one drawing, it need not be discussed again in subsequent drawings.

[0039] The security control method disclosed herein is applied to robots, which can be various intelligent robots such as service robots and security patrol robots. This security control method can be applied in various scenarios such as homes and schools; the following explanation uses a home scenario as an example.

[0040] Figure 1 This is a flowchart illustrating some embodiments of the security control method according to this disclosure. Figure 1 As shown, the security control method includes steps S101-S105.

[0041] Step S101 involves matching the facial image information collected by the sensing device with the preset authorized facial image information.

[0042] The robot can be equipped with sensing devices, communication modules, intelligent decision-making modules, displays, and other devices. Sensing devices may include visual perception components, voice interaction modules, microphones, and other components. Visual perception components may include cameras, depth cameras, and various sensors. The robot can use these sensing devices to acquire real-time images of the home environment, perform face detection and recognition processing on the image data, extract facial image information, and then perform similarity matching processing between the extracted facial image information and preset authorized facial image information.

[0043] The pre-set authorized facial image information can be stored in the robot's local memory or in a cloud-based authorized facial image information database. This authorized facial image information includes pre-recorded facial images of family members and other acquaintances. Upon initial power-on, the robot can record the names, faces, and voiceprints of family members and acquaintances through natural interaction with the user. During subsequent use, the robot can archive information about family members and acquaintances, generating an authorized facial image information database.

[0044] In step S102, if the matching fails, the robot is controlled to perform an identity inquiry on the target person, where the target person is the person corresponding to the face image information of the person whose matching failed.

[0045] The robot can use various matching algorithms to match the collected facial image information with preset authorized facial image information. If the match fails, the target person is determined to be a stranger. For example, the matching algorithm can be a deep learning-based facial feature point comparison algorithm with a matching degree threshold of 90%. If the matching degree between the collected facial image information and the preset authorized facial image information is lower than the matching degree threshold, the match is determined to be failed.

[0046] The robot can collect home environment data through RGBD (Red, Green, Blue – Depth) sensors and perform semantic spatial recognition to determine the location of the target person. If a match fails, the robot triggers a pre-set stranger identity verification process. Through its voice interaction module, the robot initiates an identity verification process with the target person, asking questions such as, "Hello, who are you? Which family member are you visiting?" This proactively verifies the identity of unfamiliar individuals.

[0047] The robot can collect one or more facial image information through sensing devices. If multiple facial image information fails to match the preset authorized facial image information, it is determined that there are multiple target persons. In the case of multiple target persons, the robot can sequentially perform identity inquiry processing on each target person and proceed to steps S103 to S105.

[0048] Alternatively, if any face image from multiple face image information successfully matches the preset authorized face image information, it can be determined that an authorized person exists and that there are 0 target persons. The robot will then stop performing identity inquiry processing and will not execute steps S103 to S105, returning to the normal state.

[0049] Step S103: Control the robot to follow the target person and collect audio data and first image data corresponding to the target person through the sensing device.

[0050] When performing identity verification on the target person, the robot continuously follows the target person based on their movements to ensure that the target person remains within the robot's shooting and perception range.

[0051] The robot can collect audio data of the target person through the voice interaction module of the sensing device, and collect the first image data of the target person through the visual perception component of the sensing device. The audio data may include the target person's voice response, environmental sounds, etc.; the first image data may include continuously captured pictures and / or real-time recorded videos, etc.; the audio data and the first image data can be time-stamped and synchronized to ensure the correlation between the data.

[0052] During the process of the robot tracking the target person, the robot's tracking function and the functions of collecting audio data and first image data can be set to the highest priority, and all external commands can be blocked. Only the user can send exclusive commands through the preset terminal device to turn off the tracking function and the functions of collecting audio data and first image data, so as to prevent the target person from evading the monitoring and ensure the continuity of tracking and data collection.

[0053] Step S104: Determine the safety risk level of the target personnel based on the audio data and the first image data.

[0054] The robot can perform comprehensive analysis on the collected audio data and first image data, and determine the safety risk level of the target personnel by combining the pre-set risk assessment rules; the safety risk level can include multiple levels such as high risk and low risk.

[0055] The robot can determine the safety risk level of a target person in real time based on audio data and the first image data. Alternatively, the robot can periodically determine the safety risk level of a target person at preset time intervals, such as 10 seconds or 15 seconds. When the robot stops tracking the target person and stops collecting audio data and the first image data, the determination of the target person's safety risk level can be stopped, and step S105 can be stopped.

[0056] Step S105: Based on the security risk level, perform the corresponding security protection measures.

[0057] Robots can perform differentiated safety protection operations based on the level of safety risk. Safety protection measures can be pre-set; for example, for high-risk levels, they can trigger emergency warnings and encrypt data storage; for low-risk levels, they can perform operations such as pushing information and waiting for remote commands.

[0058] The security control method disclosed herein proactively performs identity inquiry and tracking processing when the collected facial image information fails to match the authorized facial image information. It collects audio and image data, determines the security risk level of the target person, and performs security protection processing based on the security risk level. This enables proactive identity verification and tracking of unauthorized personnel (strangers), effectively reducing the false alarm rate, eliminating monitoring blind spots, and effectively improving the initiative, risk assessment, and accuracy of security measures.

[0059] In some embodiments, the robot performs recognition processing on second image data corresponding to the home environment collected by sensing devices to obtain facial image information. The robot can collect images of the entire home area through sensing devices to generate second image data. The scope of the second image data collection covers all home areas such as the living room, bedroom, kitchen, and balcony. The second image data can be photos and / or videos.

[0060] The robot can use various object detection algorithms such as Faster R-CNN (Faster Region-based Convolutional Neural Network) to detect face regions from the second image data. It can then extract face image information from the face regions using various feature extraction algorithms such as FaceNet. The robot will match the obtained face image information with preset authorized face image information. If the collected face image information does not match any authorized face image information, the robot will determine that the target person corresponding to the face image information is a stranger and perform identity inquiry processing.

[0061] Figure 2 This is a flowchart illustrating the execution of identity query processing in some embodiments of the security control method according to this disclosure, such as... Figure 2 As shown: Step S201: Determine the first location information of the target person based on the second image data.

[0062] The robot can pre-construct a digital map of the home environment based on spatial data collected by sensing devices, perform semantic segmentation processing, establish a semantic layer of the home environment, determine the location information of different areas such as the living room, bedroom, kitchen, and study, and label the space with functional areas and obstacles to generate a semantic map of the home space; functional areas include the living room, bedroom, and kitchen; obstacles include coffee tables, wardrobes, doors and windows.

[0063] The robot analyzes the collected second image data and uses the constructed family spatial semantic map and various visual positioning algorithms, such as the CNN (Convolutional Neural Network) visual positioning algorithm, to identify the spatial three-dimensional coordinates and spatial semantics corresponding to the target person, thereby obtaining the target person's first location information in the family scene. The first location information includes three-dimensional spatial coordinates and may also include spatial semantic information, such as living room, bedroom, and kitchen.

[0064] Step S202: Based on the robot's current position and the first position information, determine the query docking position. The distance between the query docking position and the first position is greater than a preset distance threshold, which can be set, for example, 1-2 meters.

[0065] Step S203: Generate route information to avoid obstacles and reach the inquiry and docking location.

[0066] For example, if the question asks for the docking location to be 1 meter away from the target person, the robot can use a semantic map of the home space to... Multiple algorithms, such as the Dijkstra algorithm, are used to generate route information that avoids obstacles and reaches the query stop location. The route can be the shortest possible route.

[0067] Step S204: Based on the route information, control the robot to reach the inquiry stopping position.

[0068] Step S205: Based on the preset inquiry information, perform voice inquiry and / or display the inquiry information.

[0069] The robot is controlled to reach the inquiry docking position based on its operating route information. It then proactively approaches the target person, maintaining a safe distance of 1-2 meters to balance safety and user-friendliness in human-computer interaction, avoiding any potential aversion to close contact. The robot can play preset voice inquiries and / or display text inquiries via its onboard display components, enabling both voice and text-based inquiry functionality. This adaptable approach enhances the success rate of inquiries across various scenarios.

[0070] Figure 3 This is a flowchart illustrating the tracking process in some embodiments of the security control method according to this disclosure, such as... Figure 3 As shown: Step 301: Based on the first image data, determine the second location information and environmental information of the target personnel in real time.

[0071] The robot continuously collects initial image data of the target person through sensing devices. Utilizing a pre-constructed semantic map of the home space, and based on various visual positioning algorithms such as CNN-based visual positioning algorithms, it identifies the target person's corresponding three-dimensional spatial coordinates and spatial semantics, determines environmental information, and obtains the target person's second location information within the home scene. The second location information includes three-dimensional spatial coordinates and may also include spatial semantic information. Spatial semantic information includes locations such as the living room, bedroom, and kitchen. Environmental information includes the locations of obstacles, such as tables and chairs.

[0072] Step 302: Generate obstacle avoidance following path information based on the second location information and environmental information.

[0073] The robot can utilize a semantic map of the home space, based on secondary location information and environmental information, and employ various path planning algorithms to generate obstacle-avoiding following paths in real time. The following paths can be adapted to the movement speed of the target person.

[0074] Step 303: Control the robot to run according to the following path information.

[0075] Based on real-time updated following path information, the robot adjusts its direction and speed to achieve continuous, accurate, and collision-free tracking of the target person, ensuring that the target person remains within the effective range of audio and video capture.

[0076] In some embodiments, the preset authorized facial image information includes pre-recorded facial information of family members or other authorized personnel. When the robot acquires facial image information A, if facial image information A matches the preset authorized facial image information, the robot determines that the person corresponding to facial image information A is the authorized target person, and the robot terminates the execution of the remaining method steps in the security control method of this disclosure.

[0077] If facial image information A fails to match the preset authorized facial image information, the person corresponding to facial image information A is determined to be an unauthorized target person, i.e., a stranger. The robot determines the target person's first location information. Based on the robot's current location and the target person's first location information, it determines the inquiry docking position. The distance between the inquiry docking position and the target person's first location is greater than a preset distance threshold of 1 meter to ensure a safe distance for human-computer interaction.

[0078] Based on a semantic map of the home space and utilizing various path planning algorithms, the robot generates a route that avoids obstacles and reaches the inquiry docking location. Following this route information, the robot moves to the inquiry docking location and then conducts a voice inquiry based on preset questions. The inquiry information can also be displayed on the robot's display component. The preset inquiry is: "Hello, I have not recognized your authorization information. Who are you? Which family member are you visiting?" This proactively verifies the identity of the target person.

[0079] While initiating identity inquiry processing, the robot collects first image data corresponding to the target person through sensing devices. Based on the first image data, it determines the second location information of the target person in real time through various vision-based tracking algorithms. Combined with environmental information collected by environmental sensors, it generates following path information to avoid obstacles.

[0080] Based on the following path information, the robot controls its movement along the path to continuously track the target person, ensuring that the person remains within the robot's shooting and perception range at all times. The robot sets its tracking and data collection functions to the highest priority, blocking all external voice commands to disable them; they can only be turned off by the user sending a dedicated authorized command through a preset terminal device. This prevents the target person from evading monitoring and ensures the continuity of tracking and data collection.

[0081] In some embodiments, during the process of acquiring first image data through a sensing device, the robot generates a report image and the acquisition time of the report image based on the first image data; the robot sends the report information to a preset terminal device, and the report information includes an encrypted report image and the acquisition time, and the report information may also include the second location information of the target person, etc.

[0082] For example, during the process of the sensing device collecting the first image data, the robot extracts report images from the first image data at a preset frequency. The report images include the target person's face, behavior, etc., and the acquisition time of the report images is determined, and the current scene location information of the target person is associated with them (second location information). Various algorithms, such as symmetric encryption algorithms, can be used to encrypt the report images to ensure the security of data transmission.

[0083] The preset terminal device can be the homeowner's mobile phone, etc. The robot can be configured with a communication module, such as a WiFi module or a cellular network communication module. The report information is pushed to the preset terminal device silently without any prompts or changes in device status during the push process. The robot maintains its original working state without attracting the attention of the target personnel.

[0084] By generating encrypted report images with the collection time, homeowners can monitor the dynamic characteristics and time trajectory of strangers in real time. A silent push mode can be used to ensure that the robot has no prompts or light alerts, avoiding alerting strangers and achieving covert reporting.

[0085] Figure 4 This is a flowchart illustrating the process of determining a security risk level according to some embodiments of the security control method of this disclosure, such as... Figure 4 As shown: Step S401: Using a security risk identification model, determine the target person's response identification information to identity inquiry based on the audio data.

[0086] Safety risk identification models can be of various types, such as Transformer models. Multiple training methods can be used to pre-train the safety risk identification model, resulting in a pre-trained model. This model can be deployed in the robot's decision-making module or in a cloud-based database.

[0087] The robot inputs the collected audio data into a security risk identification model. This model performs speech recognition, semantic understanding, and sentiment analysis to extract response information from identified individuals in response to identity inquiries. This response information includes information indicating whether the individual's identity and the person being visited are clearly defined; whether the individual's response is contradictory; whether the individual's emotional state during the response is abnormal; and whether the response is invalid. Abnormal states include panic, resistance, and rage; invalid responses include no response and irrelevant answers.

[0088] Step S402: Using the security risk identification model, determine the behavioral identification information of the target person based on the first image data.

[0089] The robot can input the first image data it collects into a security risk identification model. This model performs human behavior recognition and motion analysis to determine the target person's behavioral information. This behavioral information can include details indicating whether the target person is evading image capture or exhibiting abnormal behavior. Evading image capture behaviors include covering the face, moving quickly to escape pursuit, and turning one's back to the camera; abnormal behaviors include touching door and window locks, rummaging through household items, and damaging furniture.

[0090] Step S403: Determine the security risk level based on the response identification information, or based on both the response identification information and the behavior identification information.

[0091] By using a security risk identification model to determine response and behavior identification information, the subjectivity of manual judgment can be avoided, thus improving the objectivity, consistency, and accuracy of risk level assessment.

[0092] In some embodiments, the robot will ask questions to the target person through dialogue. If the person is a guest or a temporary service worker, the robot will answer the questions in a natural manner, with a natural tone and mood.

[0093] When the robot responds to unclear identification information about the target person or the person being visited, or when there are contradictions in the target person's response, it determines the security risk level to be high.

[0094] For example, if the target person cannot explain their identity or the person they are visiting, the response identification information indicates that the target person's identity and the person they are visiting are unclear. Alternatively, if the target person's response regarding their identity or the person they are visiting is contradictory, the response identification information indicates that the target person's response is contradictory, and the security risk level is determined to be high.

[0095] If the robot determines that the target person's emotional state is abnormal when responding to the identification information, and if the target person exhibits behavior that avoids image acquisition, the safety risk level is determined to be high.

[0096] For example, if the target person's emotions are abnormal, such as panic or resistance, the response recognition information indicates that the target person's emotions are abnormal. Furthermore, if the target person exhibits behaviors such as covering their face, moving quickly to escape pursuit, or turning their back to the camera to avoid image capture, the behavior recognition information indicates that the target person is engaging in behaviors to avoid image capture. In this case, the robot determines the safety risk level to be high.

[0097] If the robot determines that the response of the target person is invalid and the behavior recognition information indicates that the target person is exhibiting abnormal behavior, the safety risk level is set to high risk.

[0098] For example, if the target person does not respond, the response identification information indicates that the target person's response is invalid. Furthermore, if the target person exhibits suspicious behaviors such as touching door and window locks, rummaging through household items, or damaging the home, the behavior identification information indicates that the target person has abnormal behavior. In this case, the robot determines the safety risk level to be high risk.

[0099] If the robot responds with information that clearly identifies the target person and the person being visited, and if the behavior identification information indicates that the target person's behavior is normal, then the robot determines the security risk level to be low.

[0100] For example, if the target person can clearly state their identity and the person they are visiting, the response identification information indicates that the target person's identity and the person they are visiting are clear, and the target person does not exhibit the aforementioned suspicious behavior, and the behavior identification information indicates that the target person does not exhibit abnormal behavior, then the robot determines the security risk level to be low risk.

[0101] In some embodiments, the security risk level may include a high-risk level, a low-risk level, etc. When the security risk level is low, the robot pushes the target person's identity information and visitor information to a preset terminal device, and performs corresponding processing according to the processing instructions sent by the terminal device.

[0102] For example, if the security risk level is low, the target person may be a regular visitor. The robot will use the security risk identification model to obtain the target person's identity information, visitor information, and other information, and push it to the preset homeowner's terminal device via the communication module; alternatively, it can push the target person's first image data to the preset homeowner's terminal device.

[0103] After the push is completed, the robot continues to follow the target person and collect audio and video, enters a waiting state, and receives processing instructions sent by the homeowner through the terminal device. The robot then performs the corresponding processing according to the processing instructions.

[0104] The instructions can include directions to a waiting area, termination of pursuit, or further inquiries. For example, if the instruction is "guide to the living room sofa," the robot will guide the target person to the living room sofa using voice guidance and physical navigation.

[0105] In some embodiments, when the safety risk level is high, the robot obtains a preset alarm phone number and initiates a phone call; after the phone call is connected, the robot plays a preset alarm prompt message; the robot performs encrypted storage processing on the audio data and the first image data according to the preset storage location information.

[0106] For example, in cases where the security risk level is high, the robot obtains the alarm phone number preset by the homeowner, which can be the mobile phone number of the homeowner or an emergency contact. The robot then initiates an automatic phone call to the preset alarm phone number through the communication module.

[0107] Once the call is connected, the robot automatically plays a preset alarm message, such as, "Hello, a highly suspicious stranger has been detected in your home. Their current location is [location]. We have sent you a real-time image; please view and take appropriate action." The phone call is the highest priority alert method, ensuring homeowners are aware of their home's security status immediately.

[0108] Storage location information includes cloud storage location information. Simultaneously with initiating a phone call, the robot can encrypt the collected audio and initial image data based on the preset storage location information and store them in the cloud.

[0109] In situations where the safety risk level is high, the robot's voice playback module can also emit a high-decibel warning sound to attract the attention of people in the vicinity and deter the target personnel.

[0110] In some embodiments, upon receiving a termination command from a preset terminal device, the robot stops controlling the robot to perform the following process and stops collecting audio data and first image data through the sensing device.

[0111] For example, the robot's communication module receives termination commands from preset terminal devices such as homeowners in real time, verifies the termination commands, and if the verification is successful, it determines that the termination command received by the robot from the preset terminal device is an authorized termination command.

[0112] Upon receiving the termination command, the robot stops tracking and ceases collecting audio and first image data via sensing devices, resuming its normal operating state.

[0113] In some embodiments, if it is determined that there is no target person in the first image data and this continues for a preset time, the robot is stopped from following the target person, and the acquisition of audio data and the first image data through the sensing device is stopped.

[0114] For example, the preset duration can be set, such as 5 minutes. The robot's sensing devices continuously collect first image data, and the safety risk identification model analyzes the first image data in real time to determine whether there is a target person in the first image data. If it is determined that there is no target person in the first image data, and this continues for 5 minutes, it is determined that the target person has left the home environment, the robot stops following the target, and the collection of audio data and first image data through the sensing devices is stopped, resuming normal operation.

[0115] In some embodiments, when a family member returns to the home environment, after the robot identifies the authorized person through facial recognition, it stops controlling the robot to follow the person and stops collecting audio data and first image data through the sensing device, and resumes normal operation.

[0116] If the robot is restricted during the pursuit and is unable to continue following the target person, it will send an abnormal situation notification to the preset terminal device through the communication module.

[0117] In some embodiments, if the robot is detected to have been vandalized or its external power supply is cut off, the robot's internal backup power supply is activated, and the highest level alarm message is sent to a preset alarm number.

[0118] Current detection modules can be used to detect whether the robot's external power supply has been cut off. Destructive actions can take many forms, such as impacts and disassembly. The robot has multiple built-in sensor modules and an internal power supply. These sensor modules can include vibration sensors and anti-tamper detection modules. When the vibration sensor detects an impact force greater than a threshold, it determines that the robot has been impacted.

[0119] There are various implementation schemes for the anti-tamper sensor module. For example, the anti-tamper sensor module includes an anti-tamper trigger switch, which can be a normally closed tactile limit switch, etc. The anti-tamper trigger switch is installed at key splicing points such as the splicing gap of the robot shell, the connection between the head and the body, and the joints of the robot arm and legs.

[0120] The tamper-proof trigger switch normally outputs a low level. When a stranger forcibly disassembles the robot (e.g., prying open the outer shell, removing the bottom cover, etc.), the tamper-proof trigger switch is triggered, and its output level changes to a high level. When the tamper-proof trigger switch outputs a high level, it indicates that the robot has been disassembled.

[0121] The preset alarm number can be the homeowner's terminal device (mobile phone, etc.). When vandalism is detected on the robot or an external power supply is cut off, the internal backup power supply is immediately activated, and at the same time, the highest level alarm information is sent to the homeowner's terminal device number via the communication module. The highest level alarm information includes the time of the vandalism and the robot's current location information.

[0122] The highest-level alarm message has the highest transmission priority, exceeding the transmission priority of all other robot data; the communication module will prioritize network bandwidth to complete the transmission of the highest-level alarm message. When the highest-level alarm message is sent to the homeowner's terminal device, it can forcibly trigger the terminal device to start a mandatory reminder mode.

[0123] By activating its internal backup power and sending the highest-level alarm message after the robot detects that it has been damaged or that a power outage has occurred, it can effectively prevent behaviors such as violent destruction and power cut-off from evading monitoring, thereby improving the robot's anti-attack capability, reliability, and actual protection effect.

[0124] The security control method in the above embodiments, when the collected facial image information fails to match the authorized facial image information, proactively performs identity inquiry processing and tracking processing, collects audio and image data, determines the security risk level of the target person, and performs security protection processing based on the security risk level. This achieves proactive identity verification and tracking processing for unauthorized personnel (strangers), effectively reducing the false alarm rate, eliminating monitoring blind spots, and improving the coverage and continuity of security. By fusing multi-dimensional information from audio and images to determine the security risk level, it enables differentiated handling of high and low risks, ensuring home safety and improving the user-friendliness of human-computer interaction. It effectively improves the proactiveness of security, the accuracy of risk assessment and handling.

[0125] In some embodiments, such as Figure 5 As shown, this disclosure provides a security control device for use in robots. The security control device includes: a matching processing module 501, an inquiry processing module 502, an information acquisition module 503, a risk determination module 504, and a protection execution module 505.

[0126] The matching processing module 501 matches the facial image information collected by the sensing device with preset authorized facial image information. In the event of a matching failure, the inquiry processing module 502 controls the robot to perform an identity inquiry on the target person corresponding to the failed facial image information.

[0127] The information acquisition module 503 controls the robot to track the target person and collects audio data and corresponding first image data through sensing devices. The risk assessment module 504 determines the safety risk level of the target person based on the audio data and the first image data. The protection execution module 505 executes corresponding safety protection measures based on the safety risk level.

[0128] In some embodiments, the matching processing module 501 performs recognition processing on the second image data corresponding to the home environment collected by the sensing device to obtain face image information.

[0129] The query processing module 502 determines the first location information of the target person based on the second image data; the query processing module 502 determines the query docking position based on the robot's current position and the first location information, wherein the distance between the query docking position and the first position is greater than a preset distance threshold.

[0130] The query processing module 502 generates running route information to avoid obstacles and reach the query docking position; the query processing module 502 controls the robot to reach the query docking position according to the running route information; the query processing module 502 performs voice query and / or displays query information according to preset query information.

[0131] The information acquisition module 503 determines the second location information and environmental information of the target person in real time based on the first image data; the information acquisition module 503 generates a following path information to avoid obstacles based on the second location information and environmental information; the information acquisition module 503 controls the robot to run based on the following path information.

[0132] In some embodiments, the risk determination module 504 uses a security risk identification model to determine the target person's response identification information to identity inquiry processing based on audio data; the risk determination module 504 uses a security risk identification model to determine the target person's behavior identification information based on first image data; the risk determination module 504 determines the security risk level based on the response identification information, or based on both the response identification information and the behavior identification information.

[0133] When the response identification information is unclear regarding the identity of the target personnel and the person being visited, or when there are contradictions in the response content of the target personnel, the risk assessment module 504 determines the security risk level to be high.

[0134] If the response identification information indicates that the target person's emotions are abnormal and the behavior identification information indicates that the target person is evading image acquisition, the risk determination module 504 determines the security risk level to be high.

[0135] If the response identification information indicates that the target person's response is invalid, and the behavior identification information indicates that the target person has abnormal behavior, the risk determination module 504 determines the security risk level to be high risk.

[0136] If the response identification information clearly identifies the target person's identity and the person being visited, and the behavior identification information indicates that the target person does not exhibit any abnormal behavior, the risk assessment module 504 determines the security risk level to be low.

[0137] When the security risk level is low, the protection execution module 504 pushes the identity information of the target personnel and the information of the person being visited to the preset terminal device; the protection execution module 504 performs the corresponding processing according to the processing instruction information sent by the terminal device.

[0138] When the security risk level is high, the protection execution module 504 obtains the preset alarm phone number and initiates a phone call; after the phone call is connected, the protection execution module 504 plays the preset alarm prompt information; the protection execution module 504 performs encrypted storage processing on the audio data and the first image data according to the preset storage location information.

[0139] In some embodiments, during the process of acquiring first image data through a sensing device, the information acquisition module 503 generates a report image and the acquisition time of the report image based on the first image data; the information acquisition module 503 sends the report information to a preset terminal device, wherein the report information includes an encrypted report image and the acquisition time.

[0140] When the information acquisition module 503 receives a termination command sent by a preset terminal device, it stops controlling the robot to perform the following process and stops collecting audio data and first image data through the sensing device.

[0141] If the information acquisition module 503 determines that there is no target person in the first image data and continues for a preset time, it stops controlling the robot to perform the following process and stops collecting audio data and first image data through the sensing device.

[0142] When the protection execution module 504 detects that the robot has been damaged or that the robot's external power supply has been cut off, it activates the robot's internal backup power supply; the protection execution module 504 sends the highest level alarm information to the preset alarm number.

[0143] like Figure 6 As shown, the security control device may include a memory 601, a processor 602, a communication interface 603, and a bus 604. The memory 601 is used to store instructions, and the processor 602 is coupled to the memory 601. The processor 602 is configured to execute the above-described security control method based on the instructions stored in the memory 601.

[0144] The memory 601 can be a high-speed RAM, non-volatile memory, or a memory array. The memory 601 may also be divided into blocks, and these blocks can be combined into virtual volumes according to certain rules. The processor 602 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the security control method of this disclosure.

[0145] In some embodiments, this disclosure provides a robot including the security control device as described in any of the above embodiments. The robot can be a service robot, a security patrol robot, or other types of robots.

[0146] In some embodiments, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the security control method as described in any of the foregoing embodiments.

[0147] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (not an exhaustive list) of readable storage media may include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0148] Embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0149] The security control method, device, robot, storage medium, and computer program product in the above embodiments realize layered and progressive interactive guidance for users, reduce user input operations, and lower the error rate of user input; it can reduce the number of interface switching or operations performed by users, effectively improve the operational efficiency of natural language interaction, increase the user's interaction success rate, improve the user's interactive experience, and achieve a more intuitive, efficient, stable, and easy-to-use visual natural language interaction effect.

[0150] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0151] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0152] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0153] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0154] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0155] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although several exemplary aspects and embodiments have been discussed above, those skilled in the art will understand that the above embodiments are illustrative only and do not limit the scope of this disclosure. Those skilled in the art will understand that the above embodiments can be combined, modified, or replaced without departing from the scope and spirit of this disclosure.

Claims

1. A security control method applied to a robot, the method comprising: The facial image information collected by the sensing device is matched with the preset authorized facial image information; In the event of a failed match, the robot is controlled to perform an identity inquiry on the target person corresponding to the failed facial image information. The robot is controlled to follow the target person and collects audio data and first image data corresponding to the target person through the sensing device. Based on the audio data and the first image data, the security risk level of the target person is determined; Based on the security risk level, corresponding security protection measures will be implemented.

2. The method according to claim 1, wherein, Determining the security risk level of the target person based on the audio data and the first image data includes: Using a security risk identification model, the target person's response identification information to the identity inquiry process is determined based on the audio data; Using the security risk identification model, the behavioral identification information of the target person is determined based on the first image data; The security risk level is determined based on the response identification information, or based on the response identification information and the behavior identification information.

3. The method according to claim 2, wherein, Determining the security risk level based on the response identification information includes: If the response identification information does not clearly identify the target person's identity and the person being visited, or if the target person's response is contradictory, the security risk level is determined to be high risk.

4. The method according to claim 2, wherein, Determining the security risk level based on the response identification information and the behavior identification information includes: If the response recognition information indicates that the target person's emotional state is abnormal, and the behavior recognition information indicates that the target person is evading image acquisition, then the security risk level is determined to be high risk.

5. The method according to claim 2, wherein, Determining the security risk level based on the response identification information and the behavior identification information includes: If the response identification information indicates that the target person's response is invalid, and the behavior identification information indicates that the target person exhibits abnormal behavior, then the security risk level is determined to be a high-risk level.

6. The method according to claim 2, wherein, Determining the security risk level based on the response identification information and the behavior identification information includes: If the response identification information clearly indicates that the target person's identity and the person being visited are clear, and the behavior identification information indicates that the target person does not exhibit any abnormal behavior, then the security risk level is determined to be low risk.

7. The method according to claim 6, wherein, The corresponding security protection measures implemented based on the security risk level include: When the security risk level is low, the identity information of the target personnel and the information of the person being visited are pushed to the preset terminal device; The corresponding processing is performed according to the processing instruction information sent by the terminal device.

8. The method according to claim 1, wherein, The corresponding security protection measures implemented based on the security risk level include: If the security risk level is high, obtain the preset alarm phone number and initiate a phone call; After the phone call is connected, a preset alarm message is played; The audio data and the first image data are encrypted and stored according to the preset storage location information.

9. The method according to claim 1, further comprising: During the process of acquiring the first image data through the sensing device, a report image and the acquisition time of the report image are generated based on the first image data; The report information is sent to a preset terminal device, wherein the report information includes the encrypted report image and the acquisition time.

10. The method according to claim 1, further comprising: Upon receiving a termination command from a preset terminal device, the robot stops performing the following process and ceases collecting the audio data and the first image data through the sensing device.

11. The method according to claim 1, further comprising: If it is determined that the target person is not present in the first image data and this continues for a preset duration, the robot is stopped from performing the following process, and the acquisition of audio data and the first image data through the sensing device is stopped.

12. The method according to claim 1, further comprising: The second image data, which is collected by the sensing device and corresponds to the home environment, is processed to obtain the face image information.

13. The method according to claim 12, wherein, The controlled robot performs identity verification on the target person, including: Based on the second image data, the first location information of the target person is determined; Based on the robot's current position and the first position information, an inquiry docking position is determined, wherein the distance between the inquiry docking position and the first position is greater than a preset distance threshold; Generate route information to avoid obstacles and reach the requested stopping location; Based on the route information, control the robot to reach the inquiry stopping position; Based on preset inquiry information, perform voice inquiry and / or display inquiry information.

14. The method of claim 1, further comprising: If the robot is detected to have been vandalized or its external power supply is cut off, the robot's internal backup power supply will be activated. Send the highest level alarm message to the preset alarm number.

15. The method according to any one of claims 1 to 14, wherein, The process of controlling the robot to follow the target person includes: Based on the first image data, the second location information and environmental information of the target person are determined in real time; Based on the second location information and the environmental information, generate obstacle avoidance following path information; The robot is controlled to run based on the following path information.

16. A security control device applied to a robot, the device comprising: The matching processing module is used to match the facial image information collected by the sensing device with the preset authorized facial image information. The query processing module is used to control the robot to perform identity query processing on the target person corresponding to the face image information of the failed match when the match fails. The information acquisition module is used to control the robot to follow the target person and to acquire audio data and first image data corresponding to the target person through the sensing device. The risk assessment module is used to determine the safety risk level of the target person based on the audio data and the first image data. The protection execution module is used to perform corresponding security protection processing based on the security risk level.

17. A security control device applied to a robot, comprising: Memory; and a processor coupled to the memory, the processor being configured to perform the method according to any one of claims 1 to 15 based on instructions stored in the memory.

18. A robot comprising: The security control device as described in claim 16 or 17.

19. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 15.

20. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15.

Images