Intelligent door lock protocol control method and control system

Abstract

A smart door lock protocol control method and control system are disclosed. This method is applied to a dual-core system (core A and core B). Core A receives face recognition or registration commands from the host computer and forwards them to core B. Core B executes the corresponding face processing flow and reports intermediate status messages at preset nodes. Core A feeds back the intermediate status and final result to the host computer. The system includes an core A module, a core B module, and a protocol management module. The protocol management module encapsulates and parses messages according to a predefined structured format. The core B module performs face image acquisition and algorithm processing, stores fused feature templates in a template library, and dynamically updates the templates based on the recognition results. This application achieves standardized dual-core collaboration, provides full-process status feedback, and improves system stability and maintainability.

Description

Technical Field

[0001] This application relates to the technical field of smart door lock control, specifically to a smart door lock protocol control method and control system. Background Technology

[0002] With the popularization of smart homes, facial recognition smart door locks have been widely used due to their convenience and security. The core of such locks typically consists of a host computer, a main control unit (such as the A core), and a coprocessor unit (such as the B core) responsible for image acquisition and AI processing. In existing technologies, the communication protocol design of smart door locks suffers from the following common defects: Firstly, the status feedback is incomplete: existing systems typically only report the final result of "success" or "failure" when recognition is complete, failing to provide feedback on intermediate process statuses such as "image acquisition in progress," "algorithm processing in progress," or "poor image quality." This makes it difficult to quickly pinpoint the root cause of problems when malfunctions occur (e.g., whether it's a camera malfunction, network transmission issue, or algorithm misjudgment), greatly increasing the difficulty of debugging and maintenance.

[0003] Secondly, the dual-core collaboration mechanism is chaotic: In the dual-core architecture using core A (main control) and core B (AI processing), existing solutions lack clear and standardized inter-core communication protocols. Unclear division of responsibilities and asynchronous message responses can easily lead to problems such as system state disorder and execution process freezes.

[0004] Thirdly, the protocols are inconsistent and untraceable: different manufacturers use proprietary communication protocols with varying fields and formats, leading to difficulties in integrating different hardware modules with host computer software and poor compatibility. Furthermore, the lack of effective message identification and logging mechanisms makes the operational process untraceable.

[0005] Therefore, there is an urgent need for an intelligent door lock control system that can provide transparent status feedback throughout the entire process, achieve dual-core standardized collaboration, and has a unified and scalable communication protocol to improve the system's stability, maintainability, and development efficiency. Summary of the Invention

[0006] This application provides a smart door lock protocol control method and control system, which solves the technical problems of incomplete status feedback, chaotic dual-core collaborative mechanism, and inconsistent and untraceable protocols in existing smart door lock communication protocols.

[0007] The first aspect of this application provides a smart door lock protocol control method, which adopts the following technical solution: A smart door lock protocol control method, characterized in that it is applied to a dual-core system including a core A and a core B, the method comprising: Core A receives control commands from the host computer, the control commands including at least a face recognition command or a face registration command; Core A forwards the control command to Core B; B core executes the face processing flow corresponding to the control command, and at multiple preset nodes of the face processing flow, reports intermediate status messages representing the execution status of the node to A core. Core A will feed back the received intermediate state messages and / or the final result of the face processing process to the host computer.

[0008] By adopting the above technical solution, after receiving control commands from the host computer, core A does not execute them directly, but forwards them to core B, which is responsible for the core face processing task, thus achieving a reasonable division of functions. When core B executes specific face processing procedures such as face recognition or face registration, it actively sends intermediate status messages to core A at several preset key nodes in the process, such as when face image acquisition is complete, feature extraction begins, feature comparison is in progress, and registration information is successfully stored. This allows core A to monitor the processing progress of core B in real time, rather than only obtaining the result after the process is completed. Core A then feeds back these real-time acquired intermediate status messages and the final processing result to the host computer. This design allows the host computer not only to know the final operation result but also to clearly understand the key state changes throughout the entire processing, thereby solving the problem of incomplete state feedback in existing technologies and making the host computer's control and monitoring of the smart lock more refined and transparent.

[0009] Optionally, when the control command includes at least a face recognition command, the face processing flow executed by the B core includes a face recognition sub-flow, which includes: One or more frames of raw image are acquired; Face detection is performed on each frame of the original image to obtain at least one face detection bounding box; Key point localization is performed on the image containing the face detection box to obtain the coordinates of the face key points; An affine transformation is performed on the image based on the coordinates of the facial key points to obtain an aligned facial image. The quality of the aligned face image is assessed, and the quality assessment result is output. Feature extraction is performed on the aligned face images that have passed the quality assessment to obtain the target feature vector; The target feature vector is matched with the user feature templates pre-stored in the template library to obtain a matching similarity score; Based on the matching similarity score, combined with the preset dual threshold rule and continuity decision rule, the final recognition result is generated.

[0010] By adopting the above technical solution, when the control command includes at least a face recognition command, the face recognition sub-process executed by the B core ensures the accuracy and reliability of the recognition through multi-step refined processing. First, one or more frames of original images are acquired to provide a data foundation for subsequent processing. Face detection is performed on each frame of the original image to obtain at least one face detection box. This step accurately selects regions where faces may exist from complex backgrounds, which is a prerequisite for subsequent processing. Next, keypoint localization is performed on the image containing the face detection box to obtain the coordinates of facial keypoints. These keypoints, such as the corners of the eyes, the tip of the nose, and the corners of the mouth, provide accurate references for subsequent image alignment. Based on the coordinates of the facial keypoints, an affine transformation is performed on the image to obtain the aligned face image. Through this geometric transformation, faces with different poses and angles can be adjusted to a unified standard pose, eliminating the influence of pose differences on subsequent feature extraction and matching. Subsequently, the aligned face images are quality-assessed, and only images meeting the quality standards proceed to the next step of feature extraction. This step effectively filters out low-quality images that affect recognition, such as blurry images, uneven lighting, and severe occlusion, ensuring the reliability of subsequent features. Feature extraction is then performed on the aligned face images that meet the quality standards, yielding a target feature vector. This vector is a compact representation of the face image in a high-dimensional space, containing unique biometric information. The target feature vector is then matched with pre-stored user feature templates in a template library to obtain a similarity score, which measures the similarity between the face to be identified and known users in the template library. Finally, based on the similarity score, and combined with preset dual-threshold rules and continuity decision rules, the final recognition result is generated. The dual-threshold rule, by setting high and low thresholds, allows for rapid preliminary judgment of the matching results, while the continuity decision rule, by statistically analyzing the matching results of multiple consecutive frames within a preset recognition time window, effectively avoids the impact of single-frame image recognition errors, further improving the accuracy and stability of face recognition and reducing the false recognition and rejection rates.

[0011] Optionally, the application of the dual threshold rule includes: The matching similarity score is compared with a preset high threshold and a preset low threshold; If the matching similarity score is greater than or equal to the preset high threshold, a candidate decision for recognition is generated; If the matching similarity score is less than or equal to the preset low threshold, a rejection decision is generated. If the matching similarity score is between the preset low threshold and the preset high threshold, a pending decision is generated.

[0012] By adopting the above technical solution, the dual-threshold rule constructs a multi-level decision logic by setting preset high and low thresholds. When the matching similarity score is greater than or equal to the preset high threshold, it indicates that the feature to be identified is highly similar to the corresponding feature in the template library. In this case, a candidate decision for recognition is directly generated, ensuring rapid approval under high confidence conditions. If the matching similarity score is less than or equal to the preset low threshold, it indicates that the difference between the two is significant, and a rejection decision is generated, effectively avoiding false approval of low-similar features. When the score is between the low and high thresholds, the system does not simply make a judgment, but generates a pending decision, providing room for intervention in subsequent continuous decision rules. This design avoids the misjudgment that may be caused by a single threshold and leaves room for further decision-making to improve recognition accuracy.

[0013] Optionally, the application of the continuity decision rule includes: Within a preset recognition time window, the matching similarity scores and judgment results corresponding to multiple consecutive frames of images are obtained; The number of times the candidate decision is passed is counted. If the number of times the candidate decision appears reaches a first preset number, then the final recognition result of successful recognition is generated; If the number of times the recognition rejection decision occurs reaches a second preset number, or the total time consumed by the face recognition sub-process exceeds a preset duration, then the final recognition result indicating recognition failure is generated.

[0014] By adopting the above technical solution, the continuous decision rule introduces a preset recognition time window to dynamically analyze and statistically analyze the matching results of multiple consecutive frames of images. Within this time window, the system continuously acquires the matching similarity score corresponding to each frame of image and the decision result generated according to the dual threshold rule, and focuses on counting the number of times "recognition passed candidate decision" occurs. When the cumulative number of "recognition passed candidate decision" reaches a preset first preset number, the system determines that there are enough high similarity frames to support identity verification, thereby generating a final recognition result of "recognition successful". This effectively reduces the risk of misjudgment caused by accidental factors (such as sudden changes in lighting or instantaneous abnormal pose) in a single frame image. Conversely, if the number of "recognition rejected decision" occurs reaches a second preset number, or the total time of the entire face recognition sub-process exceeds a preset time (e.g., the user does not complete the effective face presentation for a long time), the system generates a final recognition result of "recognition failed". The former strengthens the reliability of the decision through a majority voting mechanism, while the latter avoids the invalid occupation of system resources through timeout control and promptly feeds back the abnormal termination status of the recognition process, further improving the robustness of face recognition and user experience.

[0015] Optionally, when the control command includes at least a face registration command, the face processing flow executed by the B core includes a face registration sub-flow, which includes: Acquire multiple frames of original registration images; For each frame of the original registration image, face detection, key point localization, affine transformation and quality judgment are performed sequentially to select multiple frames of qualified registration images; Extract features from each frame of the qualified registered image to obtain multiple registration feature vectors; The multiple registered feature vectors are fused to generate a fused feature template; The fused feature template is stored in the template library, and a user identifier is assigned to the fused feature template to generate a registration success result containing the user identifier.

[0016] By adopting the above technical solution, when the control command includes at least a face registration command, the face registration sub-process executed by the B core constructs a more robust user feature template through multi-frame image acquisition and fusion processing. First, the system acquires multiple frames of original registration images, rather than single frames. This is because multiple frames can capture facial information of the user under different angles, expressions, or slight pose changes, laying the foundation for generating a comprehensive feature template. Next, for each frame of original registration images, similar to the face recognition sub-process, face detection, key point localization, affine transformation, and quality assessment steps are sequentially performed. Face detection ensures accurate selection of the face region from each frame; key point localization and affine transformation unify the alignment of faces in different poses, eliminating the influence of geometric deformation; quality assessment rigorously selects clear, appropriately lit, and unobstructed qualified registration images, eliminating the interference of low-quality images on the template quality. Subsequently, feature extraction is performed on the selected multiple qualified registration images to obtain multiple registration feature vectors, each representing the user's facial features from a specific angle or state. To leverage the advantages of these multi-frame features, the system fuses multiple registered feature vectors, using methods such as weighted averaging, feature concatenation, or more complex deep learning fusion algorithms, to generate a fused feature template. This fused template integrates information from multiple images, containing richer facial details and variation patterns, thereby improving the accuracy and anti-interference capabilities of subsequent recognition. Finally, the generated fused feature template is stored in a template library and assigned a unique user identifier. This identifier is bound to the facial feature template for identity verification during subsequent recognition. After storage, the system generates a registration success result containing the user identifier and reports the successful completion of the registration process to the host computer, ensuring the user is clearly aware of their registration status. This design of multi-frame acquisition, rigorous screening, and feature fusion significantly improves the quality and reliability of the registration template, providing a solid guarantee for the safe and efficient operation of the entire smart lock system.

[0017] The second aspect of this application provides an intelligent door lock protocol control system, which adopts the following technical solution: A smart door lock protocol control system, characterized in that it is applied to a smart door lock and is used to execute the smart door lock protocol control method as described in the first aspect and any possible implementation thereof, the smart door lock protocol control system comprising an A-core module, a B-core module, and a protocol management module: The A-core module is used to establish a communication connection with the host computer; The B-core module is connected to the camera and is used to perform face image acquisition and algorithm processing; The protocol management module is used to encapsulate and parse control commands, intermediate status messages and final results according to a predefined structured format; Specifically, the A-core module receives control commands from the host computer through the protocol management module and forwards them to the B-core module; the B-core module reports intermediate status messages to the A-core module through the protocol management module at multiple preset nodes in the face image acquisition and algorithm processing process; and the A-core module feeds back the intermediate status messages and / or the final results to the host computer through the protocol management module.

[0018] By adopting the above technical solution, the intelligent door lock protocol control system achieves efficient collaboration and standardized communication by clearly defining the responsibilities of the A-core module, B-core module, and protocol management module. The A-core module, acting as a "bridge" between the system and the host computer, is responsible for establishing and maintaining the communication connection, ensuring accurate reception of control commands and timely feedback of status results. The B-core module focuses on core facial image acquisition and algorithm processing, directly connecting to the camera and utilizing its powerful computing capabilities to execute complex facial recognition and registration sub-processes, ensuring efficient and accurate biometric processing. The protocol management module acts as a "translator" and "order maintainer," uniformly encapsulating and parsing control commands, intermediate status messages, and final results transmitted between the A-core module, B-core module, and the host computer according to a predefined structured format. This structured protocol design ensures that data transmission has a unified format standard. For example, the message header can be specified to include fields such as command type and data length, the message body to include specific parameters or status information, and the message tail to include a checksum to ensure data integrity. When core module A receives a control command from the host computer, the protocol management module parses it, extracting the command type (such as face recognition or face registration) and related parameters. Core module A then forwards the parsed command to core module B. During the face processing flow, intermediate status messages generated by core module B at preset key nodes (such as data acquisition completion, feature extraction, and matching) are also encapsulated in a prescribed format by the protocol management module before being reported to core module A. Similarly, the final processing result generated by core module B also needs to be encapsulated by the protocol management module before being fed back to the host computer by core module A. This mechanism of unified encapsulation and parsing through the protocol management module not only ensures the accuracy and consistency of data during transmission, avoiding communication chaos and parsing errors caused by inconsistent protocols, but also makes the entire system's communication process standardized and orderly, facilitating subsequent maintenance, upgrades, and functional expansion. This effectively solves the problem of inconsistent and untraceable protocols in existing technologies, improving the overall stability and development efficiency of the system.

[0019] Optionally, the B core module includes: A door lock control processor is used to control the camera to complete image acquisition; A face recognition processor is used to run algorithms for face detection, feature extraction, template matching, and result determination. The door lock control processor works in conjunction with the face recognition processor to complete the face image acquisition and algorithm processing flow, and generates the intermediate state message at multiple preset nodes of the flow.

[0020] By adopting the above technical solution, the B-core module is further subdivided into a door lock control processor and a face recognition processor. These two processors work together to improve the professionalism and efficiency of face image acquisition and algorithm processing. The door lock control processor, as the direct control unit for image acquisition, is responsible for low-level communication with the camera. Based on control commands received from the B-core module (such as face recognition or registration commands), it precisely controls the camera's activation, frame rate adjustment, and exposure parameter settings to ensure that the quality of the acquired original image meets the requirements of subsequent algorithm processing. The face recognition processor, on the other hand, focuses on complex algorithmic operations, integrating a series of core algorithm modules such as face detection, key point localization, affine transformation, quality assessment, feature extraction, template matching, and final result determination. After the door lock control processor completes the acquisition of one or more frames of image data, it transmits the original image data to the face recognition processor, which processes it according to a preset algorithm flow. Throughout the entire processing flow, the door lock control processor and the face recognition processor achieve efficient data interaction and status synchronization through internal data buses or shared memory. For example, in the face recognition sub-process, after the door lock control processor acquires an image, it notifies the face recognition processor to perform face detection. After the face recognition processor completes the detection and determines the face region, it may send feedback to the door lock control processor, prompting it to proceed with the next frame acquisition or adjust the acquisition parameters. Simultaneously, at multiple preset nodes in the process, such as successful camera startup, image acquisition start, image acquisition completion, entry into the feature extraction stage, and end of the matching process, the door lock control processor and the face recognition processor will generate corresponding intermediate status messages, either individually or collaboratively, according to their respective responsibilities, and report them to the A-core module through the protocol management module. This division of responsibilities clearly defines the functional boundaries within the B-core module. The door lock control processor focuses on the real-time performance of hardware control and data transmission, while the face recognition processor focuses on the high accuracy and efficiency of the algorithm. Working together, they ensure the smooth execution of the face processing flow and the accurate reporting of intermediate statuses.

[0021] Optionally, the predefined structured protocol message includes a command field, a status code field, a data payload field, and a unique message number field.

[0022] By adopting the above technical solution, predefined structured protocol messages construct complete and standardized communication data units by setting command word fields, status code fields, data payload fields, and unique message number fields. The command word field clarifies the type of the current message, such as whether it is a control command (e.g., face registration command, face recognition command), an intermediate status message, or a final result feedback, enabling the receiver to quickly identify the core intent of the message and process it accordingly. The status code field identifies the execution status of the current operation or process; for example, "00" indicates success, "01" indicates image acquisition failure, and "02" indicates feature extraction error. Through standardized status coding, the receiver can intuitively understand the progress of the operation or the cause of the failure, facilitating problem localization and system debugging. The data payload field is the core content carrier of the message. Depending on the message type, the data it contains varies: for control commands, the data payload may include command parameters (such as the recognition time window length, the first preset threshold number, etc.); for intermediate status messages, it may contain a description of the current processing stage (such as "feature fusion in progress") or key data (such as the number of qualified image frames acquired); for the final result, it contains the specific judgment result (such as "recognition successful") and related information (such as user identifier). The unique message number field assigns a unique identifier to each message. This number remains consistent throughout the message's sending, transmission, reception, and feedback processes, enabling the system to track each message throughout its entire process. For example, when the host computer sends a face registration command, it carries a unique message number; all intermediate status messages reported by the B-core module during processing, as well as the final registration result, will carry the same message number. This mechanism effectively solves the message traceability problem. When communication anomalies or discrepancies occur, developers can quickly locate the corresponding complete communication link using the message number and query information such as the message's sending time, processing node, and status changes, greatly improving system maintainability and troubleshooting efficiency. This structured protocol message design with multiple fields ensures the accuracy, integrity, and traceability of various information transmissions in the smart door lock protocol control system, providing a solid communication guarantee for the stable operation of the system.

[0023] Optionally, the smart door lock protocol control system further includes a template update module; The template update module is used to update the feature templates of the corresponding users stored in the template library based on the qualified images and their matching similarity scores when the B-core module performs the face recognition process.

[0024] By adopting the above technical solution, the introduction of the template update module enables dynamic optimization of user feature templates, further enhancing the system's adaptability and recognition accuracy. In practical applications, a user's facial features may change slowly over time, such as with age, hairstyle changes, wearing glasses, etc., or the initial registration template may not fully cover all common user states. When the B-core module executes the face recognition process, if the final recognition is successful (i.e., generating the final result of "recognition successful"), the template update module analyzes the images and matching data involved in this recognition process. First, it acquires the face images that have passed the quality assessment in this recognition (these images have undergone preprocessing such as face detection, key point localization, and affine transformation, and have high quality), and simultaneously extracts the real-time feature vector corresponding to the image and the matching similarity score with the corresponding user feature template in the template library. When the similarity score reaches the preset update threshold (this threshold is usually set at a high level to ensure a high degree of consistency between the updated image and the original template), the template update module triggers the template update mechanism. The update method can be to fuse the real-time feature vector with the original feature template (for example, using a weighted average, where the weights of the real-time feature vector can be dynamically adjusted according to the similarity score, with higher scores resulting in higher weights) to generate a new fused feature template; or it can directly replace the original template with the high-quality real-time feature vector, or retain multiple representative feature templates for the user in a template library to form a template set. Subsequently, the newly generated feature template or template set will overwrite the original user feature template, completing the dynamic update of the template. In this way, the user's feature template can be continuously optimized with the high-quality images collected during actual use, continuously adapting to subtle changes in the user's facial features, thereby avoiding the problem of decreased recognition rate due to template aging, maintaining high recognition accuracy and robustness of the system in the long term, and improving the long-term user experience.

[0025] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Through a registration mechanism involving multi-frame acquisition, rigorous screening, and feature fusion, the quality and richness of facial feature templates have been significantly improved, effectively enhancing the accuracy and anti-interference capabilities of subsequent recognition, thus laying a core foundation for the reliable operation of smart door locks; 2. By defining the responsibilities of the A-core module, B-core module, and protocol management module and designing a structured protocol, the standardized transmission and parsing of control commands, intermediate status messages, and final results are achieved. This solves the communication chaos and untraceability problems caused by inconsistent protocols in existing technologies, and improves the overall stability, communication efficiency, and ease of development and maintenance of the system. 3. By coordinating the door lock control processor and the face recognition processor within the B-core module, a professional division of labor between image acquisition control and algorithm processing is achieved, ensuring the efficient execution of the face processing flow and the accurate reporting of intermediate states, further optimizing system performance; 4. By introducing a template update module, the user feature template is dynamically updated based on high-quality images and matching scores during the recognition process. This allows the template to continuously adapt to the natural changes in the user's facial features, effectively avoiding the template aging problem and maintaining the system's high recognition accuracy and user experience in the long term. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a smart door lock protocol control method disclosed in an embodiment of this application; Figure 2 This is a schematic diagram of the architecture of an intelligent door lock protocol control system disclosed in an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0028] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized. like Figure 1 The diagram shown is a flowchart illustrating a smart door lock protocol control method provided in an embodiment of this application. This method is applied to a dual-core system containing an A core and a B core, and includes the following steps: S1, core A receives control commands from the host computer. The control commands include at least a face recognition command or a face registration command.

[0030] Specifically, the A core, as the main control core of the system, is responsible for communicating and interacting with the host computer. When facial recognition verification or new user facial registration is required for the smart lock, it sends corresponding control commands to the host computer. The A core receives these control commands through a preset communication interface (such as UART, SPI, or Ethernet interface) and performs preliminary verification of the command's legality and completeness to ensure that the received command conforms to the system's specified format and protocol requirements. For example, it verifies the start bit, end bit, and checksum of the command frame. If an error in the command format or a verification failure is found, the A core returns a command error response to the host computer, waiting for the host computer to resend the correct control command; if the verification passes, it proceeds to the next processing step.

[0031] In this embodiment of the application, the control command received by the host computer may be triggered by the user through an APP or by the door lock's local sensor.

[0032] S2, core A forwards control commands to core B.

[0033] Specifically, core A and core B communicate via an internal high-speed communication bus (such as the internal integrated circuit I). 2 C. Data interaction is achieved through a serial peripheral interface (SPI) or a dedicated inter-core communication channel. Before forwarding control commands, core A encapsulates the control commands according to a protocol, adding information such as command identifier, data length, and target core address according to a preset inter-core communication protocol to ensure that core B can accurately identify and parse the command.

[0034] For example, if the control command includes at least a "face registration command", core A will add "0x02" as a command identifier to the header of the command frame, followed by metadata such as the user ID of the face to be registered and the data length, and then send the encapsulated complete command frame to core B through the communication bus. After receiving the command frame, core B first performs frame header verification and length verification, and then proceeds to the command parsing stage after confirming that there are no errors.

[0035] The command scheduling module of core A determines the task type based on the parsed cmd field. Subsequently, it does not directly process the business logic, but instead accurately forwards the complete or repackaged command message to core B via inter-core communication (IPC) (e.g., writing to a shared memory region agreed upon by both cores and sending an interrupt signal). Simultaneously, core A maintains a task status table with msgid as the key, recording the task's initiation time and current status (e.g., "forwarded to core B"), laying the foundation for subsequent status tracking and timeout management.

[0036] S3, the B core executes the face processing flow corresponding to the control commands, and reports intermediate status messages representing the execution status of the node to the A core at multiple preset nodes in the face processing flow.

[0037] Specifically, the B core, acting as a dedicated face processing core, initiates the corresponding face processing flow upon receiving control commands forwarded by the A core. For face recognition commands, the B core first invokes the image acquisition driver to control the camera module to capture the face image of the person to be identified. It then preprocesses the image, including grayscale conversion, noise reduction, face region detection, and cropping, to extract clear facial feature regions. Next, the B core inputs the preprocessed face image into a built-in face recognition algorithm model (such as a deep neural network-based model) for feature extraction and template comparison, calculating the similarity between the current face features and registered face features in the database. For face registration commands, the B core similarly performs image acquisition and preprocessing, then extracts face features, associates the extracted feature template with the user ID, and stores it in a local secure storage area or a cloud database.

[0038] When the control command includes at least a face recognition command, the face processing flow is a face recognition sub-flow. This sub-flow specifically includes: the B-core controlling the camera to acquire one or more frames of original images, and reporting a first intermediate state after acquisition; performing face detection on each frame of the original image to obtain at least one face detection box; locating key points on the image containing the face detection box to obtain face key point coordinates; performing an affine transformation on the image based on the face key point coordinates to obtain an aligned face image; performing quality judgment on the aligned face image and outputting a quality judgment result; if the quality is unqualified, reporting an intermediate state message containing a rejection reason; extracting features from the aligned face image that has passed the quality judgment to obtain a target feature vector; performing similarity matching between the target feature vector and pre-stored user feature templates in the template library to obtain a matching similarity score; and generating a final recognition result based on the matching similarity score, combined with preset dual-threshold rules and continuity decision rules.

[0039] The dual-threshold rule sets a high threshold and a low threshold, and generates a "candidate decision for recognition pass", "rejection decision", or "pending decision" based on the matching similarity score. Specifically, the matching similarity score is compared with a preset high threshold and a preset low threshold; if the matching similarity score is greater than or equal to the preset high threshold, a candidate decision for recognition pass is generated; if the matching similarity score is less than or equal to the preset low threshold, a rejection decision is generated; if the matching similarity score is between the preset low threshold and the preset high threshold, a pending decision is generated.

[0040] The continuous decision rule counts the decision results of multiple consecutive frames of images within a preset recognition time window. When the number of occurrences of "recognition passed candidate decision" reaches a first preset number, a final recognition result of successful recognition is generated; otherwise, a final recognition result of failure is generated. Specifically, within the preset recognition time window, the matching similarity score and decision result corresponding to multiple consecutive frames of images are obtained; the number of occurrences of the recognition passed candidate decision is counted; if the number of occurrences of the recognition passed candidate decision reaches a first preset number, the final recognition result of successful recognition is generated; if the number of occurrences of the recognition rejected decision reaches a second preset number, or the total time of the face recognition sub-process exceeds a preset duration, the final recognition result of failure is generated.

[0041] For example, the preset high threshold is 85 points, the preset low threshold is 70 points, the first preset number is 3 times, the second preset number is 2 times, and the preset duration is 5 seconds. When the camera continuously captures 5 frames of images within 5 seconds, the first frame has a similarity score of 88 points (above the high threshold, generating a candidate judgment for recognition success), the second frame has a score of 75 points (between the high and low thresholds, generating a pending judgment), and the third frame has a score of 90 points (above the high threshold, generating a candidate judgment for recognition success). At this point, the number of times "candidate judgment for recognition success" appears has reached the first preset number of 3 times (note that only 3 frames are listed in the example; if strictly calculated as 5 frames, it can be assumed that the score of the subsequent fourth frame is 86 points, which also generates a candidate judgment for recognition success, and the first preset number is met at the fourth frame). Then, the system immediately generates the final recognition result of successful recognition and controls the door lock to perform the unlocking operation. Conversely, if within the 5-second recognition time window, the score of the first frame is 65 points (below the low threshold, generating a recognition rejection decision), and the score of the second frame is 68 points (below the low threshold, generating a recognition rejection decision), and the number of times the "recognition rejection decision" occurs reaches the second preset number of 2, then the system generates the final recognition result of recognition failure, the door lock remains locked, and an intermediate status message of "face feature mismatch" is reported.

[0042] When the control command includes at least a face registration command, the face processing flow is a face registration sub-flow. This sub-flow specifically includes: acquiring multiple frames of original registration images; sequentially performing face detection, key point localization, affine transformation, and quality assessment on each frame of the original registration images to select multiple qualified registration images; extracting features from each frame of the qualified registration images to obtain multiple registration feature vectors; fusing the multiple registration feature vectors to generate a fused feature template; storing the fused feature template in a template library, assigning a user identifier to the fused feature template, and generating a registration success result containing the user identifier.

[0043] For example, when acquiring multiple frames of original registration images, the frame rate can be set to 5-10 frames to ensure sufficient samples for subsequent screening. In the face detection stage, the MTCNN (Multi-Task Cascaded Convolutional Neural Network) algorithm, based on deep learning, is used to quickly and accurately locate face regions in images, maintaining a high detection rate even in low light or when the face is slightly angled. Keypoint localization precisely extracts 68 feature points on the face, including key areas such as the eyes, nose, mouth, and chin, providing accurate coordinates for subsequent affine transformations. The affine transformation process uniformly adjusts face images of different poses and angles to a preset standard pose and size, such as centering the face and ensuring the line connecting the eyes is parallel to the horizontal line, normalizing the size to 150×150 pixels for subsequent feature extraction and comparison. The quality assessment process evaluates images from multiple dimensions, including sharpness (measured by calculating the image's gradient value or variance; a larger gradient value or variance indicates a sharper image), illumination uniformity (by detecting the image's brightness distribution to avoid excessively large overexposed or underexposed areas), pose angle (ensuring the face's angle is within ±15 degrees), and occlusion (if key areas such as the eyes, nose, and mouth are occluded by more than 30%, the image is deemed unqualified). Only images that simultaneously meet all quality indicators are marked as qualified registration images. Assuming 8 original registration images are collected, after the above processing, 5 qualified registration images are selected. Then, depth features are extracted from these 5 qualified images, for example, using a ResNet-50 network as the feature extractor, resulting in 5 registration feature vectors with a dimension of 2048. During the fusion process, a weighted average method is used. Different weights are assigned to each qualified image frame based on its quality score; images with higher quality scores have greater weights. For example, the image with the highest quality score has a weight of 0.3, and the weights of the remaining images are 0.25, 0.2, 0.15, and 0.1, respectively. The five registered feature vectors are then summed according to their corresponding weights to obtain the fused feature template. Finally, this fused feature template is stored in a local template library and automatically assigned a unique user identifier, such as "USER001." Simultaneously, a message "Registration successful, User identifier: USER001" is generated and sent to the user.

[0044] At several key pre-defined nodes in the aforementioned face processing flow, core B proactively sends intermediate status messages to core A. For example, when core B begins image acquisition, it sends a "Face image acquisition started" status message; after image acquisition is complete and preprocessing is error-free, it sends a "Image preprocessing completed, proceeding to feature extraction stage" message; if a camera malfunction or insufficient lighting prevents the acquisition of a valid image during image acquisition, it sends a "Image acquisition failed: [Specific reason, such as abnormal camera connection / inadequate image quality]" status message. During feature extraction, if feature extraction fails due to excessive facial pose or severe occlusion, core B sends a "Feature extraction failed: [Specific reason, such as face deflection angle exceeding limits / excessive occlusion area]" message; after successful feature extraction, it sends a "Feature extraction completed, preparing for template comparison / storage" message. For face recognition commands, after template comparison is completed, core B will send an intermediate result message: "Template comparison complete, similarity value: [specific value]". For face registration commands, when feature template storage is completed or fails, it will send a status message: "Face feature template stored successfully" or "Template storage failed: [specific reason, such as storage medium full / insufficient permissions]". These intermediate status messages are all encapsulated according to a preset format, including message type, command ID, node identifier, status code, and optional additional information (such as error reason, similarity value, etc.) so that core A can clearly grasp the real-time progress of the face processing flow.

[0045] S4, the A core feeds back the received intermediate state messages and / or the final result of the face processing process to the host computer.

[0046] Specifically, after receiving the intermediate status message reported by core B, core A parses the message, extracting the command ID, status code, and additional information. Then, core A encapsulates this intermediate status information into a format that meets the requirements of the host computer (such as JSON format or a custom binary protocol format) according to a preset communication protocol, and sends it to the host computer in real time through the communication interface (such as the previously mentioned UART, SPI, or Ethernet interface). For example, when receiving the intermediate status message "Image acquisition failed: Camera connection abnormal," core A will package this information together with the corresponding command ID and send it to the host computer so that the user or the host computer system can be informed of the current abnormal situation of the face processing task. After core B completes the entire face processing flow and generates the final result (such as face recognition success / failure, face registration success / failure), core B will send the final result to core A in a specific message format. After receiving the final result, core A also encapsulates it according to the protocol to ensure the accuracy and completeness of the result, and sends it to the host computer as the final response. At the same time, core A updates the status of the corresponding task in the task status table to "completed" and records the completion time and the final result. If the host computer needs to confirm the final result or perform further operations, the A core will process the subsequent instructions from the host computer. like Figure 2 The diagram shown is an architectural schematic of a smart door lock protocol control system provided by this application. This embodiment provides a smart door lock protocol control system for implementing the smart door lock protocol control method described in Embodiment 1. (Refer to...) Figure 2 This intelligent door lock protocol control system includes an A-core module, a B-core module, and a protocol management module. The A-core module is responsible for communication with the host computer, the B-core module is responsible for image acquisition and algorithm processing, and the protocol management module is responsible for message encapsulation and parsing. These modules work together to enable the intelligent door lock system to accurately and efficiently process control commands and feedback messages, achieving standardized and stable system communication and enabling full-process control. The predefined structured format makes message transmission clearer, and the clear division of labor among modules avoids state asynchrony and execution errors.

[0047] Specifically, the A-core module is used to establish a communication connection with the host computer. It can connect to the host computer via various wireless communication methods, such as Bluetooth and Wi-Fi. The A-core module contains a communication interface circuit and a processing chip. The communication interface circuit is responsible for data interaction with the host computer, and the processing chip performs preliminary data parsing and processing. Alternatively, an integrated module with communication and processing capabilities can be used instead.

[0048] The B-core module connects to the camera and is used for facial image acquisition and algorithm processing. The door lock control processor within the B-core module controls the camera to complete image acquisition; it can send control signals to adjust camera parameters such as exposure time and gain. The facial recognition processor runs algorithms for face detection, feature extraction, template matching, and result determination. The door lock control processor and the facial recognition processor interact and work collaboratively via an internal bus. When processing images, the door lock control processor first controls the camera to acquire images, and then transmits the image data to the facial recognition processor for further processing. The door lock control processor can be implemented using a microcontroller, while the facial recognition processor can use a chip with powerful computing capabilities, such as an FPGA chip, or other chips capable of performing the corresponding functions.

[0049] The protocol management module encapsulates and parses control commands, intermediate status messages, and final results according to a predefined structured format. The predefined structured protocol message includes a command word field, a status code field, a data payload field, and a unique message number field. The command word field identifies different commands, the status code field indicates the current status, the data payload field carries the specific data, and the unique message number field is used for message acknowledgment and tracking. The protocol management module can be implemented using software or a dedicated protocol processing chip.

[0050] In this module, core A receives control commands from the host computer through the protocol management module and forwards them to core B. Core A encapsulates the received commands according to the protocol format and sends them to the protocol management module. The protocol management module parses and verifies the commands and then forwards them to core B.

[0051] The B-core module, through the protocol management module, reports intermediate status messages to the A-core module at multiple preset nodes in the face image acquisition and algorithm processing process. The B-core module generates intermediate status messages at key nodes, encapsulates them into protocol message formats, and sends them to the A-core module through the protocol management module.

[0052] The A-core module feeds back intermediate status messages and / or final results to the host computer through the protocol management module. The A-core module organizes the received messages and results, encapsulates them into protocol messages again through the protocol management module, and sends them to the host computer.

[0053] The intelligent door lock protocol control system also includes a template library, which stores the fused feature templates generated by the B-core module. The template library can be a storage device such as flash memory or EEPROM, or it can use an external memory card. It has data storage and retrieval functions, enabling rapid storage and retrieval of feature templates.

[0054] The smart door lock protocol control system also includes a template update module. This module updates the feature templates for the corresponding user stored in the template library based on qualified images and their matching similarity scores during the face recognition process executed by the B-core module. When the real-time score is greater than or equal to (T_high + 0.05) and the quality is qualified, the template update module will continuously replace the oldest template. The template update module can be implemented in software or as a hardware module with data processing and storage capabilities.

[0055] The implementation principle of this embodiment is as follows: This smart lock protocol control system achieves comprehensive control over the smart lock identification and registration process through the collaborative work of its various modules. The protocol management module ensures the standardization and accuracy of message transmission, while the template library and template update module enable the system to continuously update and optimize user feature templates, improving identification accuracy. The clear division of labor and close cooperation among the modules solves the problems of fragmented communication protocols and incomplete status feedback in existing lock identification systems, enhancing system stability and reliability and providing users with a safer and more convenient experience.

[0056] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0057] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A smart door lock protocol control method, characterized in that, Applied to a dual-core system containing core A and core B, the method includes: Core A receives control commands from the host computer, the control commands including at least a face recognition command or a face registration command; Core A forwards the control command to Core B; B core executes the face processing flow corresponding to the control command, and at multiple preset nodes of the face processing flow, reports intermediate status messages representing the execution status of the node to A core. Core A will feed back the received intermediate state messages and / or the final result of the face processing process to the host computer. 2.The intelligent door lock protocol control method of claim 1, wherein, When the control command includes at least a face recognition command, the face processing flow executed by the B core includes a face recognition sub-flow, which includes: One or more frames of raw image are acquired; Face detection is performed on each frame of the original image to obtain at least one face detection bounding box; Key point localization is performed on the image containing the face detection box to obtain the coordinates of the face key points; An affine transformation is performed on the image based on the coordinates of the facial key points to obtain an aligned facial image. The quality of the aligned face image is assessed, and the quality assessment result is output. Feature extraction is performed on the aligned face images that have passed the quality assessment to obtain the target feature vector; The target feature vector is matched with the user feature templates pre-stored in the template library to obtain a matching similarity score; Based on the matching similarity score, combined with the preset dual threshold rule and continuity decision rule, the final recognition result is generated.

3. The intelligent door lock protocol control method according to claim 2, characterized in that, The application of the dual threshold rule includes: The matching similarity score is compared with a preset high threshold and a preset low threshold; If the matching similarity score is greater than or equal to the preset high threshold, a candidate decision for recognition is generated; If the matching similarity score is less than or equal to the preset low threshold, a rejection decision is generated. If the matching similarity score is between the preset low threshold and the preset high threshold, a pending decision is generated.

4. The intelligent door lock protocol control method according to claim 3, wherein, The application of the continuity decision rule includes: Within a preset recognition time window, the matching similarity scores and judgment results corresponding to multiple consecutive frames of images are obtained; The number of times the candidate decision is passed is counted. If the number of times the candidate decision appears reaches a first preset number, then the final recognition result of successful recognition is generated; If the number of times the recognition rejection decision occurs reaches a second preset number, or the total time consumed by the face recognition sub-process exceeds a preset duration, then the final recognition result indicating recognition failure is generated. 5.The intelligent door lock protocol control method of claim 1, wherein, When the control command includes at least a face registration command, the face processing flow executed by the B core includes a face registration sub-flow, which includes: Acquire multiple frames of original registration images; For each frame of the original registration image, face detection, key point localization, affine transformation and quality judgment are performed sequentially to select multiple frames of qualified registration images; Extract features from each frame of the qualified registered image to obtain multiple registration feature vectors; The multiple registered feature vectors are fused to generate a fused feature template; The fused feature template is stored in the template library, and a user identifier is assigned to the fused feature template to generate a registration success result containing the user identifier.

6. An intelligent door lock protocol control system, characterized by, Applied to smart locks, and used to execute the smart lock protocol control method as described in any one of claims 1 to 5, the smart lock protocol control system includes an A-core module, a B-core module, and a protocol management module: The A-core module is used to establish a communication connection with the host computer; The B-core module is connected to the camera and is used to perform face image acquisition and algorithm processing; The protocol management module is used to encapsulate and parse control commands, intermediate status messages and final results according to a predefined structured format; The A-core module receives control commands from the host computer through the protocol management module and forwards them to the B-core module; the B-core module reports intermediate status messages to the A-core module through the protocol management module at multiple preset nodes in the face image acquisition and algorithm processing process. The A-core module feeds back the intermediate status messages and / or final results to the host computer through the protocol management module.

7. The intelligent door lock protocol control system of claim 6, wherein, The B-core module includes: A door lock control processor is used to control the camera to complete image acquisition; A face recognition processor is used to run algorithms for face detection, feature extraction, template matching, and result determination. The door lock control processor works in conjunction with the face recognition processor to complete the face image acquisition and algorithm processing flow, and generates the intermediate state message at multiple preset nodes of the flow.

8. The intelligent door lock protocol control system of claim 6, wherein, The predefined structured protocol message includes a command field, a status code field, a data payload field, and a unique message number field.

9. The intelligent door lock protocol control system of claim 6, wherein, The intelligent door lock protocol control system also includes a template library, which stores the fused feature templates generated by the B core module.

10. The intelligent door lock protocol control system of claim 9, wherein, The intelligent door lock protocol control system also includes a template update module; The template update module is used to update the feature templates of the corresponding users stored in the template library based on the qualified images and their matching similarity scores when the B-core module performs the face recognition process.

Images