Data processing method and apparatus, device, and storage medium

Abstract

The application provides a data processing method, device and equipment and a storage medium. The data processing method comprises the following steps: in response to a data processing request for a target smart door lock, determining a region environment type of a target access control region of the target smart door lock; based on the region environment type, controlling a multi-modal acquisition module of the target smart door lock to acquire multi-modal biological data of a target interactive object in the target access control region; based on the multi-modal biological data, performing object recognition on the target interactive object to obtain access control interaction permission of the target interactive object; and based on the access control interaction permission, performing access control management on the target interactive object to obtain an access control result. The technical solution of the application can improve the data acquisition quality of the smart door lock, and the object recognition is performed in a multi-modal manner, thereby improving the safety and accuracy of the access control management.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, with the rapid development of IoT and smart lock technologies, more and more users are using smart locks for biometric identification to achieve intelligent access control. Existing smart locks generally adopt a "single biometric identification + basic access management" model, typically using a fixed-angle camera to collect the user's facial information. This information is then compared with pre-stored templates locally or in the cloud, and unlocked upon successful matching. However, this access control method suffers from drawbacks such as low recognition accuracy, inconvenient operation, and low security. Summary of the Invention

[0003] This application provides a data processing method, apparatus, device, and storage medium, aiming to solve the technical problem of low recognition accuracy in the unlocking process of smart access control in the prior art.

[0004] On one hand, embodiments of this application provide a data processing method, which includes the following steps: In response to a data processing request for the target smart lock, determine the regional environment type of the target access control area of ​​the target smart lock; Based on the regional environment type, the multimodal acquisition module of the target smart door lock is controlled to collect multimodal biological data of the target interactive object in the target access control area; Based on the multimodal biological data, object recognition is performed on the target interaction object to obtain the access control interaction permissions of the target interaction object; Access control is performed on the target interactive object based on the access control interaction permissions to obtain the access control result.

[0005] In one possible implementation of this application, determining the regional environment type of the target access control area of ​​the target smart door lock includes: Collect the regional environmental information of the target access control area of ​​the target smart door lock; The regional environment type of the target access control area is identified based on the regional environmental information.

[0006] In one possible implementation of this application, the regional environmental information includes any one or more of ambient light intensity data, image grayscale distribution parameters, and moving pixel percentage data; The step of identifying the regional environment type of the target access control area based on the regional environment information includes: When the ambient light intensity data is less than a preset ambient light intensity threshold, the area environment type of the target access control area is determined to be a low light environment type; When the image grayscale distribution parameters meet the preset backlighting conditions, the regional environment type of the target access control area is determined to be a backlighting environment type. The preset backlighting conditions are that the first distribution parameter in the image grayscale distribution parameters is greater than the first distribution threshold and the second distribution parameter is greater than the second threshold. When the proportion of moving pixels exceeds a preset pixel proportion threshold, the area environment type of the target access control area is determined to be a dynamic environment type.

[0007] In one possible implementation of this application, the multimodal acquisition module that controls the target smart door lock based on the regional environment type acquires multimodal biometric data of the target interactive object in the target access control area, including: Adjust the acquisition settings parameters of the multimodal acquisition module of the target smart door lock based on the regional environment type; The multimodal acquisition module is driven to acquire multimodal biological data of the target interactive object in the target access control area according to the acquisition settings parameters.

[0008] In one possible implementation of this application, the step of driving the multimodal acquisition module to acquire multimodal biometric data of the target interactive object in the target access control area according to the acquisition setting parameters includes: The first image of the target interactive object is acquired according to the first image acquisition module in the multimodal acquisition module and the acquisition setting parameters to obtain the first object image data; The second image of the target interactive object is acquired by the second image acquisition module in the multimodal acquisition module and the acquisition setting parameters to obtain the second object image data; The audio acquisition module in the multimodal acquisition module is used to acquire the voice interaction commands output by the target interactive object to obtain the object's audio data; The first object image data, the second object image data, and the object audio data are identified as the multimodal biological data of the target interactive object.

[0009] In one possible implementation of this application, the step of performing object recognition on the target interaction object based on the multimodal biological data to obtain the access control interaction permissions of the target interaction object includes: Feature extraction is performed based on the multimodal biological data and feature fusion model to obtain multimodal biological features; Based on the multimodal biometrics and preset member features, identity matching is performed to determine the object identity information of the target interaction object; Access a preset database to determine the access control interaction permissions associated with the object's identity information in the preset database.

[0010] In one possible implementation of this application, the step of performing access control on the target interaction object based on the access control interaction permission to obtain the access control result includes: Based on the access control interaction permissions, the target interaction object is verified and / or unlocked to obtain the access control interaction result. If the access control interaction result and / or the lock status of the target smart lock meet the abnormal access control conditions, then the access control interaction result is determined to be an abnormal interaction result. Based on the preset alarm strategy, the abnormal alarm message corresponding to the abnormal interaction result is output to obtain the access control result.

[0011] On the other hand, this application provides a data processing apparatus, the data processing apparatus comprising: The environmental detection module is configured to respond to a data processing request for the target smart lock and determine the regional environmental type of the target access control area of ​​the target smart lock; The data acquisition module is configured to control the multimodal acquisition module of the target smart door lock to acquire multimodal biological data of the target interactive object in the target access control area based on the regional environment type; The object recognition module is configured to perform object recognition on the target interactive object based on the multimodal biological data, and obtain the access control interaction permissions of the target interactive object; The access control module is configured to perform access control management on the target interaction object based on the access control interaction permissions, and obtain the access control result.

[0012] On the other hand, this application also provides a data processing device, the data processing device comprising: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps of the data processing method.

[0013] On the other hand, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the data processing method.

[0014] In this application, by responding to a data processing request for a target smart lock, the regional environment type of the target access control area of ​​the target smart lock is determined; based on the regional environment type, the multimodal acquisition module of the target smart lock is controlled to collect multimodal biometric data of the target interactive object in the target access control area; based on the multimodal biometric data, object recognition is performed on the target interactive object to obtain the access control interaction permission of the target interactive object; based on the access control interaction permission, access control is performed on the target interactive object to obtain the access control result. This system enables the identification of the target access control area's environmental type during smart lock access control. The operating mode of the corresponding multimodal acquisition module in the smart lock is adjusted according to this environmental type, allowing the multimodal acquisition module to accurately and effectively collect multimodal biometric data of the target interaction object within that environmental type. This improves the accuracy of subsequent access control. The system then uses the collected multimodal biometric data to identify the target interaction object, thereby obtaining its identity information and associated access control permissions. These permissions are then used for access control management. This process enhances the data acquisition quality of the smart lock and enables multimodal object identification, ultimately improving the security and accuracy of access control. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram illustrating a scenario of the data processing method in an embodiment of this application. Figure 2 This is a flowchart illustrating one embodiment of the data processing method in this application. Figure 3 A flowchart illustrating one embodiment of the data processing method for identifying regional environment types provided in this application; Figure 4 A flowchart illustrating an embodiment of access control in the data processing method provided in this application; Figure 5 A schematic diagram of the structure of one embodiment of the data processing apparatus provided in this application; Figure 6 This is a schematic diagram of one embodiment of the data processing device provided in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0019] In this application, the term "exemplary" is used to mean "serving as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0020] Financial and banking service companies typically need to interface with multiple brand vendors during operation. The current data interface method involves customized development according to the brand vendor's requirements, designing table structures to store and send data based on the outgoing fields, without checking whether the data was sent successfully. This method increases the error rate and fails to guarantee data integrity, leading to increased workload in the version process and extending the time to go live, which is required for scenarios that demand rapid integration and deployment.

[0021] Based on this, this application proposes a data processing method, apparatus, device, and computer-readable storage medium to solve the technical problem of high error rate in the data docking process in the prior art.

[0022] The data processing method in this embodiment of the invention is applied to a data processing device, which is disposed in a data processing equipment. The data processing equipment is provided with one or more processors, a memory, and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the data processing method. The data processing equipment can be a smart terminal, such as a mobile phone, tablet computer, network device, and smart computer. Optionally, the data processing equipment can also be a server or a service cluster composed of multiple servers.

[0023] like Figure 1 As shown, Figure 1 This is a schematic diagram of a data processing method according to an embodiment of the present application. The data processing scenario in this embodiment includes a data processing device 100 (the data processing device 100 integrates a data processing unit), and the data processing device 100 is equipped with a computer-readable storage medium corresponding to the data processing method to execute the steps of the data processing method.

[0024] Understandable Figure 1 The data processing device in the data processing method scenario shown, or the device included in the data processing device, does not constitute a limitation on the embodiments of the present invention. That is, the number or type of data processing device in the data processing method scenario, or the number or type of device included in each device, does not affect the overall implementation of the technical solution in the embodiments of the present invention, and can all be considered as equivalent substitutions or derivatives of the technical solutions claimed in the embodiments of the present invention.

[0025] In this embodiment of the invention, the data processing device 100 is mainly used to: respond to a data processing request for a target smart door lock and determine the regional environment type of the target access control area of ​​the target smart door lock; Based on the regional environment type, the multimodal acquisition module of the target smart door lock is controlled to collect multimodal biological data of the target interactive object in the target access control area; Based on the multimodal biological data, object recognition is performed on the target interaction object to obtain the access control interaction permissions of the target interaction object; Access control is performed on the target interactive object based on the access control interaction permissions to obtain the access control result.

[0026] This application provides a data processing method, apparatus, device, and computer-readable storage medium, which will be described in detail below.

[0027] It will be understood by those skilled in the art that Figure 1 The application environment shown is only one application scenario related to the solution of this application and does not constitute a limitation on the application scenario of this application. Other application environments may include more than one application scenario. Figure 1 The number of data processing devices shown, or the data processing network connections, for example Figure 1 Only one data processing device is shown in the diagram. It is understood that the scenario of this data processing method may also include one or more data processing devices, which are not specifically limited here. The data processing device 100 may also include a memory for storing regional environment types, multimodal biological data, access control interaction permissions, and other data.

[0028] It should be noted that, Figure 1 The schematic diagram of the data processing method shown is merely an example. The scenarios of the data processing method described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided in the embodiments of the present invention.

[0029] Based on the scenarios described above, various embodiments of the data processing method disclosed in this invention are proposed.

[0030] like Figure 2 As shown, Figure 2 This is a flowchart illustrating one embodiment of the data processing method in this application. The data processing method includes the following steps 201 to 204: 201. In response to a data processing request for the target smart door lock, determine the regional environment type of the target access control area of ​​the target smart door lock; The data processing method in this embodiment is applied to a data processing device. The type and number of data processing devices are not specifically limited. That is, the data processing device can be one or more smart terminals or servers that serve as data processing modules for the target smart lock. In a specific embodiment, the data processing device can be a smart terminal or server used to process data on the target smart lock. In addition, the data processing device can also be the target smart lock itself.

[0031] Optionally, the target smart lock is a smart lock terminal that collects multimodal biometric data of the target access control area through multimodal acquisition of regional environmental types to perform object recognition and achieve access control. Optionally, in a specific embodiment, the target smart lock includes a multimodal acquisition module, an access control display module, an access control communication module, and an access control alarm module.

[0032] Optionally, the multimodal acquisition module is a data acquisition module used to acquire multimodal biometric data of the target access control area corresponding to the target smart door lock. Optionally, in a specific embodiment, to improve data acquisition accuracy and subsequent access control recognition efficiency, the multimodal acquisition module includes a first image acquisition module, a second image acquisition module, and an audio acquisition module. The first image acquisition module is a camera component that supports acquiring image data of target interactive objects in the target access control area from multiple perspectives. The second image acquisition module is a camera component used to assist in acquiring image data of target interactive objects corresponding to special groups from a top-down or bottom-up perspective. The audio acquisition module is a microphone component used to acquire audio data of target interactive objects in the target access control area. The target interactive objects corresponding to special groups refer to people who are too short (e.g., children), too tall, or have disabilities (e.g., people using wheelchairs) whose frontal images cannot be effectively acquired by the first image acquisition module. Optionally, in a specific embodiment, the first image acquisition module is a camera component with a 180° field of view and supports infrared illumination. The second image acquisition module is a camera component that acquires images from a top-down perspective. The audio acquisition module can be a voiceprint microphone.

[0033] Optionally, the target access control area is the area to be monitored corresponding to the target smart lock; that is, the target access control area is the area where the target interactive object is located when interacting with the target smart lock. For example, in a specific embodiment, the target access control area is the area in front of the target smart lock, and the target interactive object is located in the target access control area to perform object recognition and access control interaction in order to control the access control.

[0034] Optionally, the area environment type is a scene type that characterizes the lighting conditions and the movement of the object to be monitored in the target access control area, used to adjust the operating mode of the multimodal acquisition module of the target smart lock according to different area types. Optionally, in a specific embodiment, the area environment type includes any one or more of the following: strong light environment type / weak light environment type, front lighting environment type / backlighting environment type, and static environment type / dynamic environment type.

[0035] Optionally, the data processing request is an operation event that drives the target smart lock to collect multimodal biometric interactions for access control. That is, when the target interaction object is in the target access control area and interacts with the target smart lock, the data processing request is actively triggered.

[0036] Optionally, when the terminal receives a data processing request, it also determines the regional environment type of the target access control area corresponding to the target smart door lock. In subsequent steps, the terminal adjusts the operating parameters of the multimodal acquisition module accordingly based on the regional environment type to improve the effectiveness of the acquired data. That is, the terminal detects the regional environment information of the target access control area of ​​the target smart door lock and identifies the regional environment type of the target access control area based on this information.

[0037] 202. Based on the regional environment type, control the multimodal acquisition module of the target smart door lock to acquire multimodal biometric data of the target interactive object in the target access control area; Optionally, after obtaining the regional environment type of the target access control area, the terminal also controls the multimodal sampling module of the target smart door lock to collect multimodal biological data of the target interactive object in the target access control area based on the regional environment type.

[0038] Optionally, the target interaction object is a user object that is in the target access control area and interacts with the target smart lock.

[0039] Optionally, the multimodal biometric data refers to the data information collected by the target smart lock from the target interactive object through the multimodal acquisition module, used to characterize the biometric features of the target interactive object. Optionally, in a specific embodiment, the multimodal biometric data includes any one or more of the following: first object image data, second object image data, object height data, and object audio data.

[0040] The first object image data consists of facial image data obtained by acquiring the frontal face of the target interactive object through the first image acquisition module. The second object image data consists of facial image data obtained by adjusting the tilt angle of the target interactive object through the second image acquisition module. The object height data consists of the height data of the target interactive object obtained by distance measurement calculation through the first image acquisition module and / or the second image acquisition module. The object audio data consists of audio feature data (such as fundamental frequency parameters and formant parameters) obtained by acquiring the voice interaction commands of the target interactive object through the audio acquisition module in the multimodal acquisition module.

[0041] Optionally, in order to improve the accuracy of multimodal biological data acquisition, after the terminal obtains the regional environment type of the target access control area, it adaptively adjusts the acquisition setting parameters of the multimodal acquisition module of the target smart lock based on the regional environment type, so that the operating parameters of the multimodal acquisition module can be adapted to the regional environment type, thereby improving the accuracy and effectiveness of multimodal biological data.

[0042] Optionally, the acquisition setting parameters are the operational settings parameters for the multimodal acquisition module when acquiring data. These acquisition setting parameters can be operational mode parameters and specific acquisition parameters within the multimodal acquisition module. For example, these acquisition setting parameters can be operational settings parameters such as infrared illumination parameters, acquisition frame rate, filtering algorithm parameters, and HDR acquisition mode.

[0043] Optionally, in one specific embodiment, when the regional environment type is a low-light environment type, the terminal determines that the acquisition setting parameters corresponding to the multimodal acquisition module to be adjusted are infrared supplementary lighting parameters and noise suppression mode. The infrared supplementary lighting parameters are adjusted within a preset supplementary lighting power range, and the corresponding noise suppression mode is enabled to obtain the acquisition setting parameters of the multimodal acquisition module corresponding to the low-light environment type. Optionally, in one specific embodiment, the supplementary lighting power range corresponding to the infrared supplementary lighting parameters is [1,3] watts. The noise suppression mode is the RetinexNet algorithm.

[0044] Optionally, in another specific embodiment, when the regional environment type is a backlight environment type, the terminal determines that the acquisition setting parameters corresponding to the multimodal acquisition module are the operating mode parameters, the terminal adjusts the operating mode parameters to HDR mode, and enables multi-frame exposure synthesis when acquiring images, so as to obtain the acquisition setting parameters of the multimodal acquisition module corresponding to the backlight environment mode.

[0045] Optionally, in another specific embodiment, when the area environment type is a dynamic environment type, the terminal determines that the acquisition setting parameters corresponding to the multimodal acquisition module are the acquisition frame rate and motion acquisition mode. The terminal increases the acquisition frame rate to the target acquisition frame rate and enables the motion blur elimination algorithm to obtain the acquisition setting parameters of the multimodal acquisition module corresponding to the dynamic environment type.

[0046] Optionally, after the terminal adaptively adjusts the acquisition setting parameters according to the regional environment type, it also drives the multimodal acquisition module to acquire multimodal biological data of the target interactive object in the target access control area based on the acquisition setting parameters.

[0047] Optionally, the terminal performs a first image acquisition on the target interactive object based on the first image acquisition module in the multimodal acquisition module and the acquisition setting parameters, thereby obtaining the first object image data. The acquisition setting parameters corresponding to the first image acquisition module can be any one or more of the following: acquisition frame rate, infrared illumination parameters, acquisition frame rate, filtering algorithm parameters, and HDR acquisition mode.

[0048] Optionally, the terminal performs a second image acquisition on the target interactive object based on the second image acquisition module in the multimodal acquisition module and the acquisition setting parameters, thereby obtaining second object image data. The acquisition setting parameters corresponding to the second image acquisition module include acquisition parameters such as the tilt angle.

[0049] Optionally, the terminal may also use the audio acquisition module in the multimodal acquisition module to acquire the voice interaction commands output by the target interactive object, thereby obtaining the object's audio data.

[0050] Optionally, the terminal may determine the acquired first object image data, second object image data, and object audio data as the multimodal biological data of the target interactive object.

[0051] 203. Based on the multimodal biological data, perform object recognition on the target interaction object to obtain the access control interaction permissions of the target interaction object; Optionally, after acquiring the multimodal biometric data of the target interaction object, the terminal may also perform object recognition on the target object based on the multimodal biometric data to determine the object identity information and corresponding access control interaction permissions of the target interaction object.

[0052] Optionally, after acquiring multimodal biological data, the terminal performs feature extraction based on the multimodal biological data and a feature fusion model to obtain multimodal biological features. That is, the terminal uses the acquired multimodal biological data as model input to perform feature extraction and feature fusion in the feature fusion model to obtain the multimodal biological features corresponding to the target interaction object. Here, the multimodal biological features are fused feature information representing the feature information corresponding to the multimodal biological data of the target interaction object. The feature fusion model is an artificial intelligence model used for feature extraction and weighted fusion of multimodal biological data. This feature fusion model includes a feature extraction module and a feature fusion module.

[0053] That is, after acquiring multimodal biological data, the terminal inputs the multimodal biological data into the feature extraction module for feature extraction, obtaining the first object image features corresponding to the first object image data, the second object image features corresponding to the second object image, the object height features corresponding to the object height data, and the object audio features corresponding to the object audio data from the multimodal biological data. For example, the feature extraction module extracts the first object image features corresponding to the first object image data using an image feature extraction algorithm; and extracts the second object image features corresponding to the second object image data using an image feature extraction algorithm. In addition, the feature extraction module can also extract the object audio features from the object audio data using an audio feature extraction algorithm. Optionally, in a specific embodiment, the image feature algorithm is an improved MobileNetV3 algorithm, and the extracted first and second object image features are 128-dimensional feature vectors. The audio feature extraction algorithm is the MFCC algorithm, and the extracted object audio features are 64-dimensional feature vectors.

[0054] Optionally, after acquiring the first object image features, the second object image features, the object height features, and the object audio features, the terminal inputs these features into a feature fusion module for weighted fusion. Specifically, the feature fusion module pre-sets feature weight information corresponding to each feature of the multimodal biological data. This feature weight information includes image weight information, height weight information, and audio weight information. The image weight information is used to weight the first object image features and / or the second object image features. The height weight information is used to weight the object's height features. The audio weight information is used to weight the object's audio features. In one specific embodiment, the image weight information is set to 0.6, the audio weight information is set to 0.3, and the height weight information is set to 0.1.

[0055] Optionally, the feature fusion module weights the image features of the first object and / or the image features of the second object using image weight information to obtain weighted image features; weights the audio features of the object using audio weight information to obtain weighted audio features; and weights the height features of the object using height weight information to obtain weighted height features. After obtaining the weighted image features, weighted audio features, and weighted height features, the module calculates the sum of the weighted image features, weighted audio features, and weighted height features to obtain the multimodal biometrics of the target object.

[0056] Optionally, after acquiring the multimodal biometrics, the terminal performs identity matching based on the multimodal biometrics and preset member features to determine the object identity information of the target interaction object. That is, after acquiring the multimodal biometrics, the terminal matches the multimodal biometrics with preset member features corresponding to the user identity pre-stored by the target smart lock, calculates the feature similarity between the multimodal biometrics and the preset member features, and compares the feature similarity with a preset identity similarity threshold. The target identity information corresponding to the preset member features with a feature similarity greater than the identity similarity threshold is determined as the object identity information of the target interaction object. Here, the identity similarity threshold is a feature similarity threshold used to determine the object identity information of the target interaction object. Optionally, in a specific embodiment, the identity similarity is a first similarity threshold used to assess whether the target interaction object is an administrator and a second similarity threshold used to assess whether the target interaction object is a temporary visitor. In a specific embodiment, the first similarity threshold is 80%, and the second similarity threshold is 90%. That is, if the similarity between the target interaction object and the preset member characteristics corresponding to any administrator identity is greater than 80%, then the identity of the target interaction object is determined to be the administrator identity corresponding to the preset member characteristics.

[0057] Optionally, the object identity information is category information used to identify or verify the identity of the target interactive object. Optionally, in one specific embodiment, the object identity information includes administrator identity information, visitor identity information, and restricted identity information. Administrator identity information indicates that the target interactive object is an administrator of the target smart lock and has full permissions to the target smart lock. Visitor identity information indicates that the target interactive object is a temporary visitor and has limited access permissions to the target smart lock. Restricted identity information indicates that the target interactive object is a restricted user and has limited access permissions to the target smart lock.

[0058] Optionally, after determining the object identity information of the target interaction object, the terminal accesses a preset database to determine the access control interaction permissions associated with the object identity information in the preset database. That is, the terminal inputs the object identity information into the preset database and obtains the access control interaction permissions associated with the object identity information stored in the preset database. These access control interaction permissions include any one or more of full access permissions, time-limited access permissions, and restricted access permissions. Full access permissions refer to having all permissions to interact with the target smart lock, including unlocking, permission allocation, and log viewing. Time-limited access permissions are limited to unlocking the target smart lock a limited number of times (e.g., once or multiple times) within a set window period (e.g., one hour). Restricted access permissions are limited to unlocking the target smart lock and accessing the restricted logs within a specified time period (e.g., 9:00-11:00).

[0059] Optionally, the terminal can also respond to permission setting commands triggered by the administrator, setting the validity period of the access control interaction permission through these commands, and automatically revoking the access control interaction permission after the validity period expires. In addition, the terminal can also remind the target interaction object when it fails to use the access control interaction permission on time, and synchronize the permission adjustment time to the access control operation log.

[0060] 204. Based on the access control interaction permissions, perform access control management on the target interaction object to obtain the access control result.

[0061] Optionally, after obtaining access control permissions for the target interaction object, the terminal performs access control management on the target interaction object based on these permissions, and obtains the access control management result. The access control management result represents whether the access control for the target interaction object is unlocked and / or whether the target interaction object triggers an abnormal event.

[0062] That is, after the terminal obtains the access control interaction permission of the target interaction object, it unlocks the access control according to the access control interaction permission and performs status detection on the target smart door lock, and triggers the corresponding abnormal alarm message when the target interaction object has an abnormal interaction result.

[0063] In this embodiment, the data processing device determines the regional environment type of the target access control area of ​​the target smart lock by responding to a data processing request for the target smart lock; based on the regional environment type, it controls the multimodal acquisition module of the target smart lock to collect multimodal biometric data of the target interactive object in the target access control area; based on the multimodal biometric data, it performs object recognition on the target interactive object to obtain the access control interaction permission of the target interactive object; based on the access control interaction permission, it performs access control management on the target interactive object to obtain the access control result. This system enables the identification of the target access control area's environmental type during smart lock access control. Based on this environmental type, the operating mode of the corresponding multimodal acquisition module of the smart lock is adjusted. This allows the multimodal acquisition module to accurately and effectively collect multimodal biometric data of the target interaction object within that environmental type, improving the accuracy of subsequent access control. The system then uses the collected multimodal biometric data to identify the target interaction object, obtaining its identity information and associated access control permissions. These permissions are then used for access control management, improving the data acquisition quality of the smart lock and enhancing the security and accuracy of access control through multimodal object identification.

[0064] like Figure 3 As shown, Figure 3 This is a flowchart illustrating one embodiment of the data processing method for identifying regional environment types provided in this application. Figure 3 In the illustrated embodiment, the data processing method further includes steps 301 to 303: 301. When the ambient light intensity data is less than the preset ambient light intensity threshold, the area environment type of the target access control area is determined to be a low light environment type; 302. When the image grayscale distribution parameters meet the preset backlighting conditions, the regional environment type of the target access control area is determined to be a backlighting environment type; 303. When the proportion of moving pixels exceeds a preset pixel proportion threshold, the area environment type of the target access control area is determined to be a dynamic environment type.

[0065] Based on the above embodiments, in this embodiment, when the terminal receives a data processing request, it also determines the regional environment type of the target access control area corresponding to the target smart door lock. Then, in subsequent steps, the operating parameters of the multimodal acquisition module are adjusted accordingly based on the regional environment type to improve the effectiveness of the acquired data. That is, the terminal detects the regional environment information of the target access control area of ​​the target smart door lock and identifies the regional environment type of the target access control area based on this information. The regional environment information includes any one or more of the following: ambient light intensity data, image grayscale distribution parameters, and moving pixel ratio data.

[0066] Optionally, ambient light intensity data refers to parameters characterizing the illumination intensity in the target access control area. Image grayscale distribution parameters refer to parameters characterizing the image brightness distribution in the image corresponding to the target access control area acquired by the multimodal acquisition module. Moving pixel proportion data refers to parameters characterizing the proportion of moving pixels in the image corresponding to the target access control area. This moving pixel proportion data can be calculated using the inter-frame differencing method.

[0067] Optionally, after acquiring any one or more of the ambient light intensity data, image grayscale distribution parameters, and motion pixel ratio data, the terminal can identify the regional environment type of the target access control area using the ambient light intensity data, image grayscale distribution parameters, and motion pixel ratio data.

[0068] Optionally, the terminal determines whether the target access control area is a low-light environment type based on ambient light intensity data. That is, when the ambient light intensity data is less than a preset ambient light intensity threshold, the area environment type of the target access control area is determined to be a low-light environment type. The preset ambient light intensity threshold is a light intensity threshold used to assess the illumination intensity of the target access control area. In a specific embodiment, the ambient light intensity threshold is 50 lux.

[0069] Optionally, the terminal also determines whether the target access control area is a backlit environment type based on image grayscale distribution parameters. Specifically, when the image grayscale distribution parameters meet a preset backlit condition, the terminal determines the target access control area to be a backlit environment type. This preset backlit condition is that a first distribution parameter in the image grayscale distribution parameters is greater than a first distribution threshold and a second distribution parameter is greater than a second distribution threshold. The first distribution parameter is the proportion of bright areas in the image grayscale distribution parameters. The second distribution parameter is the proportion of dark areas in the image grayscale distribution parameters. The first distribution threshold is 60%, and the second distribution threshold is 20%. That is, when the proportion of bright areas in the image grayscale distribution parameters is >60% and the proportion of dark areas is <20%, the terminal determines the target access control area to be a backlit environment type.

[0070] Optionally, the terminal also determines whether the target access control area is a dynamic environment type based on the proportion of moving pixels. Specifically, when the proportion of moving pixels exceeds a preset pixel proportion threshold, the terminal determines that the environment type of the target access control area is a dynamic environment type. The preset pixel proportion threshold is a threshold for assessing the presence of dynamic objects in the target access control area. In other words, when the proportion of moving pixels exceeds the preset pixel proportion threshold, the terminal determines that a dynamic object exists in the target access control area and classifies the environment type of the target access control area as a dynamic environment type.

[0071] In this embodiment, the terminal acquires the regional environmental information of the target access control area and identifies the regional environmental type corresponding to the target access control area through the regional environmental information. Then, in subsequent steps, the operating parameters of the multimodal acquisition module are adjusted accordingly based on the regional environmental type to improve the effectiveness of the acquired data.

[0072] like Figure 4 As shown, Figure 4 This is a flowchart illustrating an embodiment of access control in the data processing method provided in this application. Specifically, as shown in the example... Figure 4 As shown, in this embodiment, the data processing method further includes steps 401 to 403: 401. Based on the access control interaction permissions, perform permission verification and / or unlocking operations on the target interaction object to obtain the access control interaction result; 402. If the access control interaction result and / or the lock status of the target smart door lock meet the abnormal access control conditions, then the access control interaction result is determined to be an abnormal interaction result. 403. Output the abnormal alarm message corresponding to the abnormal interaction result based on the preset alarm strategy to obtain the access control result.

[0073] Based on the above embodiments, in this embodiment, after the terminal obtains the access control interaction permission of the target interaction object, it performs access control management on the target interaction object based on the access control interaction permission and obtains the access control management result.

[0074] Optionally, the terminal performs permission verification and / or unlocking operations on the target interaction object based on the access control interaction permission, and obtains the access control interaction result. The access control interaction result represents the interaction result between the target interaction object and the target smart lock during permission verification and unlocking operations.

[0075] Optionally, after obtaining the access control interaction result, the terminal also checks the lock status of the target smart lock and determines whether the access control interaction result is an abnormal interaction result based on the access control interaction result and the lock status. That is, if the terminal determines that the access control interaction result and / or the lock status of the target smart lock meet the abnormal access control conditions, then the access control interaction result is determined to be an abnormal interaction result. The abnormal access control conditions are the judgment conditions used to detect whether the access control interaction result is an abnormal interaction result. In a specific embodiment, the abnormal access control conditions include a first abnormal access control condition, a second abnormal access control condition, and a third abnormal access control condition. The first abnormal access control condition is that the number of access control recognitions within a preset interaction window period in the access control interaction result is greater than a preset recognition threshold (e.g., 3 times). The second abnormal access control condition is that the interaction event of the access control interaction result is triggered within a non-permitted time period or an uncommon time period. The third abnormal access control condition is that the lock status of the target smart lock shows that the multimodal acquisition module is obstructed and / or the communication link is interrupted.

[0076] That is, the terminal determines that the access control interaction result is an abnormal interaction result when it determines that the number of access control recognitions within a preset interaction window period is greater than a preset recognition threshold, the interaction event of the access control interaction result is triggered within a non-permitted time period or an unused time period (i.e., the access control has not been unlocked during the same period in history), and the door lock status of the target smart door lock shows that the multimodal acquisition module is blocked and / or the communication link is interrupted.

[0077] Optionally, the terminal outputs an abnormal alarm message corresponding to the abnormal interaction result based on a preset alarm policy, thereby obtaining the access control result. That is, the terminal sets different preset alarm policies based on different abnormal access control conditions. After determining that the access control interaction result is an abnormal alarm result through the abnormal access control conditions, it outputs the abnormal alarm message associated with the preset alarm policy corresponding to the abnormal access control condition.

[0078] Optionally, in one specific embodiment, the preset alarm strategy corresponding to the first abnormal access control condition is a buzzer alarm and a flashing light. The preset alarm strategy corresponding to the second abnormal access control condition is to push abnormal notification information to the administrator device bound to the target smart lock. The third abnormal access control condition is to use an auxiliary acquisition module to record abnormal behavior video and push abnormal messages to the associated property security system.

[0079] In this embodiment, the terminal performs permission verification and / or unlocking operations on the target interaction object based on the access control interaction permissions to obtain the access control interaction result. If the access control interaction result and / or the lock status of the target smart lock meet the abnormal access control conditions, the access control interaction result is determined to be an abnormal interaction result. Based on a preset alarm strategy, an abnormal alarm message corresponding to the abnormal interaction result is output to obtain the access control result. This achieves multi-level identification of abnormal behavior in the access control interaction process and multi-level abnormal alarms, improving the security and timeliness of the smart lock interaction process.

[0080] To better implement the data processing method in the embodiments of this application, based on the data processing method, the embodiments of this application also provide a data processing apparatus, such as... Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of one embodiment of the data processing apparatus provided in this application. Specifically, the data processing apparatus 500 includes: The environment detection module 501 is configured to respond to a data processing request for the target smart door lock and determine the regional environment type of the target access control area of ​​the target smart door lock; The data acquisition module 502 is configured to control the multimodal acquisition module of the target smart door lock to acquire multimodal biological data of the target interactive object in the target access control area based on the regional environment type; The object recognition module 503 is configured to perform object recognition on the target interactive object based on the multimodal biological data, and obtain the access control interaction permission of the target interactive object; Access control module 504 is configured to perform access control management on the target interaction object based on the access control interaction permissions, and obtain access control results.

[0081] In one possible implementation of this embodiment, the data processing device determines the regional environment type of the target access control area of ​​the target smart door lock, including: Collect the regional environmental information of the target access control area of ​​the target smart door lock; The regional environment type of the target access control area is identified based on the regional environmental information.

[0082] In one possible implementation of this embodiment, the regional environmental information in the data processing device includes any one or more of the following: ambient light intensity data, image grayscale distribution parameters, and moving pixel percentage data. The data processing device identifies the regional environment type of the target access control area based on the regional environment information, including: When the ambient light intensity data is less than a preset ambient light intensity threshold, the area environment type of the target access control area is determined to be a low light environment type; When the image grayscale distribution parameters meet the preset backlighting conditions, the regional environment type of the target access control area is determined to be a backlighting environment type. The preset backlighting conditions are that the first distribution parameter in the image grayscale distribution parameters is greater than the first distribution threshold and the second distribution parameter is greater than the second threshold. When the proportion of moving pixels exceeds a preset pixel proportion threshold, the area environment type of the target access control area is determined to be a dynamic environment type.

[0083] In one possible implementation of this embodiment, the data processing device controls the multimodal acquisition module of the target smart door lock to collect multimodal biometric data of the target interactive object in the target access control area based on the regional environment type, including: Adjust the acquisition settings parameters of the multimodal acquisition module of the target smart door lock based on the regional environment type; The multimodal acquisition module is driven to acquire multimodal biological data of the target interactive object in the target access control area according to the acquisition settings parameters.

[0084] In one possible implementation of this embodiment, the data processing device drives the multimodal acquisition module to acquire multimodal biometric data of the target interactive object in the target access control area according to the acquisition setting parameters, including: The first image of the target interactive object is acquired according to the first image acquisition module in the multimodal acquisition module and the acquisition setting parameters to obtain the first object image data; The second image of the target interactive object is acquired by the second image acquisition module in the multimodal acquisition module and the acquisition setting parameters to obtain the second object image data; The audio acquisition module in the multimodal acquisition module is used to acquire the voice interaction commands output by the target interactive object to obtain the object's audio data; The first object image data, the second object image data, and the object audio data are identified as the multimodal biological data of the target interactive object.

[0085] In one possible implementation of this embodiment, the data processing device performs object recognition on the target interaction object based on the multimodal biological data to obtain the access control interaction permissions of the target interaction object, including: Feature extraction is performed based on the multimodal biological data and feature fusion model to obtain multimodal biological features; Based on the multimodal biometrics and preset member features, identity matching is performed to determine the object identity information of the target interaction object; Access a preset database to determine the access control interaction permissions associated with the object's identity information in the preset database.

[0086] In one possible implementation of this embodiment, the data processing device performs access control management on the target interaction object based on the access control interaction permission to obtain the access control result, including: Based on the access control interaction permissions, the target interaction object is verified and / or unlocked to obtain the access control interaction result. If the access control interaction result and / or the lock status of the target smart lock meet the abnormal access control conditions, then the access control interaction result is determined to be an abnormal interaction result. Based on the preset alarm strategy, the abnormal alarm message corresponding to the abnormal interaction result is output to obtain the access control result.

[0087] In this embodiment, the data processing device determines the regional environment type of the target access control area of ​​the target smart lock by responding to a data processing request for the target smart lock; based on the regional environment type, it controls the multimodal acquisition module of the target smart lock to collect multimodal biometric data of the target interactive object in the target access control area; based on the multimodal biometric data, it performs object recognition on the target interactive object to obtain the access control interaction permission of the target interactive object; based on the access control interaction permission, it performs access control management on the target interactive object to obtain the access control result. This system enables the identification of the target access control area's environmental type during smart lock access control. Based on this environmental type, the operating mode of the corresponding multimodal acquisition module of the smart lock is adjusted. This allows the multimodal acquisition module to accurately and effectively collect multimodal biometric data of the target interaction object within that environmental type, improving the accuracy of subsequent access control. The system then uses the collected multimodal biometric data to identify the target interaction object, obtaining its identity information and associated access control permissions. These permissions are then used for access control management, improving the data acquisition quality of the smart lock and enhancing the security and accuracy of access control through multimodal object identification.

[0088] This invention also provides a data processing device, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of one embodiment of the data processing device provided in this application.

[0089] The data processing device integrates any of the data processing apparatuses provided in the embodiments of the present invention, and the data processing device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor of the steps in the data processing method described in any of the embodiments of the above data processing method.

[0090] Specifically, the data processing device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 6 The data processing device structure shown does not constitute a limitation on the data processing device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 601 is the control center of the data processing device. It connects various parts of the data processing device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, it performs various functions of the data processing device and processes data, thereby providing overall monitoring of the data processing device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.

[0091] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the data processing device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0092] The data processing device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0093] The data processing device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0094] Although not shown, the data processing device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the data processing device loads the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 601 runs the applications stored in the memory 602 to realize various functions, as follows: In response to a data processing request for the target smart lock, determine the regional environment type of the target access control area of ​​the target smart lock; Based on the regional environment type, the multimodal acquisition module of the target smart door lock is controlled to collect multimodal biological data of the target interactive object in the target access control area; Based on the multimodal biological data, object recognition is performed on the target interaction object to obtain the access control interaction permissions of the target interaction object; Access control is performed on the target interactive object based on the access control interaction permissions to obtain the access control result.

[0095] Therefore, embodiments of the present invention provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, which is loaded by a processor to execute the steps of any of the data processing methods provided in the embodiments of the present invention. For example, the computer program loaded by the processor can execute the following steps: In response to a data processing request for the target smart lock, determine the regional environment type of the target access control area of ​​the target smart lock; Based on the regional environment type, the multimodal acquisition module of the target smart door lock is controlled to collect multimodal biological data of the target interactive object in the target access control area; Based on the multimodal biological data, object recognition is performed on the target interaction object to obtain the access control interaction permissions of the target interaction object; Access control is performed on the target interactive object based on the access control interaction permissions to obtain the access control result.

[0096] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0097] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0098] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0099] The above provides a detailed description of a data processing method provided by the embodiments of this application. Specific embodiments have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data processing method, characterized in that, The data processing method, applied to the target smart door lock, includes: In response to a data processing request for the target smart lock, determine the regional environment type of the target access control area of ​​the target smart lock; Based on the regional environment type, the multimodal acquisition module of the target smart door lock is controlled to collect multimodal biological data of the target interactive object in the target access control area; Based on the multimodal biological data, object recognition is performed on the target interaction object to obtain the access control interaction permissions of the target interaction object; Access control is performed on the target interactive object based on the access control interaction permissions to obtain the access control result.

2. The data processing method according to claim 1, characterized in that, The determination of the regional environment type of the target access control area of ​​the target smart door lock includes: Collect the regional environmental information of the target access control area of ​​the target smart door lock; The regional environment type of the target access control area is identified based on the regional environmental information.

3. The data processing method according to claim 2, characterized in that, The regional environmental information includes any one or more of the following: ambient light intensity data, image grayscale distribution parameters, and moving pixel percentage data. The step of identifying the regional environment type of the target access control area based on the regional environment information includes: When the ambient light intensity data is less than a preset ambient light intensity threshold, the area environment type of the target access control area is determined to be a low light environment type; When the image grayscale distribution parameters meet the preset backlighting conditions, the regional environment type of the target access control area is determined to be a backlighting environment type. The preset backlighting conditions are that the first distribution parameter in the image grayscale distribution parameters is greater than the first distribution threshold and the second distribution parameter is greater than the second threshold. When the percentage of moving pixels exceeds a preset pixel percentage threshold, the area environment type of the target access control area is determined to be a dynamic environment type.

4. The data processing method according to claim 1, characterized in that, The multimodal acquisition module, which controls the target smart door lock based on the regional environment type, acquires multimodal biometric data of the target interactive object in the target access control area, including: Adjust the acquisition settings parameters of the multimodal acquisition module of the target smart door lock based on the regional environment type; The multimodal acquisition module is driven to acquire multimodal biological data of the target interactive object in the target access control area according to the acquisition settings parameters.

5. The data processing method according to claim 4, characterized in that, The step of driving the multimodal acquisition module to acquire multimodal biometric data of the target interactive object in the target access control area according to the acquisition setting parameters includes: The first image of the target interactive object is acquired according to the first image acquisition module in the multimodal acquisition module and the acquisition setting parameters to obtain the first object image data; The second image of the target interactive object is acquired by the second image acquisition module in the multimodal acquisition module and the acquisition setting parameters to obtain the second object image data; The audio acquisition module in the multimodal acquisition module is used to acquire the voice interaction commands output by the target interactive object to obtain the object's audio data; The first object image data, the second object image data, and the object audio data are identified as the multimodal biological data of the target interactive object.

6. The data processing method according to claim 1, characterized in that, The step of performing object recognition on the target interaction object based on the multimodal biological data to obtain the access control interaction permissions of the target interaction object includes: Feature extraction is performed based on the multimodal biological data and feature fusion model to obtain multimodal biological features; Based on the multimodal biometrics and preset member features, identity matching is performed to determine the object identity information of the target interaction object; Access a preset database to determine the access control interaction permissions associated with the object's identity information in the preset database.

7. The data processing method according to claim 1, characterized in that, The step of performing access control management on the target interaction object based on the access control interaction permissions to obtain the access control result includes: Based on the access control interaction permissions, the target interaction object is verified and / or unlocked to obtain the access control interaction result. If the access control interaction result and / or the lock status of the target smart lock meet the abnormal access control conditions, then the access control interaction result is determined to be an abnormal interaction result. Based on the preset alarm strategy, the abnormal alarm message corresponding to the abnormal interaction result is output to obtain the access control result.

8. A data processing apparatus, characterized in that, The data processing device includes: The environmental detection module is configured to respond to a data processing request for the target smart lock and determine the regional environmental type of the target access control area of ​​the target smart lock; The data acquisition module is configured to control the multimodal acquisition module of the target smart door lock to acquire multimodal biological data of the target interactive object in the target access control area based on the regional environment type; The object recognition module is configured to perform object recognition on the target interactive object based on the multimodal biological data, and obtain the access control interaction permissions of the target interactive object; The access control module is configured to perform access control management on the target interaction object based on the access control interaction permissions, and obtain the access control result.

9. A data processing device, characterized in that, The data processing device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps of the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the steps of the data processing method according to any one of claims 1 to 7.

Images