Dynamic control system and method based on multi-source data fusion

Abstract

The present invention relates to the technical field of public rental housing management. The present invention discloses a dynamic control system and method based on multi-source data fusion, including determining whether the current object is a tenant in the public rental housing according to the image to be recognized. If the current object is a tenant in the public rental housing, determining password-related information according to the image to be recognized, obtaining the first response information of the current object, judging whether to generate a face verification sequence according to the first response information. If so, performing face verification on the current object based on the face verification sequence. If the current object is not a tenant in the public rental housing, generating verification requirement information according to the intention information, and performing intention verification on the current object according to the second response information. By dynamically judging whether the object is a tenant and respectively adopting password verification and intention verification methods for tenants and non-tenants, the problem that the existing system cannot flexibly identify different identities is solved, and the flexibility and accuracy of the management system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of public rental housing management. More specifically, the present invention relates to a dynamic control system and method based on multi-source data fusion. Background Art

[0002] For example, the Chinese patent application with the publication number CN117273873A provides a public rental housing management system and method based on a face recognition intelligent lock. The system realizes the registration of housing, personnel, and equipment information through a registration module, the management module authorizes the management of housing and equipment, the public module supports online communication between renters and managers, the monitoring module monitors equipment operations in real time, and the database is used to store and process system information; the Chinese patent application with the publication number CN110310179A provides a method for managing public rental housing tenants based on smart contracts. Data such as the entry and exit, electricity meters, and water meters of public rental housing users are collected through sensors and uploaded to the blockchain, and the smart contract analyzes the data. According to the analysis results, the system automatically reports the situation of long-term non-occupancy or non-owner use of public rental housing to prevent abuse.

[0003] Although the prior art has managed the entry and use of public rental housing through intelligent systems, most public rental housing management systems still rely on single face recognition or only process data through smart contracts, lacking real-time response and dynamic adjustment to the actual behaviors and intentions of users. For example, when a technical worker needs to enter a public rental housing for maintenance, since they are not tenants, the system may not be able to effectively identify their identities, resulting in the technical worker being unable to enter smoothly. This situation limits the flexibility and accuracy of the management system.

[0004] In view of this, the present invention proposes a dynamic control system and method based on multi-source data fusion to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a dynamic control system and method based on multi-source data fusion.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] In the first aspect, a dynamic control method based on multi-source data fusion is provided, including:

[0008] S10: Obtain the image to be recognized and the voice to be recognized of the current object, and judge whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, transfer to S20; if not, transfer to S30;

[0009] S20: Determine password-related information based on the image to be recognized, send the password-related information to the current object, obtain the first response information of the current object, determine whether to generate a face verification sequence according to the first response information, if so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification passes, open the smart door lock; when the verification fails, close the smart door lock.

[0010] S30: Determine the intention information of the current object according to the image to be recognized and the voice to be recognized, generate verification requirement information according to the intention information, send the verification requirement information to the current object, and obtain the second response information of the current object. Perform intention verification on the current object according to the second response information. When the verification passes, open the smart door lock; when the verification fails, close the smart door lock.

[0011] Furthermore, the method for determining whether the current object is a tenant in the public rental housing according to the image to be recognized includes:

[0012] Obtain the light distribution information of the image to be recognized, determine the corresponding light compensation template according to the light distribution information, construct a target recognition image according to the image to be recognized and the light compensation template, input the target recognition image into the pre-constructed object classification model, obtain the object classification result. If the object classification result is a tenant, then the current object is a tenant in the public rental housing; if the object classification result is a non-tenant, then the current object is not a tenant in the public rental housing.

[0013] Furthermore, the method for obtaining the light distribution information of the image to be recognized includes:

[0014] Obtain the pixel gradient value of the image to be recognized, construct a distribution contour map corresponding to the image to be recognized according to the pixel gradient value, divide the distribution contour map into a highlight area, a shadow area, and a uniform illumination area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniform illumination area as the light distribution information.

[0015] Furthermore, the method for determining the corresponding light compensation template according to the light distribution information includes:

[0016] Sequentially determine the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area according to the light distribution information, and splice the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area to obtain the light compensation template.

[0017] Furthermore, the construction method of the object classification model includes:

[0018] Obtain training data, divide the training data into a training set and a test set, use the historical target recognition images in the training set as the input data of the initial support vector machine, use the historical object classification results in the training set as the output labels of the initial support vector machine, determine the optimal separating hyperplane of the initial support vector machine through the Lagrange multiplier method, map the optimal separating hyperplane from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function, use the test set to verify the performance of the initial support vector machine, and when the classification accuracy meets the preset requirements, train an object classification model.

[0019] Further, the first response information includes user action images. The method for determining whether to generate a face verification sequence based on the first response information includes:

[0020] Extract Y human body key points from the user action images based on the OpenPose algorithm, construct a human action sequence according to the Y human body key points, determine whether the human action sequence is similar to a pre-constructed standard action sequence. If it is similar, generate a face verification sequence; if it is not similar, do not generate a face verification sequence. The standard action sequence represents the standard behavioral action sequence of normally viewing the password and attempting to unlock after receiving password-related information.

[0021] Further, the method for determining whether the human action sequence is similar to the pre-constructed standard action sequence includes:

[0022] Calculate the similarity between the human action sequence and the standard action sequence. When the similarity is greater than the preset similarity threshold, it is similar; when the similarity is less than or equal to the preset similarity threshold, it is not similar.

[0023] Further, the method for generating a face verification sequence includes:

[0024] Extract features from a preset face image set based on the Histogram of Oriented Gradients (HOG) algorithm to generate a face image feature sequence, randomly fuse the face image feature sequence with the unlock password in the password-related information to generate a face verification sequence. The face image set is the face image set uploaded by the tenant.

[0025] Further, the second response information includes intention response images. The method for performing intention verification on the current object based on the second response information includes:

[0026] Extract corresponding intention response features from the intention response images in the second response information, compare the intention response features with the standard response features in the verification requirement information. When the intention response features match the standard response features, the verification passes; when the intention response features do not match the standard response features, the verification fails.

[0027] Second aspect, a dynamic control system based on multi-source data fusion is provided, which is used to implement the above-mentioned dynamic control method based on multi-source data fusion, including:

[0028] Object recognition module: used to obtain the image and voice to be recognized of the current object, and judge whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, transfer to the first verification module; if not, transfer to the second verification module;

[0029] First verification module: used to determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first response information of the current object, judge whether to generate a face verification sequence according to the first response information. If so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification is passed, open the intelligent door lock; when the verification fails, close the intelligent door lock;

[0030] Second verification module: used to determine the intention information of the current object according to the image to be recognized and the voice to be recognized, generate verification requirement information according to the intention information, send the verification requirement information to the current object, and obtain the second response information of the current object. Perform intention verification on the current object according to the second response information. When the verification is passed, open the intelligent door lock; when the verification fails, close the intelligent door lock.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] The present invention first judges whether the current object is a tenant in the public rental housing according to the image to be recognized. If the current object is a tenant in the public rental housing, determine the password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first response information of the current object, and judge whether to generate a face verification sequence according to the first response information. If so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. If the current object is not a tenant in the public rental housing, generate verification requirement information according to the intention information, send the verification requirement information to the current object, and obtain the second response information of the current object. Perform intention verification on the current object according to the second response information; the present invention dynamically judges whether the object is a tenant, and respectively adopts password verification and intention verification methods for tenants and non-tenants, solves the problem that the existing system cannot flexibly identify different identities, improves the flexibility and accuracy of the management system, and significantly optimizes the user experience. Brief Description of the Drawings

[0033] Figure 1 It is a schematic flow chart of the dynamic control method based on multi-source data fusion in the present invention;

[0034] Figure 2It is a schematic structural diagram of the dynamic management and control system based on multi-source data fusion in the present invention;

[0035] Figure 3 It is a schematic flow diagram of the method for judging whether the current object is a tenant in public rental housing according to the image to be recognized in the present invention;

[0036] Figure 4 It is a schematic flow diagram of the method for generating a face verification sequence in the present invention. Specific embodiments

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] Embodiment 1

[0039] Please refer to Figure 1 As shown, the present embodiment discloses and provides a dynamic management and control method based on multi-source data fusion, including:

[0040] S10: Obtain the image to be recognized and the voice to be recognized of the current object, and judge whether the current object is a tenant in public rental housing according to the image to be recognized. If so, go to S20; if not, go to S30;

[0041] In this embodiment, the current object can be a tenant in the public rental housing, or a staff member who needs to enter the public rental housing to check the gas data, or a technical worker who needs to enter the public rental housing for maintenance. This embodiment does not limit this. The image to be recognized can be a face image of the current object or an upper body image of the current object. The voice to be recognized can be an intention voice issued by the current object. The intention voice expresses whether there is a tenant in the current public rental housing or the current intention. Taking the above technical worker as an example, the current intention can clearly express the purpose of his entering the public rental housing, for example: "I am here for maintenance work", or "I am here to check the gas equipment".

[0042] It should be added that the above image to be recognized or the voice to be recognized can be collected through the camera and sensor inside the intelligent door lock, or through the connected external devices, such as the monitoring camera and sensor outside the door. This embodiment does not limit this.

[0043] As Figure 3 shown, the method for judging whether the current object is a tenant in public rental housing according to the image to be recognized includes:

[0044] Obtain the light distribution information of the image to be recognized, determine the corresponding light compensation template according to the light distribution information, construct the target recognition image based on the image to be recognized and the light compensation template, input the target recognition image into the pre-constructed object classification model, and obtain the object classification result. If the object classification result is a tenant, then the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, then the current object is not a tenant in the public rental housing.

[0045] It should be noted that when signing a lease contract with the rental platform, tenants in public rental housing also need to upload their own avatars to the platform. It can be understood that, usually, the background light of the tenant avatar is relatively uniform and stable. This is because tenants usually upload their avatars in an indoor environment where the light conditions are relatively fixed, avoiding strong light contrast or complex background interference. The image to be recognized is collected in the environment where the intelligent door lock is installed, and this environment is usually outdoors, resulting in the possible uneven light distribution of the image to be recognized. For example, there may be strong light, shadows, or insufficient light. Therefore, in order to improve the recognition accuracy, it is necessary to optimize the image to be recognized through the light compensation template to construct a target recognition image with higher quality, so as to adapt to different environmental light conditions.

[0046] The methods for obtaining the light distribution information of the image to be recognized include:

[0047] Obtain the pixel gradient value of the image to be recognized, construct a distribution contour map corresponding to the image to be recognized according to the pixel gradient value, divide the distribution contour map into a highlight area, a shadow area, and a uniform illumination area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniform illumination area as the light distribution information.

[0048] It is not difficult to understand that the pixel gradient value refers to the degree of brightness change between adjacent pixels in the image to be recognized, which reflects the change trend of light intensity in the image. The distribution contour map is a graphical representation of the visual image brightness gradient distribution generated based on the change trend of pixel gradient values in the image to be recognized. It draws contour lines (similar to contour lines in topographic maps) to identify pixel regions with the same or similar brightness gradient values in the image with closed curves. The distribution contour map can be drawn through the contour function in Matlab. In the above content, dividing the distribution contour map into a highlight area, a shadow area, and a uniform illumination area can be achieved through a preset gradient threshold, and this embodiment will not elaborate on this in too much detail.

[0049] The methods for determining the corresponding light compensation template according to the light distribution information include:

[0050] Determine the sub-compensation template corresponding to the high-light area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area according to the light distribution information in sequence, and splice the sub-compensation template corresponding to the high-light area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area to obtain the light compensation template.

[0051] In this embodiment, multiple sub-compensation templates can be pre-stored in the database. The role of the sub-compensation template is to provide corresponding light adjustment strategies for different light distribution areas (high-light area, shadow area, and uniform illumination area). For example, for the high-light area, the sub-compensation template can reduce the brightness of the high-light area, reduce overexposure, and make the detail information in the image clearer. It can be understood that the area of the high-light area is determined according to the position coordinate information of the high-light area, and the sub-compensation template is adjusted to match the position and area of the high-light area to ensure that the effect of light compensation can accurately cover the high-light area.

[0052] The method for constructing an object classification model includes:

[0053] Obtain training data, divide the training data into a training set and a test set, use the historical target recognition images in the training set as the input data of the initial vector machine, use the historical object classification results in the training set as the output labels of the initial vector machine, determine the optimal separation hyperplane of the initial vector machine by the Lagrange multiplier method, map the optimal separation hyperplane from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function, use the test set to verify the performance of the initial vector machine, and when the classification accuracy meets the preset requirements, train to obtain the object classification model.

[0054] It should be noted that the initial vector machine is a classifier constructed based on the support vector machine (SVM) algorithm. It inputs the features (historical target recognition images) and corresponding classification labels (historical object classification results) in the training data, and trains a model that can distinguish different categories through an optimization algorithm (such as the Lagrange multiplier method). The core of the initial vector machine is to find a hyperplane that separates different categories and expand its ability through the kernel function to handle complex non-linear classification problems.

[0055] In this embodiment, the optimal separating hyperplane is the core part of the support vector machine. The optimal separating hyperplane is a linear plane that can maximize the interval between classes and is used to separate samples into different classes. In a two-dimensional space, the optimal separating hyperplane is a line, while in a three-dimensional or higher-dimensional space, it is a hyperplane. The support vector machine finds this plane through an optimization algorithm (such as the Lagrange multiplier method) to achieve optimal classification. The Gaussian kernel function is a commonly used kernel function that can implicitly map the original low-dimensional data to a high-dimensional space. In the high-dimensional space, many data that cannot be linearly separated can be linearly separated. The Gaussian kernel function is a tool to extend the SVM to nonlinear problems and can map low-dimensional data to a high-dimensional space to achieve more complex classification. Determining the optimal separating hyperplane of the initial vector machine through the Lagrange multiplier method and mapping the optimal separating hyperplane from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function in the above content are prior arts, and this embodiment will not elaborate on them too much.

[0056] In this embodiment, first, the pixel gradient value is calculated and a distribution contour map is generated, which helps to accurately divide the highlight area, shadow area, and uniform illumination area, providing a basis for the subsequent selection of the light compensation template. The light distribution information is used to determine the corresponding sub-compensation template and sequentially stitch them into a complete light compensation template to ensure that different light distribution areas are subjected to targeted optimization processing, thereby generating a target recognition image with higher quality. The light optimization of the target recognition image improves the classification accuracy of the model, especially in outdoor scenes with complex environmental light conditions (such as strong light, shadows, insufficient light). At the same time, the process diversion of the object classification results ensures that the needs of different objects can be responded to in a timely manner, thereby enhancing the flexibility and efficiency of the system in practical applications.

[0057] S20: Determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first response information of the current object, and judge whether to generate a face verification sequence according to the first response information. If so, generate a face verification sequence and perform face verification on the current object based on the face verification sequence. When the verification passes, open the intelligent door lock; when the verification fails, close the intelligent door lock.

[0058] In this embodiment, the password-related information at least includes the unlocking password of the intelligent door lock and the mobile phone number of the corresponding tenant. It should be noted that usually, there are multiple tenants in the public rental housing, and each tenant corresponds to an unlocking password. When the tenant signs a lease contract with the rental platform, the rental platform will also send the corresponding unlocking password via text message. Therefore, sending the password-related information to the current object means sending the unlocking password of the intelligent door lock to the mobile phone number or WeChat account of the corresponding tenant, etc. This embodiment does not limit this.

[0059] It should be added that the first response information includes at least the user action image, which refers to the image of the response action made by the current object after sending password-related information.

[0060] The method for determining whether to generate a face verification sequence based on the first response information includes:

[0061] Based on the OpenPose algorithm, Y human key points are extracted from the user action image, and a human action sequence is constructed according to the Y human key points. It is determined whether the human action sequence is similar to a pre-constructed standard action sequence. If it is similar, a face verification sequence is generated; if it is not similar, a face verification sequence is not generated. The standard action sequence represents the standard behavioral action sequence of normally viewing the password and attempting to unlock after receiving password-related information.

[0062] It should be noted that the OpenPose algorithm is a human pose estimation algorithm based on deep learning. It can extract the key points of the human body (such as the head, arms, wrists, knees, etc.) from images or videos and construct a human skeleton model. In the above, the user action image can be several adjacent action images, and the human key points include elbow key points, shoulder key points, wrist key points, and head key points, etc. In the above, the human action sequence refers to a time series graph generated by the dynamic change of human key points. The human action sequence can be constructed by existing Matlab software or Unity software, and this embodiment will not elaborate on this too much.

[0063] The method for determining whether the human action sequence is similar to the pre-constructed standard action sequence includes:

[0064] Calculate the similarity between the human action sequence and the standard action sequence. When the similarity is greater than the preset similarity threshold, it is similar; when the similarity is less than or equal to the preset similarity threshold, it is not similar.

[0065] In this embodiment, the similarity can be achieved by calculating the Euclidean distance or the cosine value.

[0066] As Figure 4 shown, the method for generating a face verification sequence includes:

[0067] Based on the Histogram of Oriented Gradients (HOG) algorithm, feature extraction is performed on a preset face image set to generate a face image feature sequence. The face image feature sequence is randomly fused with the unlock password in the password-related information to generate a face verification sequence. The face image set is the face image set uploaded by the tenant.

[0068] It should be noted that when signing a lease contract with a rental platform, tenants in public rental housing also need to upload their own avatars to the platform. Therefore, a set of face images uploaded by tenants will be stored in the database. The Histogram of Oriented Gradients (HOG) algorithm is an image feature extraction algorithm that calculates the gradient direction distribution features in local regions of an image. A face image feature sequence is composed of several image feature vectors combined in sequence. Similarly, an unlocking password can be converted into a set of password feature vectors. Random fusion generally refers to randomly combining image feature vectors and password feature vectors to form new vectors.

[0069] Exemplarily, the face image feature sequence includes features of turning the head to the left, blinking, nodding, turning the head to the right, and opening the mouth. If the unlocking password is 24587, then the password feature vectors include the feature of the number 2, the feature of the number 4, the feature of the number 5, the feature of the number 8, and the feature of the number 7. The above-mentioned random fusion of the face image feature sequence and the unlocking password in the password-related information can be the combination of the feature of the number 2 and the feature of turning the head to the left, the combination of the feature of the number 4 and the feature of opening the mouth, the combination of the feature of the number 5 and the feature of blinking, the combination of the feature of the number 8 and the feature of nodding, and the combination of the feature of the number 7 and the feature of turning the head to the right. Therefore, the fused face verification sequence is: turning the head to the left, opening the mouth, blinking, nodding, turning the head to the right.

[0070] It can be understood that when signing a lease contract with a rental platform, a tenant in public rental housing can upload their own face image, and the rental platform can also collect the tenant's face image. Features such as turning the head, blinking, and nodding can be obtained from the collected face image. This embodiment does not limit this.

[0071] It should be added that the reason why this embodiment uses a face verification sequence for verification instead of methods such as fingerprints is that the flow of people in public rental housing is intensive. Before using fingerprint unlocking, fingerprint collection and entry are required, and this method requires additional equipment support and high labor costs. Moreover, when personnel are frequently replaced, the update and management of fingerprint information are quite inconvenient. Therefore, this embodiment realizes verification through a face verification sequence, which can directly utilize the face image data uploaded by users or the face image data collected by the platform, avoiding both equipment dependence and complex operations, and adapting to the actual situation of frequent personnel replacement, thus improving the flexibility and efficiency of verification.

[0072] It is not difficult to understand that in this embodiment, it is judged whether the human body action sequence is similar to a pre-constructed standard action sequence. If it is similar, it indicates that the tenant has received the unlocking password through the mobile phone, so subsequent face verification is not required. When it is not similar, it indicates that the tenant may not be carrying a mobile phone or other situations. Therefore, to further ensure security, subsequent verification is required.

[0073] In this embodiment, the effect of randomly fusing the face image feature sequence with the unlocking password in the password-related information is that the generated face verification sequence is unique for each tenant, directly binding the tenant's identity information. During the verification process, the unlocking password directly participates in the fusion, avoiding the storage and management costs brought by generating additional passwords. After integrating the unlocking password into the verification sequence, the verification process depends on the correctness of the unlocking password, and at the same time combines the correctness of the face feature actions. The dual verification greatly improves the security.

[0074] S30: Determine the intention information of the current object according to the image to be recognized and the voice to be recognized, generate verification requirement information according to the intention information, send the verification requirement information to the current object, and obtain the second response information of the current object. Verify the intention of the current object according to the second response information. If the verification passes, open the intelligent door lock. If the verification fails, close the intelligent door lock.

[0075] In this embodiment, the intention information may be text information expressing the reason for entering the public rental housing. Exemplarily, taking a technical worker as an example, the voice to be recognized may be "I am here to check the gas equipment", and it is recognized from the image to be recognized which company the technical worker belongs to or the institution to which the work uniform identification worn belongs. Then the intention information may be "Technical worker, from a certain gas company, responsible for checking gas equipment".

[0076] The method for generating verification requirement information according to the intention information includes:

[0077] Input the intention information into a pre-constructed requirement generation model to obtain the verification requirement information.

[0078] The construction of the requirement generation model can be extended and customized based on existing large language models or rule-based generation engines, and is used to generate verification requirement information for specific intention information. Existing large language models can be BERT, ChatGLM series or Spark large models. Take the historical intention information as the input of the large language model and the historical verification requirement information as the output of the large language model, so as to fine-tune or train the large language model.

[0079] In this embodiment, the verification requirement information represents the specific requirements for verifying the authenticity of the identity and the legality of the behavior generated for the intention information of the current object. For example, if the intention information is "Technical worker, from a certain gas company, responsible for checking gas equipment", the verification requirement information is "Please show the work permit of the gas company".

[0080] In the above, the second response information at least includes an intention response image. The intention response image refers to an image of the response behavior made by the current object to complete the verification requirement after receiving the verification requirement information. The method for verifying the intention of the current object according to the second response information includes:

[0081] Extract corresponding intended response features from the intended response image in the second response information, and compare the intended response features with the standard response features in the verification requirement information. When the intended response features match the standard response features, the verification passes; when the intended response features do not match the standard response features, the verification fails.

[0082] Exemplarily, if the verification requirement information is "Please show the work permit of the gas company", then the intended response image is a picture of the current object holding the work permit and showing it to the camera. The intended response features include the extracted text information (such as "a certain gas company", "employee name", etc.) and the posture of holding the work permit. It can be understood that when the intended response features match the standard response features, it indicates that the identity authenticity and behavior legality of the current object have been verified. Therefore, the intelligent door lock can be opened, which can not only ensure security but also avoid excessive verification steps or additional manual intervention.

[0083] In this embodiment, first, determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If the current object is a tenant in the public rental housing, determine the password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first response information of the current object, and determine whether to generate a face verification sequence according to the first response information. If so, generate a face verification sequence and perform face verification on the current object based on the face verification sequence. If the current object is not a tenant in the public rental housing, generate verification requirement information according to the intention information, send the verification requirement information to the current object, obtain the second response information of the current object, and perform intention verification on the current object according to the second response information. In this embodiment, by dynamically judging whether the object is a tenant and adopting password verification and intention verification methods for tenants and non-tenants respectively, the problem that the existing system cannot flexibly identify different identities is solved, the flexibility and accuracy of the management system are improved, and the user experience is also significantly optimized.

[0084] Embodiment 2

[0085] Please refer to Figure 2 As shown, based on the same inventive concept, this embodiment discloses and provides a dynamic control system based on multi-source data fusion. For the details not described in this embodiment, please refer to the relevant parts in Embodiment 1. The system includes:

[0086] Object recognition module: used to obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, transfer to the first verification module; if not, transfer to the second verification module;

[0087] The method for determining whether the current object is a tenant in the public rental housing according to the image to be recognized includes:

[0088] Obtain the light distribution information of the image to be recognized, determine the corresponding light compensation template according to the light distribution information, construct a target recognition image based on the image to be recognized and the light compensation template, input the target recognition image into a pre-constructed object classification model, obtain the object classification result. If the object classification result is a tenant, then the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, then the current object is not a tenant in the public rental housing.

[0089] The method for obtaining the light distribution information of the image to be recognized includes:

[0090] Obtain the pixel gradient value of the image to be recognized, construct a distribution contour map corresponding to the image to be recognized according to the pixel gradient value, divide the distribution contour map into a highlight area, a shadow area and a uniform illumination area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area and the position coordinate information of the uniform illumination area as the light distribution information.

[0091] The method for determining the corresponding light compensation template according to the light distribution information includes:

[0092] Sequentially determine the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area and the sub-compensation template corresponding to the uniform illumination area according to the light distribution information, and splice the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area and the sub-compensation template corresponding to the uniform illumination area to obtain the light compensation template.

[0093] The first verification module: used to determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first response information of the current object, judge whether to generate a face verification sequence according to the first response information. If so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification passes, open the intelligent door lock. When the verification fails, close the intelligent door lock;

[0094] It should be added that the first response information at least includes the user action image, and the user action image refers to the image of the response action made by the current object after receiving the password-related information.

[0095] The method for judging whether to generate a face verification sequence according to the first response information includes:

[0096] Extract Y human body key points from the user action image based on the OpenPose algorithm, construct a human body action sequence according to the Y human body key points, judge whether the human body action sequence is similar to a pre-constructed standard action sequence. If similar, generate a face verification sequence. If not similar, do not generate a face verification sequence. The standard action sequence represents the standard behavior action sequence of normally viewing the password and attempting to unlock after receiving the password-related information.

[0097] A method for determining whether a human action sequence is similar to a pre - constructed standard action sequence includes:

[0098] Calculate the similarity between the human action sequence and the standard action sequence. When the similarity is greater than a preset similarity threshold, they are similar; when the similarity is less than or equal to the preset similarity threshold, they are not similar.

[0099] In this embodiment, the similarity can be achieved by calculating the Euclidean distance or the cosine value.

[0100] A method for generating a face verification sequence includes:

[0101] Extract features from a preset face image set based on the Histogram of Oriented Gradients (HOG) algorithm to generate a face image feature sequence, and randomly fuse the face image feature sequence with the unlock password in the password - related information to generate a face verification sequence. The face image set is the face image set uploaded by the tenant.

[0102] The second verification module: used to determine the intention information of the current object according to the image to be recognized and the voice to be recognized, generate verification requirement information according to the intention information, send the verification requirement information to the current object, and obtain the second response information of the current object. Verify the intention of the current object according to the second response information. When the verification passes, open the smart door lock; when the verification fails, close the smart door lock.

[0103] A method for generating verification requirement information according to intention information includes:

[0104] Input the intention information into a pre - constructed requirement generation model to obtain the verification requirement information.

[0105] The construction of the requirement generation model can be extended and customized based on existing large - language models or rule - based generation engines, and is used to generate verification requirement information for specific intention information. Existing large - language models can be BERT, ChatGLM series, or Spark large models. Use historical intention information as the input of the large - language model and historical verification requirement information as the output of the large - language model to fine - tune or train the large - language model.

[0106] As mentioned above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0107] Finally, the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A dynamic control method based on multi-source data fusion, characterized in that: include: S10: Obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, proceed to S20; if not, proceed to S30; S20: Determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain first reaction information of the current object, the first reaction information includes a user action image, and determine whether to generate a face verification sequence according to the first reaction information: extract Y human body key points from the user action image based on the OpenPose algorithm, construct a human body action sequence according to the Y human body key points, and determine whether the human body action sequence is similar to a pre-constructed standard action sequence. If so, generate a face verification sequence; if not, do not generate a face verification sequence. The standard action sequence is characterized by a standard behavior action sequence of normally checking the password and attempting to unlock after receiving password-related information; If yes, generate a face verification sequence: extract features from a preset face image set based on a directional gradient histogram algorithm to generate a face image feature sequence, and randomly fuse the face image feature sequence with the unlock password in the password-related information to generate a face verification sequence, wherein the face image set is a face image set uploaded by the tenant; And based on the face verification sequence, the face verification is performed on the current object. When the verification is passed, the smart door lock is opened, and when the verification fails, the smart door lock is closed; S30: Determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

2. The dynamic control method based on multi-source data fusion according to claim 1 is characterized in that: The method for determining whether the current object is a tenant in a public rental housing according to the image to be identified comprises: Obtain light distribution information of the image to be identified, determine a corresponding light compensation template based on the light distribution information, construct a target recognition image based on the image to be identified and the light compensation template, input the target recognition image into a pre-trained object classification model, and obtain the object classification result. If the object classification result is a tenant, the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, the current object is not a tenant in the public rental housing.

3. The dynamic control method based on multi-source data fusion according to claim 2 is characterized in that: The method for obtaining light distribution information of an image to be identified includes: Obtain pixel gradient values ​​of the image to be identified, construct a distribution contour map corresponding to the image to be identified according to the pixel gradient values, divide the distribution contour map into a highlight area, a shadow area, and a uniformly illuminated area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniformly illuminated area as light distribution information.

4. The dynamic control method based on multi-source data fusion according to claim 3 is characterized in that: The method for determining a corresponding light compensation template according to light distribution information comprises: The sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are determined in turn according to the light distribution information, and the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are spliced ​​to obtain the light compensation template.

5. The dynamic control method based on multi-source data fusion according to claim 2 is characterized in that: The training method of the object classification model includes: The training data is obtained and divided into a training set and a test set. The historical target recognition images in the training set are used as the input data of the initial vector machine, and the historical object classification results in the training set are used as the output labels of the initial vector machine. The optimal separation hyperplane of the initial vector machine is determined by the Lagrange multiplier method. The optimal separation hyperplane is mapped from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function. The performance of the initial vector machine is verified by the test set. When the classification accuracy meets the preset requirements, the object classification model is obtained through training.

6. The dynamic control method based on multi-source data fusion according to claim 4 is characterized in that: The method for determining whether a human motion sequence is similar to a pre-built standard motion sequence comprises: The similarity between the human motion sequence and the standard motion sequence is calculated. When the similarity is greater than a preset similarity threshold, they are similar; when the similarity is less than or equal to the preset similarity threshold, they are not similar.

7. The dynamic control method based on multi-source data fusion according to claim 6 is characterized in that: The second reaction information includes an intention reaction image, and the method for verifying the intention of the current object according to the second reaction information includes: The corresponding intention reaction feature is extracted according to the intention reaction image in the second reaction information, and the intention reaction feature is compared with the standard reaction feature in the verification requirement information. When the intention reaction feature matches the standard reaction feature, the verification passes; when the intention reaction feature does not match the standard reaction feature, the verification fails.

8. A dynamic control system based on multi-source data fusion, which is used to implement the dynamic control method based on multi-source data fusion as described in any one of claims 1 to 7, characterized in that: include: Object recognition module: used to obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, it will be transferred to the first verification module, if not, it will be transferred to the second verification module; The first verification module is used to determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first reaction information of the current object, determine whether to generate a face verification sequence according to the first reaction information, and if so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed; The second verification module: used to determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

Images