Dynamic management and control system and method based on multi-source data fusion

Abstract

The invention relates to the technical field of public rental housing management, and discloses a dynamic management and control system and method based on multi-source data fusion, and the method comprises the steps: judging whether a current object is a tenant in a public rental housing or not according to a to-be-recognized image, and if the current object is the tenant in the public rental housing, determining password related information according to the to-be-recognized image, obtaining first response information of the current object, judging whether a face verification sequence is generated or not according to the first response information, if so, performing face verification on the current object based on the face verification sequence, and if not, generating verification requirement information according to the intention information, and performing intention verification on the current object according to the second reaction information, dynamically judging whether the object is the tenant or not, and respectively adopting a password verification method and an intention verification method for the tenant and the non-tenant, so that the problem that an existing system cannot flexibly identify different identities is solved, and the flexibility and the accuracy of the management system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of public rental housing management, and more specifically, to a dynamic management and control system and method based on multi-source data fusion. Background Art

[0002] For example, the Chinese patent application with publication number CN117273873A provides a public rental housing management system and method based on facial recognition smart locks. The system realizes the registration of housing, personnel and equipment information through the registration module, the management module performs management authorization of housing and equipment, the public module supports online communication between tenants and managers, the monitoring module monitors equipment operation in real time, and the database is used to store and process system information; the Chinese patent application with publication number CN110310179A provides a public rental housing tenant management method based on smart contracts. Sensors are used to collect the entry and exit, electricity meter and water meter data of public rental housing users, and the data is uploaded to the blockchain. The smart contract performs analysis, and the system automatically reports the use of public rental housing by long-term non-occupants or non-house owners based on the analysis results to prevent abuse.

[0003] Although existing technologies have managed the entry and use of public housing through intelligent systems, most public housing management systems still rely on single facial recognition or only process data through smart contracts, lacking real-time response and dynamic adjustment to users' actual behaviors and intentions. For example, when technical workers need to enter public housing for repairs, since they are not tenants, the system may not be able to effectively identify their identities, resulting in the technical workers being unable to enter smoothly. This situation limits the flexibility and accuracy of the management system.

[0004] In view of this, the present invention proposes a dynamic control system and method based on multi-source data fusion to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a dynamic management and control system and method based on multi-source data fusion.

[0006] To achieve the above object, the present invention provides the following technical solutions: First, a dynamic control method based on multi-source data fusion is provided, including: S10: Obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, proceed to S20; if not, proceed to S30; S20: determining password-related information according to the image to be recognized, sending the password-related information to the current object, obtaining first reaction information of the current object, determining whether to generate a face verification sequence according to the first reaction information, and if so, generating a face verification sequence, and performing face verification on the current object based on the face verification sequence, and when the verification passes, opening the smart door lock, and when the verification fails, closing the smart door lock; S30: Determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

[0007] Furthermore, the method of determining whether the current object is a tenant in a public rental housing according to the image to be identified includes: Obtain light distribution information of the image to be identified, determine a corresponding light compensation template based on the light distribution information, construct a target recognition image based on the image to be identified and the light compensation template, input the target recognition image into a pre-built object classification model, and obtain the object classification result. If the object classification result is a tenant, the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, the current object is not a tenant in the public rental housing.

[0008] Furthermore, the method for obtaining light distribution information of the image to be recognized includes: Obtain pixel gradient values ​​of the image to be identified, construct a distribution contour map corresponding to the image to be identified according to the pixel gradient values, divide the distribution contour map into a highlight area, a shadow area, and a uniformly illuminated area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniformly illuminated area as light distribution information.

[0009] Furthermore, the method for determining a corresponding light compensation template according to light distribution information includes: The sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are determined in turn according to the light distribution information, and the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are spliced ​​to obtain the light compensation template.

[0010] Furthermore, the method for constructing the object classification model includes: The training data is obtained and divided into a training set and a test set. The historical target recognition images in the training set are used as the input data of the initial vector machine, and the historical object classification results in the training set are used as the output labels of the initial vector machine. The optimal separation hyperplane of the initial vector machine is determined by the Lagrange multiplier method. The optimal separation hyperplane is mapped from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function. The performance of the initial vector machine is verified by the test set. When the classification accuracy meets the preset requirements, the object classification model is obtained through training.

[0011] Furthermore, the first reaction information includes a user action image, and the method for determining whether to generate a face verification sequence according to the first reaction information includes: Based on the OpenPose algorithm, Y human body key points are extracted from the user action image, a human body action sequence is constructed according to the Y human body key points, and it is determined whether the human body action sequence is similar to a pre-constructed standard action sequence. If so, a face verification sequence is generated; if not, no face verification sequence is generated. The standard action sequence is characterized by a standard behavior action sequence of normally checking the password and attempting to unlock after receiving password-related information.

[0012] Furthermore, the method for determining whether the human motion sequence is similar to the pre-built standard motion sequence includes: The similarity between the human motion sequence and the standard motion sequence is calculated. When the similarity is greater than a preset similarity threshold, they are similar; when the similarity is less than or equal to the preset similarity threshold, they are not similar.

[0013] Furthermore, the method for generating a face verification sequence includes: Based on the directional gradient histogram algorithm, feature extraction is performed on a preset face image set to generate a face image feature sequence, and the face image feature sequence is randomly fused with an unlocking password in password-related information to generate a face verification sequence, wherein the face image set is a face image set uploaded by the tenant.

[0014] Furthermore, the second reaction information includes an intention reaction image, and the method for verifying the intention of the current object according to the second reaction information includes: The corresponding intention reaction feature is extracted according to the intention reaction image in the second reaction information, and the intention reaction feature is compared with the standard reaction feature in the verification requirement information. When the intention reaction feature matches the standard reaction feature, the verification passes; when the intention reaction feature does not match the standard reaction feature, the verification fails.

[0015] In a second aspect, a dynamic control system based on multi-source data fusion is provided, which is used to implement the above-mentioned dynamic control method based on multi-source data fusion, including: Object recognition module: used to obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, it will be transferred to the first verification module, if not, it will be transferred to the second verification module; The first verification module is used to determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first reaction information of the current object, determine whether to generate a face verification sequence according to the first reaction information, and if so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed; The second verification module: used to determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention first determines whether the current object is a tenant in a public rental housing according to the image to be identified. If the current object is a tenant in the public rental housing, password-related information is determined according to the image to be identified, the password-related information is sent to the current object, first reaction information of the current object is obtained, and whether a face verification sequence is generated according to the first reaction information. If so, a face verification sequence is generated, and face verification is performed on the current object based on the face verification sequence. If the current object is not a tenant in the public rental housing, verification requirement information is generated according to intention information, the verification requirement information is sent to the current object, and second reaction information of the current object is obtained, and intention verification is performed on the current object according to the second reaction information. The present invention solves the problem that the existing system cannot flexibly identify different identities by dynamically determining whether the object is a tenant and adopting password verification and intention verification methods for tenants and non-tenants respectively, thereby improving the flexibility and accuracy of the management system and significantly optimizing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the dynamic control method based on multi-source data fusion in the present invention; Figure 2 It is a structural schematic diagram of a dynamic control system based on multi-source data fusion in the present invention; Figure 3 It is a flowchart of a method for judging whether a current object is a tenant in a public rental housing according to an image to be recognized in the present invention; Figure 4 The figure is a flow chart of the method for generating a face verification sequence in the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Example 1 See also Figure 1 As shown, this embodiment discloses a dynamic control method based on multi-source data fusion, including: S10: Obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, proceed to S20; if not, proceed to S30; In this embodiment, the current object can be a tenant in the public rental housing, or a staff member who needs to enter the public rental housing to check the gas data, or a technical worker who needs to enter the public rental housing to perform repairs. This embodiment is not limited to this. The image to be recognized can be a face image of the current object, or an upper body image of the current object. The voice to be recognized can be the intended voice emitted by the current object. The intended voice expresses a question about whether there are tenants in the current public rental housing, or expresses the current intention. Taking the above-mentioned technical workers as an example, the current intention can be to clearly express the purpose of entering the public rental housing, for example: "I am here to perform maintenance work" or "I am here to check the gas equipment."

[0020] It should be added that the above-mentioned images to be recognized or voices to be recognized can be collected through the camera and sensors built into the smart door lock, or obtained through connected external devices, such as surveillance cameras and sensors outside the door, and this embodiment does not limit this.

[0021] like Figure 3 As shown, the method for determining whether the current object is a tenant in a public rental housing according to the image to be identified includes: Obtain light distribution information of the image to be identified, determine a corresponding light compensation template based on the light distribution information, construct a target recognition image based on the image to be identified and the light compensation template, input the target recognition image into a pre-built object classification model, and obtain the object classification result. If the object classification result is a tenant, the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, the current object is not a tenant in the public rental housing.

[0022] It should be noted that when tenants in public housing sign a lease contract with the rental platform, they also need to upload their own profile pictures to the platform. It is understandable that, under normal circumstances, the background light of the tenant's profile picture is relatively uniform and stable. This is because tenants usually upload their profile pictures in an indoor environment, and the lighting conditions are relatively fixed, avoiding strong light contrast or complex background interference. The image to be identified is collected in the environment where the smart door lock is installed. This environment is usually located outdoors, resulting in uneven light distribution of the image to be identified. For example, strong light, shadows or insufficient light may occur. Therefore, in order to improve the accuracy of recognition, it is necessary to optimize the image to be identified through a light compensation template to construct a higher quality target recognition image to adapt to different ambient light conditions.

[0023] The method for obtaining light distribution information of an image to be recognized includes: Obtain pixel gradient values ​​of the image to be identified, construct a distribution contour map corresponding to the image to be identified according to the pixel gradient values, divide the distribution contour map into a highlight area, a shadow area, and a uniformly illuminated area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniformly illuminated area as light distribution information.

[0024] It is not difficult to understand that the pixel gradient value refers to the degree of brightness change between adjacent pixels in the image to be identified, which reflects the changing trend of the light intensity in the image. The distribution contour map refers to a graphical representation of the brightness gradient distribution of a visualized image generated based on the changing trend of the pixel gradient value in the image to be identified. It draws contour lines (similar to the contour lines in a topographic map) to mark the pixel areas in the image with the same or similar brightness gradient values ​​with closed curves. The distribution contour map can be drawn by the contour function in Matlab. In the above content, the distribution contour map is divided into highlight areas, shadow areas and uniformly illuminated areas, which can be achieved by presetting the gradient threshold. This embodiment does not provide too many detailed explanations on this.

[0025] The method for determining the corresponding light compensation template according to the light distribution information includes: The sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are determined in turn according to the light distribution information, and the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are spliced ​​to obtain a light compensation template.

[0026] In this embodiment, multiple sub-compensation templates can be pre-stored in the database. The function of the sub-compensation templates is to provide corresponding light adjustment strategies for different light distribution areas (highlight area, shadow area and uniform illumination area). For example, for the highlight area, the sub-compensation template can reduce the brightness of the highlight area, reduce overexposure, and make the detail information in the image clearer. It can be understood that the area of ​​the highlight area is determined according to the position coordinate information of the highlight area, and the sub-compensation template is adjusted to match the position and area of ​​the highlight area to ensure that the light compensation effect can accurately cover the highlight area.

[0027] The methods for building object classification models include: The training data is obtained and divided into a training set and a test set. The historical target recognition images in the training set are used as the input data of the initial vector machine, and the historical object classification results in the training set are used as the output labels of the initial vector machine. The optimal separation hyperplane of the initial vector machine is determined by the Lagrange multiplier method. The optimal separation hyperplane is mapped from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function. The performance of the initial vector machine is verified by the test set. When the classification accuracy meets the preset requirements, the object classification model is obtained through training.

[0028] It should be noted that the initial vector machine is a classifier built based on the support vector machine (SVM) algorithm. It inputs the features in the training data (historical target recognition images) and the corresponding classification labels (historical object classification results) and trains a model that can distinguish different categories through optimization algorithms (such as the Lagrange multiplier method). The core of the initial vector machine is to find a hyperplane that separates different categories and expand its capabilities through kernel functions to handle complex nonlinear classification problems.

[0029] In this embodiment, the optimal separation hyperplane is the core part of the support vector machine. The optimal separation hyperplane is a linear plane that can maximize the interval between categories and is used to separate samples into different categories. In two-dimensional space, the optimal separation hyperplane is a line, and in three-dimensional space and above, it is a hyperplane. The support vector machine finds this plane through an optimization algorithm (such as the Lagrange multiplier method) to achieve optimal classification. The Gaussian kernel function is a commonly used kernel function that can implicitly map the original low-dimensional data to a high-dimensional space. In a high-dimensional space, many data that cannot be linearly separated can be linearly separated. The Gaussian kernel function is a tool for extending SVM to nonlinear problems and can map low-dimensional data to a high-dimensional space to achieve more complex classification. In the above content, determining the optimal separation hyperplane of the initial vector machine by the Lagrange multiplier method and mapping the optimal separation hyperplane from a low-dimensional space to a high-dimensional space based on the Gaussian kernel function are prior arts, and this embodiment will not go into too much detail.

[0030] This embodiment first calculates the pixel gradient value and generates a distribution contour map, which helps to accurately divide the highlight area, shadow area and uniform illumination area, which provides a basis for the subsequent selection of the light compensation template, and uses the light distribution information to determine the corresponding sub-compensation template, which is sequentially spliced ​​into a complete light compensation template to ensure that different light distribution areas are targeted for optimization processing, thereby generating a higher quality target recognition image. The light optimization of the target recognition image improves the classification accuracy of the model, especially in outdoor scenes with complex ambient light conditions (such as strong light, shadow, and insufficient light). At the same time, the process diversion of the object classification results ensures that the needs of different objects can be responded to in a timely manner, thereby improving the flexibility and efficiency of the system in practical applications.

[0031] S20: determining password-related information according to the image to be recognized, sending the password-related information to the current object, obtaining first reaction information of the current object, determining whether to generate a face verification sequence according to the first reaction information, and if so, generating a face verification sequence, and performing face verification on the current object based on the face verification sequence, and when the verification passes, opening the smart door lock, and when the verification fails, closing the smart door lock; In this embodiment, the password-related information at least includes the unlocking password of the smart door lock and the mobile phone number of the corresponding tenant. It should be noted that, usually, there are multiple tenants in the public rental housing, and each tenant has a corresponding unlocking password. When the tenant signs a lease contract with the rental platform, the rental platform will also send the corresponding unlocking password via mobile phone text message. Therefore, sending the password-related information to the current object means sending the unlocking password of the smart door lock to the mobile phone number or WeChat account of the corresponding tenant, etc. This embodiment does not limit this.

[0032] It should be added that the first reaction information at least includes a user action image, and the user action image refers to an image of a response action made by the current object after sending the password-related information.

[0033] The method for determining whether to generate a face verification sequence according to the first response information includes: Based on the OpenPose algorithm, Y human body key points are extracted from the user action image, a human body action sequence is constructed according to the Y human body key points, and it is determined whether the human body action sequence is similar to a pre-constructed standard action sequence. If so, a face verification sequence is generated; if not, no face verification sequence is generated. The standard action sequence is characterized by a standard behavior action sequence of normally checking the password and attempting to unlock after receiving password-related information.

[0034] It should be noted that the OpenPose algorithm is a human posture estimation algorithm based on deep learning. It can extract key points of the human body (such as head, arms, wrists, knees, etc.) from images or videos and construct a human skeleton model. The above-mentioned user action image can be a number of adjacent action images. The key points of the human body include elbow key points, shoulder key points, wrist key points and head key points, etc. The above-mentioned human body action sequence refers to the dynamic change of human body key points to generate a time series graph. The human body action sequence can be constructed by existing Matlab software or Unity software, and this embodiment will not go into details.

[0035] Methods for determining whether a human motion sequence is similar to a pre-built standard motion sequence include: The similarity between the human motion sequence and the standard motion sequence is calculated. When the similarity is greater than a preset similarity threshold, they are similar; when the similarity is less than or equal to the preset similarity threshold, they are not similar.

[0036] In this embodiment, the similarity can be achieved by calculating the Euclidean distance or the cosine value.

[0037] like Figure 4 As shown, the method for generating a face verification sequence includes: Based on the directional gradient histogram algorithm, feature extraction is performed on a preset face image set to generate a face image feature sequence, and the face image feature sequence is randomly fused with an unlocking password in password-related information to generate a face verification sequence, wherein the face image set is a face image set uploaded by the tenant.

[0038] It should be noted that when tenants in public housing sign a lease contract with the rental platform, they also need to upload their own avatars to the platform. Therefore, the database will store a set of face images uploaded by the tenants. The directional gradient histogram algorithm refers to an image feature extraction algorithm, which calculates the gradient direction distribution characteristics of the local area in the image. The face image feature sequence is composed of several image feature vectors combined in sequence. Similarly, the unlocking password can be converted into a set of password feature vectors. Random fusion usually refers to randomly combining image feature vectors and password feature vectors to form a new vector.

[0039] Exemplarily, the facial image feature sequence includes a left head turning feature, a blinking feature, a nodding feature, a right head turning feature and a mouth opening feature. If the unlocking password is 24587, the password feature vector includes a number 2 feature, a number 4 feature, a number 5 feature, a number 8 feature and a number 7 feature. The above-mentioned random fusion of the facial image feature sequence and the unlocking password in the password-related information can be a combination of the number 2 feature and the left head turning feature, the number 4 feature and the mouth opening feature, the number 5 feature and the blinking feature, the number 8 feature and the nodding feature, and the number 7 feature and the right head turning feature. Therefore, the fused face verification sequence is: left head turning, mouth opening, blinking, nodding, and right head turning.

[0040] It is understandable that when tenants in public housing sign a lease contract with a rental platform, the user can upload his or her own facial image, and the rental platform can also collect the user's facial image and obtain head turning features, blinking features, and nodding features based on the collected facial images. This embodiment does not limit this.

[0041] It should be added that the reason why the present embodiment adopts face verification sequence for verification instead of fingerprint and other methods is that the flow of people in public rental housing is dense, and fingerprints must be collected and entered before fingerprint unlocking, which requires additional equipment support and high labor costs, and when personnel are frequently replaced, there are great inconveniences in updating and managing fingerprint information. Therefore, the present embodiment implements verification through face verification sequence, and can directly use face image data uploaded by users or face image data collected by the platform, which not only avoids equipment dependence and complex operations, but also adapts to the actual situation of frequent personnel changes, and improves the flexibility and efficiency of verification.

[0042] It is not difficult to understand that in this embodiment, it is determined whether the human body motion sequence is similar to the pre-built standard motion sequence. If they are similar, it means that the tenant has received the unlocking password through the mobile phone, so there is no need for subsequent face verification. If they are not similar, it means that the tenant does not carry a mobile phone, etc. Therefore, in order to further ensure security, subsequent verification is required.

[0043] In this embodiment, the effect of randomly fusing the facial image feature sequence with the unlocking password in the password-related information is that the generated facial verification sequence is unique for each tenant and is directly bound to the tenant's identity information. During the verification process, the unlocking password is directly involved in the fusion, avoiding the storage and management costs of generating additional passwords. After the unlocking password is integrated into the verification sequence, the verification process depends on the correctness of the unlocking password. Combined with the correctness of the facial feature action, the double verification greatly improves security.

[0044] S30: Determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

[0045] In this embodiment, the intent information may be text information expressing the reason for entering the public rental housing. For example, taking a technical worker as an example, the voice to be recognized may be "I am here to check the gas equipment", and the image to be recognized may identify which company the technical worker is from or the organization to which the work uniform he is wearing indicates. Then the intent information may be "a technical worker from a gas company, responsible for checking the gas equipment".

[0046] The method of generating verification requirement information according to the intention information includes: Input the intent information into the pre-built requirement generation model to obtain verification requirement information.

[0047] The construction of the requirement generation model can be expanded and customized based on the existing large language model or rule-based generation engine to generate verification requirement information for specific intent information. The existing large language model can be BERT, ChatGLM series or Spark large model. The historical intent information is used as the input of the large language model, and the historical verification requirement information is used as the output of the large language model, so as to fine-tune or train the large language model.

[0048] In this embodiment, the verification requirement information represents the specific requirements generated for the intention information of the current object for verifying the authenticity of its identity and the legality of its behavior. For example, if the intention information is "a technical worker from a gas company, responsible for inspecting gas equipment", then the verification requirement information is "please show your gas company work permit."

[0049] In the above, the second reaction information at least includes an intention reaction image, and the intention reaction image refers to an image of a response behavior made by the current object to complete the verification requirement after receiving the verification requirement information. The method of performing intention verification on the current object according to the second reaction information includes: The corresponding intention reaction feature is extracted according to the intention reaction image in the second reaction information, and the intention reaction feature is compared with the standard reaction feature in the verification requirement information. When the intention reaction feature matches the standard reaction feature, the verification passes; when the intention reaction feature does not match the standard reaction feature, the verification fails.

[0050] Exemplarily, the verification requirement information is "Please show your gas company work ID", then the intention response image is a picture of the current object holding the work ID and showing it to the camera. The intention response features include extracted text information (such as "a gas company", "employee name", etc.) and the posture of holding the work ID. It can be understood that when the intention response features match the standard response features, it indicates that the identity authenticity and behavior legality of the current object have been verified, so the smart door lock can be opened, which can not only ensure security but also avoid excessive verification steps or additional manual intervention.

[0051] In this embodiment, it is first determined whether the current object is a tenant in the public rental housing based on the image to be identified. If the current object is a tenant in the public rental housing, password-related information is determined based on the image to be identified, the password-related information is sent to the current object, and the first reaction information of the current object is obtained. It is determined whether to generate a face verification sequence based on the first reaction information. If so, a face verification sequence is generated, and face verification is performed on the current object based on the face verification sequence. If the current object is not a tenant in the public rental housing, verification requirement information is generated based on the intention information, the verification requirement information is sent to the current object, and the second reaction information of the current object is obtained. The intention verification is performed on the current object based on the second reaction information. This embodiment dynamically determines whether the object is a tenant, and adopts password verification and intention verification methods for tenants and non-tenants respectively, thereby solving the problem that the existing system cannot flexibly identify different identities, thereby improving the flexibility and accuracy of the management system and significantly optimizing the user experience.

[0052] Example 2 See also Figure 2 As shown, based on the same inventive concept, this embodiment discloses a dynamic management and control system based on multi-source data fusion. For details not provided in this embodiment, please refer to the description of the relevant parts in Embodiment 1. The system includes: Object recognition module: used to obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, it will be transferred to the first verification module, if not, it will be transferred to the second verification module; The method for determining whether the current object is a tenant in a public rental housing according to the image to be recognized includes: Obtain light distribution information of the image to be identified, determine a corresponding light compensation template based on the light distribution information, construct a target recognition image based on the image to be identified and the light compensation template, input the target recognition image into a pre-built object classification model, and obtain the object classification result. If the object classification result is a tenant, the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, the current object is not a tenant in the public rental housing.

[0053] The method for obtaining light distribution information of an image to be recognized includes: Obtain pixel gradient values ​​of the image to be identified, construct a distribution contour map corresponding to the image to be identified according to the pixel gradient values, divide the distribution contour map into a highlight area, a shadow area, and a uniformly illuminated area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniformly illuminated area as light distribution information.

[0054] The method for determining the corresponding light compensation template according to the light distribution information includes: The sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are determined in turn according to the light distribution information, and the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are spliced ​​to obtain a light compensation template.

[0055] The first verification module is used to determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first reaction information of the current object, determine whether to generate a face verification sequence according to the first reaction information, and if so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed; It should be added that the first reaction information at least includes a user action image, and the user action image refers to an image of a response action made by the current object after sending the password-related information.

[0056] The method for determining whether to generate a face verification sequence according to the first response information includes: Based on the OpenPose algorithm, Y human body key points are extracted from the user action image, a human body action sequence is constructed according to the Y human body key points, and it is determined whether the human body action sequence is similar to a pre-constructed standard action sequence. If so, a face verification sequence is generated; if not, no face verification sequence is generated. The standard action sequence is characterized by a standard behavior action sequence of normally checking the password and attempting to unlock after receiving password-related information.

[0057] Methods for determining whether a human motion sequence is similar to a pre-built standard motion sequence include: The similarity between the human motion sequence and the standard motion sequence is calculated. When the similarity is greater than a preset similarity threshold, they are similar; when the similarity is less than or equal to the preset similarity threshold, they are not similar.

[0058] In this embodiment, the similarity can be achieved by calculating the Euclidean distance or the cosine value.

[0059] Methods for generating face verification sequences include: Based on the directional gradient histogram algorithm, feature extraction is performed on a preset face image set to generate a face image feature sequence, and the face image feature sequence is randomly fused with an unlocking password in password-related information to generate a face verification sequence, wherein the face image set is a face image set uploaded by the tenant.

[0060] The second verification module: used to determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

[0061] The method of generating verification requirement information according to the intention information includes: Input the intent information into the pre-built requirement generation model to obtain verification requirement information.

[0062] The construction of the requirement generation model can be expanded and customized based on the existing large language model or rule-based generation engine to generate verification requirement information for specific intent information. The existing large language model can be BERT, ChatGLM series or Spark large model. The historical intent information is used as the input of the large language model, and the historical verification requirement information is used as the output of the large language model, so as to fine-tune or train the large language model.

[0063] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0064] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dynamic control method based on multi-source data fusion, characterized in that: include: S10: Obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, proceed to S20; if not, proceed to S30; S20: determining password-related information according to the image to be recognized, sending the password-related information to the current object, obtaining first reaction information of the current object, determining whether to generate a face verification sequence according to the first reaction information, and if so, generating a face verification sequence, and performing face verification on the current object based on the face verification sequence, and when the verification passes, opening the smart door lock, and when the verification fails, closing the smart door lock; S30: Determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

2. The dynamic control method based on multi-source data fusion according to claim 1 is characterized in that: The method for determining whether the current object is a tenant in a public rental housing according to the image to be identified comprises: Obtain light distribution information of the image to be identified, determine a corresponding light compensation template based on the light distribution information, construct a target recognition image based on the image to be identified and the light compensation template, input the target recognition image into a pre-trained object classification model, and obtain the object classification result. If the object classification result is a tenant, the current object is a tenant in the public rental housing. If the object classification result is a non-tenant, the current object is not a tenant in the public rental housing.

3. The dynamic control method based on multi-source data fusion according to claim 2 is characterized in that: The method for obtaining light distribution information of an image to be identified includes: Obtain pixel gradient values ​​of the image to be identified, construct a distribution contour map corresponding to the image to be identified according to the pixel gradient values, divide the distribution contour map into a highlight area, a shadow area, and a uniformly illuminated area, and use the position coordinate information of the highlight area, the position coordinate information of the shadow area, and the position coordinate information of the uniformly illuminated area as light distribution information.

4. The dynamic control method based on multi-source data fusion according to claim 3 is characterized in that: The method for determining a corresponding light compensation template according to light distribution information comprises: The sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are determined in turn according to the light distribution information, and the sub-compensation template corresponding to the highlight area, the sub-compensation template corresponding to the shadow area, and the sub-compensation template corresponding to the uniform illumination area are spliced ​​to obtain the light compensation template.

5. The dynamic control method based on multi-source data fusion according to claim 2 is characterized in that: The training method of the object classification model includes: The training data is obtained and divided into a training set and a test set. The historical target recognition images in the training set are used as the input data of the initial vector machine, and the historical object classification results in the training set are used as the output labels of the initial vector machine. The optimal separation hyperplane of the initial vector machine is determined by the Lagrange multiplier method. The optimal separation hyperplane is mapped from the low-dimensional space to the high-dimensional space based on the Gaussian kernel function. The performance of the initial vector machine is verified by the test set. When the classification accuracy meets the preset requirements, the object classification model is obtained through training.

6. The dynamic control method based on multi-source data fusion according to claim 4 is characterized in that: The first reaction information includes a user action image, and the method for determining whether to generate a face verification sequence according to the first reaction information includes: Based on the OpenPose algorithm, Y human body key points are extracted from the user action image, a human body action sequence is constructed according to the Y human body key points, and it is determined whether the human body action sequence is similar to a pre-constructed standard action sequence. If so, a face verification sequence is generated; if not, no face verification sequence is generated. The standard action sequence is characterized by a standard behavior action sequence of normally checking the password and attempting to unlock after receiving password-related information.

7. The dynamic control method based on multi-source data fusion according to claim 6 is characterized in that: The method for determining whether a human motion sequence is similar to a pre-built standard motion sequence comprises: The similarity between the human motion sequence and the standard motion sequence is calculated. When the similarity is greater than a preset similarity threshold, they are similar; when the similarity is less than or equal to the preset similarity threshold, they are not similar.

8. The dynamic control method based on multi-source data fusion according to claim 6 is characterized in that: The method for generating a face verification sequence comprises: Based on the directional gradient histogram algorithm, feature extraction is performed on a preset face image set to generate a face image feature sequence, and the face image feature sequence is randomly fused with an unlocking password in password-related information to generate a face verification sequence, wherein the face image set is a face image set uploaded by the tenant.

9. The dynamic control method based on multi-source data fusion according to claim 1 is characterized in that: The second reaction information includes an intention reaction image, and the method for verifying the intention of the current object according to the second reaction information includes: The corresponding intention reaction feature is extracted according to the intention reaction image in the second reaction information, and the intention reaction feature is compared with the standard reaction feature in the verification requirement information. When the intention reaction feature matches the standard reaction feature, the verification passes; when the intention reaction feature does not match the standard reaction feature, the verification fails.

10. A dynamic control system based on multi-source data fusion, which is used to implement the dynamic control method based on multi-source data fusion as described in any one of claims 1 to 9, characterized in that: include: Object recognition module: used to obtain the image to be recognized and the voice to be recognized of the current object, and determine whether the current object is a tenant in the public rental housing according to the image to be recognized. If so, it will be transferred to the first verification module, if not, it will be transferred to the second verification module; The first verification module is used to determine password-related information according to the image to be recognized, send the password-related information to the current object, obtain the first reaction information of the current object, determine whether to generate a face verification sequence according to the first reaction information, and if so, generate a face verification sequence, and perform face verification on the current object based on the face verification sequence. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed; The second verification module: used to determine the intention information of the current object based on the image to be recognized and the voice to be recognized, generate verification requirement information based on the intention information, send the verification requirement information to the current object, and obtain the second reaction information of the current object, and verify the intention of the current object based on the second reaction information. When the verification passes, the smart door lock is opened, and when the verification fails, the smart door lock is closed.

Images