Smart home control method, device and equipment and computer readable medium

Abstract

The invention relates to a smart home control method, device and equipment and a computer readable medium. The method comprises the following steps: when it is detected that a target object enters a user detection range of a target smart home, collecting environment data of a space where the target object is located and an eye image of the target object; extracting at least one eye feature based on the eye image, and extracting an environment feature based on the environment data; performing weighted fusion on the at least one eye feature and the environment feature, and calculating the identity confidence of the target object based on the fused feature; and under the condition that the identity confidence reaches a target threshold value, waking up the target smart home, and synchronizing the target smart home with the personalized setting of the target object. According to the method and the device, the multi-modal decision is performed by fusing various eye features and environment features, so that the technical problem of low recognition accuracy of an existing smart home safety control scheme based on iris recognition due to environment change or proneness to spoofing attacks is solved, and the control safety of the smart home is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart home control, and in particular to a smart home control method and device, equipment and a computer readable medium. BACKGROUND

[0002] With the popularity of smart homes, users have higher demands for the convenience and security of device control. Biometric identification has become an important means of smart home identity verification because it does not require the carrying of physical media and has strong uniqueness. Iris recognition has received widespread attention in the field of biometric identification due to the uniqueness and stability of iris texture.

[0003] However, existing smart home security control schemes based on iris recognition still have the problem of low recognition accuracy due to environmental changes or vulnerability to spoofing attacks, resulting in unsafe control of smart homes.

[0004] In view of the problem of low recognition accuracy of existing smart home security control schemes based on iris recognition due to environmental changes or vulnerability to spoofing attacks, no effective solution has been proposed so far. SUMMARY

[0005] The present application provides a smart home control method, device, equipment and computer readable medium to solve the technical problem of low recognition accuracy of existing smart home security control schemes based on iris recognition due to environmental changes or vulnerability to spoofing attacks.

[0006] According to one aspect of an embodiment of the present application, the present application provides a smart home control method, comprising: in the case of detecting that a target object enters a user detection range of a target smart home, collecting environmental data of a space where the target object is located and an eye image of the target object; extracting at least one eye feature based on the eye image and extracting an environmental feature based on the environmental data; weighting and fusing the at least one eye feature and the environmental feature, and calculating an identity confidence of the target object based on the fused feature; in the case that the identity confidence reaches a target threshold, waking up the target smart home and synchronizing the target smart home with the personalized settings of the target object.

[0007] Optionally, extracting at least one eye feature based on the eye image comprises: extracting an iris feature of the target object based on the eye image; and / or, extracting an eye movement feature of the target object based on the eye image.

[0008] Optionally, extracting the iris feature of the target object based on the eye image comprises: denoising and enhancing the eye image to obtain a preprocessed image; locating the iris boundary of the target object in the preprocessed image; extracting a multi-dimensional iris feature vector from multiple directions of the iris boundary respectively; and combining the multi-dimensional iris feature vectors extracted in each direction into a feature matrix as the iris feature.

[0009] Optionally, the locating the iris boundary of the target object in the preprocessed image comprises: extracting edges in the preprocessed image by using an edge detection operator to obtain an edge image marked with edge points and gradient directions of each edge point; in the edge image, determining points on a straight line of the gradient direction of each edge point as candidate circle centers; constructing a parameter space with coordinates of each candidate circle center and a corresponding radius as circular parameters by taking distances from the edge points to the candidate circle centers as the radius; traversing each edge point to perform a weighted voting on each circle in the parameter space; and determining a target circle with the highest number of votes as the iris boundary of the target object.

[0010] Optionally, the extracting the eye movement feature of the target object based on the eye images comprises: tracking an eye movement trajectory of the target object in the plurality of eye images; splitting the eye movement trajectory into a first time series signal in the X direction and a second time series signal in the Y direction; performing a fast Fourier transform on the first time series signal and the second time series signal respectively to obtain a first frequency spectrum in the X direction and a second frequency spectrum in the Y direction; and extracting a plurality of frequency components with a frequency energy greater than or equal to a preset energy threshold from the first frequency spectrum and the second frequency spectrum respectively as the eye movement feature.

[0011] Optionally, the weighting and fusing the at least one eye feature and the environment feature, and calculating the identity confidence of the target object based on the fused feature comprises: calculating a cosine similarity between the iris feature and a preset iris feature of the target object to obtain an iris matching degree, wherein the preset iris feature is an iris feature previously extracted from the target object and used as a standard; calculating a similarity between the eye movement feature and each frequency component in a preset eye movement feature of the target object, and calculating an average value of the similarities of the frequency components to obtain an eye movement matching degree, wherein the preset eye movement feature is an eye movement feature previously extracted from the target object and used as a standard; obtaining a historical control record of the target object on the target smart home, extracting an environment preference feature of the target object from the historical control record, and calculating a similarity between the environment feature and the environment preference feature to obtain an environment matching degree; and calculating a weighted score of at least one of the iris matching degree, the eye movement matching degree, and the environment matching degree as the identity confidence.

[0012] Optionally, after the target smart home is woken up, the method further comprises: determining a target action to be performed by the target smart home according to the eye movement trajectory of the target object; and sending a target control instruction to the target smart home to make the target smart home perform the target action.

[0013] According to another aspect of the embodiments of the present application, the present application provides an intelligent home control device, comprising: a data acquisition module, configured to acquire environmental data of a space where a target object is located and an eye image of the target object when it is detected that the target object enters a user detection range of a target intelligent home; a feature extraction module, configured to extract at least one eye feature based on the eye image and extract an environmental feature based on the environmental data; a multi-modal decision module, configured to perform weighted fusion on the at least one eye feature and the environmental feature, and calculate an identity confidence of the target object based on the fused features; and a device control module, configured to wake up the target intelligent home and synchronize personalized settings of the target intelligent home to the target object when the identity confidence reaches a target threshold.

[0014] According to another aspect of the embodiments of the present application, the present application provides an electronic device, comprising a memory, a processor, a communication interface and a communication bus, the memory stores a computer program executable on the processor, the memory, the processor and the communication interface communicate through the communication bus, and the processor executes the computer program to implement the steps of the above method.

[0015] According to another aspect of the embodiments of the present application, the present application further provides a computer readable medium having a non-volatile program code executable by a processor, the program code causing the processor to execute the above method.

[0016] The above technical solutions provided by the embodiments of the present application have the following advantages compared with related art: The present application provides an intelligent home control method, comprising: acquiring environmental data of a space where a target object is located and an eye image of the target object when it is detected that the target object enters a user detection range of a target intelligent home; extracting at least one eye feature based on the eye image and extracting an environmental feature based on the environmental data; performing weighted fusion on the at least one eye feature and the environmental feature, and calculating an identity confidence of the target object based on the fused features; and waking up the target intelligent home and synchronizing personalized settings of the target intelligent home to the target object when the identity confidence reaches a target threshold. The present application performs multi-modal decision by fusing multiple eye features and environmental features, replaces single iris recognition, adapts to environmental changes through environmental features to reduce the influence of environmental interference on recognition, and improves the anti-fraud ability of identity verification through multi-feature cooperation, thereby reducing recognition error, improving identity verification accuracy, solving the technical problem that the existing intelligent home security control scheme based on iris recognition still has low recognition accuracy due to environmental changes or easy fraud attacks, and thereby improving the control security of the intelligent home. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the hardware environment for an optional smart home control method provided according to an embodiment of this application; Figure 2 This is a schematic diagram of an optional smart home control method provided according to an embodiment of this application; Figure 3 This is a schematic diagram of another optional smart home control method provided according to an embodiment of this application; Figure 4 This is a block diagram of an optional smart home control device according to an embodiment of this application; Figure 5 This is a schematic diagram of an optional electronic device structure provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0022] To address the problems mentioned in the background art, according to one aspect of the embodiments of this application, an embodiment of a smart home control method is provided.

[0023] Optionally, in the embodiments of this application, the above-described smart home control method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal 101 and server 103. Figure 1As shown, server 103 is connected to terminal 101 via a network and can be used to provide services to the terminal or clients installed on the terminal. Database 105 can be set up on the server or independently of the server to provide data storage services for server 103. The network mentioned above includes, but is not limited to, wide area network, metropolitan area network or local area network. Terminal 101 includes, but is not limited to, smart home devices, PCs, mobile phones, tablets, etc.

[0024] The smart home control method in this embodiment can be executed by server 103, or it can be jointly executed by server 103 and terminal 101, such as... Figure 2 As shown, the method may include the following steps: Step S202: When the target object is detected to have entered the user detection range of the target smart home, environmental data of the space where the target object is located and eye images of the target object are collected. Step S204: Extract at least one eye feature based on the eye image, and extract environmental features based on the environmental data; Step S206: Weighted fusion of at least one eye feature and environmental feature, and calculation of the target object's identity confidence based on the fused feature; Step S208: When the identity confidence level reaches the target threshold, wake up the target smart home and synchronize the target smart home with the personalized settings of the target object.

[0025] In this embodiment, the target object refers to a user who may trigger smart home control, such as a permanent family member or authorized visitor who needs to obtain smart home control permissions through identity verification. The user detection range is a spatial range preset by the smart home system to trigger data collection. In this embodiment, it is set to a 3-meter circular area centered on the sensor based on the TOF sensor. Environmental data are parameters reflecting the environmental state of the space where the target object is located, including but not limited to ambient temperature, relative humidity, light intensity, and the device status of smart home devices, such as whether the TV is currently on or off, and the current temperature setting of the air conditioner. Eye images refer to images containing key eye areas such as the target object's eyeball and iris, captured by an infrared camera. Eye features refer to biometric features extracted from eye images for identification, including iris features and eye movement features. Weighted fusion refers to the process of assigning preset weights to each feature based on its importance in identity verification and calculating a comprehensive score. In this embodiment, if iris features, eye-tracking features, and environmental features are weighted and fused, the weight of iris features can be set to 60%, eye-tracking features to 30%, and environmental features to 10%. If iris features and environmental features are weighted and fused, the weight of iris features can be set to 80% and environmental features to 20%. If eye-tracking features and environmental features are weighted and fused, the weight of eye-tracking features can be set to 70% and environmental features to 30%. Identity confidence is a quantitative value representing the degree of identity matching of the target object, calculated based on the fused features. The value ranges from 0 to 1, with a higher value indicating a higher degree of identity matching. The target threshold is a preset identity confidence threshold used by the system to determine whether identity verification is successful. In this embodiment, it can be set to 0.95; a value higher than this value indicates successful verification. Personalized settings refer to device parameters that the target object has pre-configured in the smart home system, such as the default temperature of the air conditioner at 26℃, the commonly used playback platform of the TV, and the brightness of the lights at 50%. They can also be smart home settings matched according to the user's historical behavior habits.

[0026] In this embodiment, when the TOF sensor detects human activity within a 3-meter range, it triggers the infrared camera to start acquiring images of the target object's eyes; simultaneously, the environmental sensor collects environmental data. The eye images acquired by the infrared camera are input into the feature extraction module, where the images are first preprocessed, and then at least one eye feature is extracted according to a preset algorithm. The acquired environmental data is also preprocessed, and environmental features are extracted. At least one extracted eye feature (such as iris feature or eye movement feature) and environmental features are input into the multimodal decision module. The identity confidence score is calculated according to preset weights. Taking the weighted fusion of iris feature, eye movement feature and environmental features as an example, the weight of iris feature can be set to 60%, the weight of eye movement feature to 30%, and the weight of environmental feature to 10%. First, the iris matching degree, eye movement matching degree and environmental matching degree are calculated separately. Then, the three matching degrees are weighted and summed according to their respective preset weights. That is, identity confidence score = iris matching degree × 60% + eye movement matching degree × 30% + environmental matching degree × 10%. When the identity confidence score reaches the target threshold, the identity of the target object is confirmed to be successful. At this time, the target smart home can be woken up. Then, the personalized settings of the target object for the smart home can be synchronized to the target smart home. For example, after waking up the air conditioner, the temperature can be adjusted to the temperature most often set by the target object. After waking up the TV, it can automatically switch to the channel most often watched by the target object, or automatically open the last watched video and jump to the last exited progress.

[0027] In this embodiment of the application, a smart home gateway can be set up. After successful verification, the system sends a wake-up command to the smart home gateway via the MQTT protocol, and the gateway sends control commands to the corresponding devices respectively.

[0028] Through steps S202 to S208, this application uses multimodal decision-making by integrating multiple eye features and environmental features to replace single iris recognition. It adapts to environmental changes by using environmental features to reduce the impact of environmental interference on recognition, and enhances the anti-spoofing ability of identity verification through multi-feature collaboration. This reduces recognition errors and improves the accuracy of identity verification, thus solving the technical problem that existing iris recognition-based smart home security control solutions still have low recognition accuracy due to environmental changes or spoofing attacks, thereby improving the control security of smart homes.

[0029] In an optional embodiment, step S204, which extracts at least one eye feature based on the eye image, includes: Extracting iris features of the target object based on eye images; and / or extracting eye movement features of the target object based on eye images.

[0030] In this embodiment, iris features are unique, genetically determined, and lifelong stable texture information on the iris surface, manifested as details such as spots, stripes, and filaments within the iris region. They possess extremely high uniqueness and are among the most accurate features in biometrics. Eye movement features are individual-specific characteristics generated during the target object's eye movements, including eye movement trajectories and frequencies. These are difficult to forge and can help improve identity verification security. Furthermore, the directional nature of eye movement trajectories can also be applied to device control.

[0031] The following sections will provide detailed explanations of iris feature extraction and eye movement feature extraction.

[0032] In an optional embodiment, extracting iris features of the target object based on eye images includes: Step S302: Denoise and enhance the eye image to obtain a preprocessed image; Step S304: Locate the iris boundary of the target object in the preprocessed image; Step S306: Extract multidimensional iris feature vectors from multiple directions of the iris boundary; Step S308: Combine the multidimensional iris feature vectors extracted from each direction into a feature matrix, which serves as the iris feature.

[0033] In this embodiment, step S302 first preprocesses the original eye image captured by the infrared camera, such as denoising and enhancement, to obtain a preprocessed image. Denoising can be achieved using a wavelet transform denoising algorithm. Specifically, the original eye image captured by the infrared camera is decomposed into three levels using the db4 wavelet basis to obtain low-frequency coefficients (LL3) and high-frequency coefficients (LH3, HL3, HH3; LH2, HL2, HH2; LH1, HL1, HH1). The low-frequency coefficients reflect the overall contour of the image, while the high-frequency coefficients reflect the details and noise of the image. Soft thresholding is applied to the high-frequency coefficients, including setting high-frequency coefficients with absolute values ​​less than the threshold to 0, and adjusting high-frequency coefficients with absolute values ​​greater than the threshold by subtracting the threshold value from the threshold. The threshold calculation formula is thr = σ × sqrt(2 × log(N)), where σ is the noise standard deviation and N is the number of image pixels. Wavelet reconstruction is then performed on the processed low-frequency and high-frequency coefficients to obtain the denoised eye image. The denoised eye image exhibits improved iris texture clarity. Image enhancement can employ the CLAHE algorithm, with the following steps: Divide the denoised eye image into 8×8 sub-blocks; set a contrast threshold, and when the contrast ratio (c) within a sub-block exceeds this threshold, the excess portion is evenly distributed to other bins; perform histogram equalization on each sub-block individually to adjust the grayscale distribution of pixels within the sub-block, thereby improving contrast; perform bilinear interpolation on the block-processed image to eliminate boundary effects between sub-blocks, resulting in a continuous enhanced image. After denoising and enhancement, the details of the iris texture in the preprocessed image are clearer, facilitating recognition.

[0034] In this embodiment of the application, step S304 further employs an improved Hough transform algorithm to locate the iris boundary of the target object in the preprocessed image.

[0035] In an optional embodiment, step S304, locating the iris boundary of the target object in the preprocessed image, includes: Step 1: Use an edge detection operator to extract edges from the preprocessed image to obtain an edge image marked with edge points and the gradient direction of each edge point; Step 2: In the edge image, the points on the straight line of the gradient direction of each edge point are determined as candidate circle centers; Step 3: Using the distance from the edge point to the candidate circle center as the radius, construct a parameter space with the coordinates of each candidate circle center and the corresponding radius as the circular parameters; Step 4: Traverse each edge point to perform weighted voting on each circle in the parameter space; Step 5: Determine the target circle with the highest number of votes as the iris boundary of the target object.

[0036] In this embodiment, edge detection operators such as Canny and Sobel can be used to extract the edges of iris candidate regions to obtain an edge image. When performing edge detection, the edge detection operator not only outputs edge points but also calculates the gradient direction of each edge point. For an ideal circle, the gradient direction (i.e., the direction of the fastest grayscale change) at a point on its edge always points towards or away from the center. The improved Hough transform algorithm does not blindly vote on all possible combinations of center and radius. For each edge point, its coordinates (x_i, y_i) and gradient direction θ_i are known. Based on the gradient direction θ_i, it can be inferred that the center must lie on a straight line along the gradient direction (or the opposite direction) from that point. This greatly narrows the candidate range of the center (a, b), reducing it from the entire image plane to a straight line, which is key to reducing computational cost. For each possible candidate center point on this straight line, a radius can be calculated based on the distance r from the edge point to the center. Then, voting is performed on this specific parameter set in the parameter space (a, b, r). The "threshold" here can be understood as how many "votes" (i.e., how many edge points support it) a circle needs to receive to be confirmed. Voting is weighted based on the degree to which the gradient direction of the current edge point matches the calculated tangent direction of the circle. If the gradient direction of an edge point exactly points to the calculated center of the circle, then its vote for this candidate circle has higher "weight" or "credibility." If the gradient direction of an edge point deviates significantly from the center direction, its voting weight will decrease, or it may even be ignored. After traversing all edge points, the votes are accumulated in the parameter space. Finally, the circle with the highest weighted votes (a_center, b_center, r_radius) is identified as the iris boundary we are looking for.

[0037] This application employs an improved Hough transform for iris localization, introducing edge gradient direction information and an adaptive threshold mechanism, which significantly improves the localization accuracy of the iris boundary and thus enhances the accuracy of iris feature extraction.

[0038] In this embodiment of the application, after determining the iris boundary in step S306, multidimensional iris feature vectors can be extracted from multiple directions of the iris boundary. For example, starting from the center of the iris, 32-dimensional features of the iris can be extracted from eight directions: up, down, left, right, upper left, lower left, upper right, and lower right.

[0039] In this embodiment of the application, step S308 arranges the 32-dimensional feature vectors in 8 directions in directional order to construct a feature matrix as iris features.

[0040] This application effectively reduces environmental noise interference and improves iris region contrast through a combination of denoising and enhancement processing; it adopts an improved Hough transform localization algorithm to narrow the parameter search range and improve localization speed and accuracy through region filtering and parameter optimization; and it captures individual differences in iris texture more comprehensively through multi-directional feature extraction, thereby improving the distinguishability of iris features.

[0041] In an optional embodiment, extracting eye movement features of the target object based on eye images includes: Step S402: Track the eye movement trajectory of the target object in multiple frames of eye images; Step S404: The eye movement trajectory is split into a first time series signal in the X direction and a second time series signal in the Y direction; Step S406: Perform Fast Fourier Transform on the first time series signal and the second time series signal respectively to obtain the first spectrum in the X direction and the second spectrum in the Y direction; Step S408: Extract multiple frequency components with frequency energy greater than or equal to a preset energy threshold from the first and second spectrograms respectively, and use them as eye-tracking features.

[0042] In this embodiment, the time-series signal refers to a one-dimensional signal that decomposes the eye movement trajectory in chronological order, including an X-direction time-series signal and a Y-direction time-series signal. The X-direction time-series signal reflects horizontal eye movement, and the Y-direction time-series signal reflects vertical eye movement. Fast Fourier Transform (FFT) converts the time-domain time-series signal into a frequency-domain spectral signal to extract the frequency features of eye movement. The spectrogram is an image obtained after FFT processing, with frequency as the horizontal axis and the energy of the frequency components as the vertical axis. It visually displays the contribution of different frequencies to eye movement; the X-direction time-series corresponds to the X-direction spectrogram, and the Y-direction corresponds to the Y-direction spectrogram. A frequency component is a signal component with a specific frequency in the frequency domain. Each frequency component corresponds to a frequency value and an energy value at that frequency, reflecting the intensity of eye movement at that frequency. The preset energy threshold is the energy threshold used to filter effective frequency components and can be set according to actual needs.

[0043] In this embodiment, a series of eye center coordinates can be obtained by tracking eye movement trajectories in multiple frames of eye images. These eye center coordinates at each moment can be decomposed into two time-series signals: a first time-series signal in the X direction and a second time-series signal in the Y direction, representing displacement in the X and Y directions, respectively. Then, a Fast Fourier Transform (FFT) is performed on each of these two time-series signals. The FFT transforms these two time-series signals from the time domain (amplitude varies with time) to the frequency domain (amplitude varies with frequency), generating two spectrograms: a first spectrogram in the X direction and a second spectrogram in the Y direction. Each point in the spectrogram represents a vibrational component of a specific frequency, and its height (energy) indicates how strong this frequency component is in the original signal. Eye movements (such as fixation, smooth tracking, saccades, etc.) are reflected at different frequencies. For example, rapid saccades contain high-frequency components, while slow drifts or smooth tracking contain low-frequency components. Therefore, this application can analyze these two spectrograms and select multiple frequency points with the highest energy (amplitude). These dominant frequency components constitute the core features of eye movements. Ultimately, for a single eye-tracking trajectory sample, taking the extraction of 6 frequency components as an example, 12 values ​​can be obtained: 6 dominant frequency points of the X-direction signal + 6 dominant frequency points of the Y-direction signal, or the signals from the two directions can be combined to obtain 6 most representative composite frequency components.

[0044] This application extracts eye movement features through dynamic tracking and frequency domain analysis, which not only eliminates external interference to ensure feature stability, but also leverages the difficulty in forging the dynamic characteristics of eye movement, complementing iris features and enhancing the system's ability to resist spoofing attacks.

[0045] In an optional embodiment, step S206 involves weighted fusion of at least one eye feature and environmental feature, and calculating the identity confidence of the target object based on the fused features, including: Step 1: Calculate the cosine similarity between the iris features and the preset iris features of the target object to obtain the iris matching degree. The preset iris features are the iris features extracted from the target object in advance and used as the standard. Step 2: Calculate the similarity of each frequency component between the eye movement features and the preset eye movement features of the target object, and calculate the average similarity of each frequency component to obtain the eye movement matching degree. The preset eye movement features are the eye movement features extracted from the target object in advance and used as the standard. Step 3: Obtain the target object's historical control records of the target smart home, extract the target object's environmental preference features from the historical control records, and calculate the similarity between the environmental features and the environmental preference features to obtain the environmental matching degree; Step 4: Calculate a weighted score of at least one of iris matching, eye movement matching, and environmental matching as the identity confidence score.

[0046] In this embodiment, the preset iris features are iris features extracted and stored when the target object initializes the smart home, serving as a standard template for iris matching. The iris matching degree is a quantitative value calculated based on cosine similarity between the iris features and the standard template, ranging from 0 to 1, reflecting the degree of iris identity matching. The preset eye-tracking features are eye-tracking features extracted and stored when the target object initializes the smart home, serving as a standard template for eye-tracking matching. The eye-tracking matching degree is the average similarity between each frequency component of the eye-tracking features and each frequency component of the standard template, ranging from 0 to 1, reflecting the overall matching degree of the eye-tracking features. The target object's historical control records for the target smart home are the target object's past control operation data stored in the system, including control time, device parameters, and environmental data at the time. The environmental preference features are statistical values ​​of the target object's preferred environmental parameters (such as commonly used temperature and humidity) extracted from historical records, forming a feature vector. The environmental matching degree is the similarity between the current environmental features and the environmental preference features, which can be calculated using Manhattan distance normalization, ranging from 0 to 1.

[0047] In this embodiment of the application, the preset eye movement features of the target object may include multiple features, which are stored in an eye movement feature template library unique to the target object. This eye movement feature template library is pre-established during the user registration phase and can be established in the following manner: Step 1, Data Acquisition. During user registration, users are required to perform a series of standardized eye-tracking tasks. For example, fixation tasks, smooth tracking tasks, etc., and the eye movement trajectories of the user during each eye-tracking task are collected.

[0048] Step 2, Feature Extraction. The acquired multiple eye-tracking trajectories are processed by trajectory tracking, time-series signal splitting, fast Fourier transform, and extraction of the dominant frequency component to obtain eye-tracking features corresponding to different eye-tracking tasks.

[0049] Step 3 involves statistically analyzing all eye-tracking features extracted from a user (e.g., calculating the mean and variance) to create a more stable and representative comprehensive template. This final template, representing the user's unique eye-tracking pattern, is then associated with the user's identity ID (such as name), iris template, and other information, and stored in a database to form a user-specific eye-tracking feature template library.

[0050] In this embodiment, the cosine similarity between iris features and preset iris features of the target object can be calculated to obtain the iris matching degree. The similarity between eye movement features and each frequency component of preset eye movement features of the target object can be calculated, and then the average similarity of each frequency component is calculated to obtain the eye movement matching degree. Historical control records of the target object for the past three months can be obtained from the system database, commonly used environmental parameters can be statistically analyzed to form an environmental preference feature vector, and the cosine similarity between the feature vector of the environmental features and the environmental preference feature vector can be calculated to obtain the environmental matching degree. Finally, the identity confidence score is calculated as: Iris matching degree × 60% + Eye movement matching degree × 30% + Environmental matching degree × 10%. If only one eye feature and environmental feature are fused, such as a weighted fusion of iris features and environmental features, the weight of the iris feature can be set to 80%, and the weight of the environmental feature to 20%. Alternatively, if a weighted fusion of eye movement features and environmental features is performed, the weight of the eye movement feature can be set to 70%, and the weight of the environmental feature to 30%. The weights of each parameter can be set according to actual needs.

[0051] This application calculates identity confidence through multi-feature weighted fusion, which not only uses iris features to ensure basic recognition accuracy, but also uses eye-tracking features to improve anti-forgery capabilities, and combines environmental preference features to adapt to scene changes, thereby improving identity verification accuracy and effectively solving the problem that single features are easily interfered with and deceived.

[0052] In this embodiment, identity recognition is performed by combining environmental feature parameters. This enables the determination of when and where to provide what service to what type of user, based on their identity. In other words, by integrating biometrics and environmental features, the recognition can be improved from a single identity recognition to a joint judgment of identity and intent. The recognition result answers the question "Who are you?" and, based on your past habits and current environment, predicts the intent of "What might you want to do now?" This provides another dimension of liveness detection information and enhances the control security of smart homes.

[0053] In an optional embodiment, after step S208 wakes up the target smart home device, the method further includes: Step 1: Determine the target action to be performed by the smart home device based on the target object's eye movement trajectory; Step 2: Send the target control command to the target smart home device so that the target smart home device can perform the target action.

[0054] In this embodiment, a mapping rule between eye movement trajectory and target action can be preset. For example, for curtains, the system can control the opening and closing of curtains if the user looks at the direction of the curtains and sweeps them left and right. For televisions, the system can control the TV channel to increase by 1 if the pupil center moves from left to right within 1 second, and control the TV channel to decrease by 1 if it moves from right to left. For air conditioners, the system can control the air conditioner temperature to increase by 1℃ if the pupil center moves from top to bottom within 1 second, and control the air conditioner temperature to decrease by 1℃ if it moves from bottom to top. Within a program, the system can be set to adjust the light brightness if the pupil center stays in a certain area of ​​the screen, such as the light icon, for ≥2 seconds and blinks to confirm.

[0055] In this embodiment, the system analyzes the eye movement trajectory in real time to determine whether it meets the above mapping rules. If it does, the corresponding target action is determined. The system generates a target control command based on the target action and sends the control command to the corresponding device via the MQTT protocol. After receiving the command, the device parses and executes the action, and the execution result is fed back to the system.

[0056] This application transforms eye movement trajectories into smart home control commands, enabling contactless and precise control that eliminates the need for manual operation and enhances convenience. Simultaneously, it triggers control based on verified identities, preventing unauthorized user operations. This ensures security while enriching the interactive methods of smart home control and improving the user experience.

[0057] In this embodiment, iris recognition and eye-tracking data can be deeply integrated with environmental sensor data and device status data to construct a whole-house smart decision-making model. For example, when a user is looking at a living room window and the system detects that the indoor temperature is too high, it can automatically turn on the air conditioner and adjust it to energy-saving mode through comprehensive analysis of multimodal data. In addition, the system can dynamically adjust the smart device interface based on the user's iris recognition results and gaze direction. For example, when a user approaches a smart tablet, after recognizing the iris, the system can display frequently used applications in a prominent position on the interface based on the user's past operating habits.

[0058] The following is combined Figure 3 The process of the smart home control method provided in this application will be fully described below.

[0059] like Figure 3As shown, a TOF sensor first monitors human activity within a 3-meter range. When a target object enters this range, a trigger signal activates an infrared camera, which begins acquiring eye images of the target object in a preset mode. The eye images acquired by the infrared camera are simultaneously fed into two parallel modules: iris processing and eye movement analysis. The eye movement analysis module tracks the target object's eye movements at a rate of 30 frames per second, generating eye movement trajectory data. The iris processing module sequentially performs image enhancement, iris localization, and feature extraction. First, image enhancement technology optimizes the quality of the eye images; then, a specific algorithm completes iris localization; finally, Fourier transform is used to generate iris frequency features, and a 256-dimensional iris feature code is extracted. The iris features and eye movement trajectory data obtained from the above processing are input into a multimodal fusion module along with the acquired environmental parameters. This module uses preset weighting rules for fusion calculation, with iris features accounting for 60%, eye movement trajectory for 30%, and environmental parameters for 10%. A weighted calculation yields the target object's comprehensive score. The system uses a threshold to determine the overall score obtained from multimodal fusion. If the overall score is greater than 0.95, the verification is considered successful, triggering subsequent smart home linkage operations. If the overall score is less than or equal to 0.95, the verification fails, and the process terminates. After successful verification, the system implements smart home control in two ways: first, waking up devices, such as triggering personalized functions like TV playback resume; second, whole-house control, such as adjusting the air conditioner temperature based on environmental parameters and user preferences. The control commands are sent to the corresponding smart home devices via the MQTT protocol to complete the final control operation.

[0060] This application provides a smart home control method, comprising: when a target object is detected entering the user detection range of a target smart home, collecting environmental data of the space where the target object is located and an eye image of the target object; extracting at least one eye feature based on the eye image and extracting environmental features based on the environmental data; weightedly fusing the at least one eye feature and the environmental features, and calculating the identity confidence score of the target object based on the fused features; and waking up the target smart home when the identity confidence score reaches a target threshold, and synchronizing the target smart home with the target object's personalized settings. This application uses multimodal decision-making by fusing multiple eye features and environmental features to replace single iris recognition. It adapts to environmental changes through environmental features to reduce the impact of environmental interference on recognition, and enhances the anti-spoofing capability of identity verification through multi-feature collaboration, thereby reducing recognition errors and improving identity verification accuracy. This solves the technical problem that existing iris recognition-based smart home security control schemes still suffer from low recognition accuracy due to environmental changes or spoofing attacks, thus improving the control security of smart homes.

[0061] According to another aspect of the embodiments of this application, such as Figure 4 As shown, a smart home control device is provided, comprising: Data acquisition module 401 is used to collect environmental data of the space where the target object is located and eye images of the target object when the target object is detected to have entered the user detection range of the target smart home. The feature extraction module 403 is used to extract at least one eye feature based on the eye image and to extract environmental features based on environmental data. The multimodal decision module 405 is used to weightedly fuse at least one eye feature and environmental feature, and calculate the identity confidence of the target object based on the fused feature; The device control module 407 is used to wake up the target smart home when the identity confidence level reaches the target threshold, and synchronize the target smart home with the personalized settings of the target object.

[0062] It should be noted that the data acquisition module 401 in this embodiment can be used to execute step S202 in this application embodiment, the feature extraction module 403 in this embodiment can be used to execute step S204 in this application embodiment, the multimodal decision module 405 in this embodiment can be used to execute step S206 in this application embodiment, and the device control module 407 in this embodiment can be used to execute step S208 in this application embodiment.

[0063] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can be implemented in software or hardware.

[0064] Optionally, the feature extraction module is specifically used to: extract iris features of the target object based on the eye image; and / or, extract eye movement features of the target object based on the eye image.

[0065] Optionally, the feature extraction module is also used to: denoise and enhance the eye image to obtain a preprocessed image; locate the iris boundary of the target object in the preprocessed image; extract multidimensional iris feature vectors from multiple directions of the iris boundary; and combine the multidimensional iris feature vectors extracted from each direction into a feature matrix as iris features.

[0066] Optionally, the feature extraction module is further configured to: extract edges from the preprocessed image using an edge detection operator to obtain an edge image marked with edge points and the gradient direction of each edge point; in the edge image, determine the points on the straight line of the gradient direction of each edge point as candidate circle centers; construct a parameter space with the coordinates of each candidate circle center and the corresponding radius as circle parameters, using the distance from the edge point to the candidate circle center as the radius; traverse each edge point to perform weighted voting on each circle in the parameter space; and determine the target circle with the highest number of votes as the iris boundary of the target object.

[0067] Optionally, the feature extraction module is further configured to: track the eye movement trajectory of the target object in multiple frames of eye images; split the eye movement trajectory into a first time-series signal in the X direction and a second time-series signal in the Y direction; perform fast Fourier transform on the first time-series signal and the second time-series signal respectively to obtain a first spectrum map in the X direction and a second spectrum map in the Y direction; and extract multiple frequency components with frequency energy greater than or equal to a preset energy threshold from the first spectrum map and the second spectrum map respectively as eye movement features.

[0068] Optionally, the multimodal decision module is specifically used for: calculating the cosine similarity between iris features and preset iris features of the target object to obtain iris matching degree, wherein the preset iris features are iris features extracted in advance from the target object and used as standards; calculating the similarity between eye movement features and each frequency component in the preset eye movement features of the target object, and calculating the average of the similarity of each frequency component to obtain eye movement matching degree, wherein the preset eye movement features are eye movement features extracted in advance from the target object and used as standards; acquiring the target object's historical control records of the target smart home, extracting the target object's environmental preference features from the historical control records, and calculating the similarity between the environmental features and the environmental preference features to obtain environmental matching degree; and calculating a weighted score of at least one of the iris matching degree, eye movement matching degree, and environmental matching degree as identity confidence.

[0069] Optionally, the device control module is further configured to: determine the target action to be performed by the target smart home based on the eye movement trajectory of the target object; and send a target control command to the target smart home to cause the target smart home to perform the target action.

[0070] According to another aspect of the embodiments of this application, this application provides an electronic device, such as... Figure 5 As shown, the system includes a memory 501, a processor 503, a communication interface 505, and a communication bus 507. The memory 501 stores a computer program that can run on the processor 503. The memory 501 and the processor 503 communicate through the communication interface 505 and the communication bus 507. When the processor 503 executes the computer program, it implements the steps of the above method.

[0071] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0072] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0073] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0074] According to another aspect of the embodiments of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above embodiments.

[0075] Optionally, in embodiments of this application, the computer-readable medium is configured to store program code for the processor to perform the following steps: When a target object is detected entering the user detection range of the target smart home, environmental data of the space where the target object is located and eye images of the target object are collected; At least one eye feature is extracted from eye images, and environmental features are extracted from environmental data. At least one eye feature and environmental feature are weighted and fused, and the identity confidence of the target object is calculated based on the fused feature; When the identity confidence level reaches the target threshold, the target smart home device is activated and synchronized with the target object's personalized settings.

[0076] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0077] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0078] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0079] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0080] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0081] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0082] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0085] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A smart home control method, characterized in that, include: When a target object is detected entering the user detection range of the target smart home, environmental data of the space where the target object is located and eye images of the target object are collected; At least one eye feature is extracted based on the eye image, and environmental features are extracted based on the environmental data; At least one of the eye features and the environmental features are weighted and fused, and the identity confidence of the target object is calculated based on the fused features; When the identity confidence level reaches the target threshold, the target smart home device is activated, and the target smart home device is synchronized with the personalized settings of the target object.

2. The method according to claim 1, characterized in that, The extraction of at least one eye feature based on the eye image includes: Extract the iris features of the target object based on the eye image; and / or, The eye movement features of the target object are extracted based on the eye image.

3. The method according to claim 2, characterized in that, The step of extracting the iris features of the target object based on the eye image includes: The eye image is denoised and enhanced to obtain a preprocessed image; Locate the iris boundary of the target object in the preprocessed image; Multidimensional iris feature vectors are extracted from multiple directions of the iris boundary; The multidimensional iris feature vectors extracted from each direction are combined into a feature matrix, which is used as the iris feature.

4. The method according to claim 3, characterized in that, The step of locating the iris boundary of the target object in the preprocessed image includes: Edges in the preprocessed image are extracted using an edge detection operator to obtain an edge image marked with edge points and the gradient direction of each edge point; In the edge image, the points on the straight line of the gradient direction of each edge point are determined as candidate circle centers; Using the distance from the edge point to the candidate circle center as the radius, a parameter space is constructed with the coordinates of each candidate circle center and the corresponding radius as the circular parameters; Traverse each of the stated edge points to perform weighted voting on each circle in the parameter space; The target circle with the highest number of votes is determined as the iris boundary of the target object.

5. The method according to claim 2, characterized in that, The step of extracting the eye movement features of the target object based on the eye image includes: In multiple frames of the eye images, the eye movement trajectory of the target object is tracked; The eye movement trajectory is divided into a first time-series signal in the X direction and a second time-series signal in the Y direction; Perform Fast Fourier Transform on the first time series signal and the second time series signal respectively to obtain a first spectrum in the X direction and a second spectrum in the Y direction; Multiple frequency components with frequency energy greater than or equal to a preset energy threshold are extracted from the first and second spectrograms respectively, and used as the eye movement features.

6. The method according to claim 2, characterized in that, The step of weightedly fusing at least one of the eye features and the environmental features, and calculating the identity confidence of the target object based on the fused features, includes: The cosine similarity between the iris feature and the preset iris feature of the target object is calculated to obtain the iris matching degree, wherein the preset iris feature is the iris feature extracted in advance from the target object and used as a standard; Calculate the similarity between the eye movement features and the preset eye movement features of the target object for each frequency component, and calculate the average similarity of each frequency component to obtain the eye movement matching degree. The preset eye movement features are eye movement features extracted from the target object in advance and used as a standard. The system obtains the target object's historical control records of the target smart home, extracts the target object's environmental preference features from the historical control records, and calculates the similarity between the environmental features and the environmental preference features to obtain the environmental matching degree. Calculate a weighted score of at least one of the iris matching degree, the eye movement matching degree, and the environmental matching degree, as the identity confidence score.

7. The method according to any one of claims 1 to 6, characterized in that, After waking up the target smart home device, the method further includes: The target action to be performed by the target smart home device is determined based on the eye movement trajectory of the target object. Send a target control command to the target smart home device so that the target smart home device performs the target action.

8. A smart home control device, characterized in that, include: The data acquisition module is used to collect environmental data of the space where the target object is located and eye images of the target object when the target object is detected to have entered the user detection range of the target smart home. The feature extraction module is used to extract at least one eye feature based on the eye image and to extract environmental features based on the environmental data. A multimodal decision module is used to perform weighted fusion of at least one of the eye features and the environmental features, and calculate the identity confidence of the target object based on the fused features; The device control module is used to wake up the target smart home device when the identity confidence level reaches the target threshold, and to synchronize the target smart home device with the personalized settings of the target object.

9. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program executable on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that, When the processor executes the computer program, it implements the smart home control method according to any one of claims 1 to 7.

10. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the smart home control method according to any one of claims 1 to 7.

Images