A face detection method and system for intelligent digital safety refrigerator

Abstract

The present application relates to the field of data processing technology, and discloses a method and system for face detection in an intelligent digital safety refrigerator. The method comprises: performing face detection based on multi-level sliding window scanning on a standardized image to obtain a face bounding box and coordinates of five facial key points; performing region segmentation on the standardized image to obtain a target face region and perform multi-angle sample generation and feature extraction to obtain a high-dimensional face feature vector; performing similarity calculation on the high-dimensional face feature vector, and performing liveness detection through a multimodal information library to obtain a verification token containing a user ID, a confidence score, and a timestamp; performing access permission analysis on the verification token to obtain an access permission table; based on the access permission table, performing temporal feature analysis on the real-time collected user refrigerator interaction behavior sequence to obtain abnormal behavior data and refrigerator safety risk level. The present application improves the efficiency and accuracy of face detection in intelligent digital safety refrigerators.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and system for face detection in an intelligent digital safety refrigerator. Background Art

[0002] In recent years, with the rapid development of the Internet of Things and artificial intelligence technologies, a new type of product has emerged in the field of smart home appliances - smart digital safety refrigerators. This type of refrigerator not only has traditional food storage and temperature regulation functions, but also integrates advanced sensor systems, face recognition technology and intelligent analysis algorithms. Some high-end smart refrigerators on the market are already able to realize functions such as user identity recognition, personalized recipe recommendations, and food management. Some systems even try to combine users' physiological cycles and eating habits to provide health advice and nutritional analysis.

[0003] Existing smart digital security refrigerators still have many challenges and shortcomings in face detection and identity authentication. The environment in front of the refrigerator door is complex and changeable, including rapid changes in light intensity (such as opening the door at night), airflow disturbances caused by temperature gradients, and image distortion caused by door frame reflections. These factors will significantly affect the quality of facial images. Traditional face detection algorithms often have difficulty coping with refrigerator-specific usage scenarios, such as users may approach the refrigerator at various angles and distances, or move quickly during the door opening process. Existing systems usually separate face recognition and user behavior analysis, lacking a systematic analysis of the entire process of user interaction with the refrigerator, making it difficult to accurately identify abnormal behaviors and potential security risks. Finally, existing technologies rarely consider permission management and privacy protection issues in multi-user scenarios in the home, and cannot meet the differentiated needs of different family members for refrigerator use. Summary of the invention

[0004] The present application provides a method and system for face detection in an intelligent digital safety refrigerator, which are used to improve the efficiency and accuracy of face detection in an intelligent digital safety refrigerator.

[0005] In a first aspect, the present application provides a method for face detection in an intelligent digital security refrigerator, the method comprising: performing adaptive histogram equalization and geometric correction on a high frame rate image sequence collected from the door front area of ​​a target refrigerator to obtain a standardized image; performing face detection based on multi-level sliding window scanning on the standardized image to obtain a face bounding box and five facial key point coordinates; performing region segmentation on the standardized image based on the face bounding box and the five facial key point coordinates to obtain a target face region, and performing multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector; performing similarity calculation on the high-dimensional face feature vector, and performing liveness detection through a preset multimodal information library to obtain a verification token including a user ID, a confidence score and a timestamp; performing access permission analysis on the verification token through pre-collected user physiological cycle data and family member interaction patterns to obtain an access permission table; based on the access permission table, performing temporal feature analysis on the real-time collected user refrigerator interaction behavior sequence to obtain abnormal behavior data and refrigerator safety risk level.

[0006] In combination with the first aspect, in a first implementation of the first aspect of the present application, the high frame rate image sequence collected from the door front area of ​​the target refrigerator is subjected to adaptive histogram equalization and geometric correction to obtain a standardized image, including: adaptively sampling the refrigerator door opening frequency of the high frame rate image sequence to obtain a dynamic key frame set, and eliminating the light interference of the refrigerator compartment on the dynamic key frame set to obtain an initial denoised image set; applying a refrigerator door inside and outside temperature difference compensation algorithm to the initial denoised image set to perform fog correction to obtain a clarity improved image, and performing refrigerator door frame surface distortion calibration on the clarity improved image, A geometrically corrected image is obtained; the geometrically corrected image is compensated for the influence of frost in the freezer to obtain a contrast enhanced image, and the contrast enhanced image is processed to eliminate the shadow of the refrigerator door handle to obtain a shadow balanced image; the shadow balanced image is subjected to color interference suppression of food stored in the refrigerator to obtain a color pure image, and the color pure image is subjected to edge enhancement with refrigerator feature preservation to obtain an edge sharpened image; the edge sharpened image is subjected to adaptive correction for refrigerator lighting non-uniformity to obtain a lighting balanced image; the lighting balanced image is subjected to adaptive cropping for refrigerator usage scenarios to obtain the standardized image.

[0007] In combination with the first aspect, in a second implementation of the first aspect of the present application, the face detection based on multi-level sliding window scanning on the standardized image to obtain a face bounding box and coordinates of five facial key points includes: adaptively scaling the standardized image according to the opening angle of the refrigerator door to obtain a multi-scale image pyramid, and compensating the multi-scale image pyramid for changes in refrigerator ambient light to obtain an illumination invariant feature map; shielding the refrigerator door handle area on the illumination invariant feature map to obtain an optimized candidate area, and performing sliding window scanning weighted by refrigerator usage frequency on the optimized candidate area to obtain a preliminary face position estimate; The preliminary face position estimation is subjected to the elimination of the interference of the refrigerator door frame reflection to obtain a fine face area, and the fine face area is subjected to the refrigerator internal temperature gradient correction to obtain a temperature-balanced face image; the temperature-balanced face image is subjected to the refrigerator door opening and closing speed adaptive boundary adjustment to obtain a dynamic boundary frame, and the dynamic boundary frame is subjected to the refrigerator stored food color interference suppression to obtain a pure face area; the pure face area is subjected to the refrigerator-specific light source shadow compensation to obtain a shadow-balanced face image, and the refrigerator user height adaptive key point positioning is performed on the shadow-balanced face image to obtain the face boundary frame and the coordinates of the five facial key points.

[0008] In combination with the first aspect, in a third implementation of the first aspect of the present application, the standardized image is segmented based on the face bounding box and the coordinates of the five facial key points to obtain a target face area, and multi-angle sample generation and feature extraction are performed on the target face area to obtain a high-dimensional face feature vector, including: adaptively adjusting the opening and closing angle of the refrigerator door of the face bounding box to obtain a dynamic face area, and eliminating the reflection of the light source inside the refrigerator for the dynamic face area to obtain a de-reflected face image; compensating the de-reflected face image for refrigerator temperature changes to obtain a temperature-balanced face image, and performing refrigerator storage on the temperature-balanced face image. The method corrects the interference of stored food odor to obtain a pure face area; performs multi-angle rotation on the pure face area weighted by the frequency of refrigerator use to obtain a multi-view face sample set, and performs refrigerator door handle occlusion simulation on the multi-view face sample set to obtain occlusion enhanced samples; performs refrigerator internal temperature gradient feature enhancement on the occlusion enhanced samples to obtain a temperature feature map, and performs refrigerator door opening frequency adaptive feature extraction on the temperature feature map to obtain a time-varying feature set; performs refrigerator user habit correlation analysis on the time-varying feature set to obtain a user feature map, and performs refrigerator environmental noise robustness enhancement on the user feature map to obtain the high-dimensional face feature vector.

[0009] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, the high-dimensional facial feature vector is subjected to similarity calculation, and liveness detection is performed through a preset multimodal information library to obtain a verification token containing a user ID, a confidence score and a timestamp, including: weighting the high-dimensional facial feature vector by the frequency of refrigerator door opening to obtain a time series enhancement feature, and compensating the time series enhancement feature for the temperature change inside the refrigerator to obtain a temperature invariant feature; compensating the temperature invariant feature for the airflow disturbance caused by the opening and closing of the refrigerator door to obtain a stable facial feature, and performing a correlation analysis on the stable facial feature during the refrigerator usage period to obtain a time-sensitive feature; The time-sensitive features are calculated for similarity through the additive angular marginal loss function to obtain an initial matching result; the initial matching result is verified by the refrigerator door handle contact pattern to obtain a tactile enhancement matching score, and the tactile enhancement matching score is adaptively adjusted to the refrigerator light change to obtain a lighting robust matching value; the lighting robust matching value is checked for consistency with the refrigerator usage habits to obtain a behavior verification result; the behavior verification result is compensated for the change of humidity inside the refrigerator to obtain the user ID, and the user ID is fused with the refrigerator door opening force feature to obtain the verification token, wherein the verification token includes: user ID, confidence score and timestamp.

[0010] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, the access permission analysis is performed on the verification token through the pre-collected user physiological cycle data and family member interaction mode to obtain the access permission table, including: performing refrigerator door opening time period distribution statistics on the verification token to obtain user time preference data, and performing refrigerator storage area access frequency correlation analysis on the user time preference data to obtain a regional access tendency map; performing refrigerator internal temperature change correlation calculation on the regional access tendency map to obtain temperature control habit characteristics, and matching the temperature control habit characteristics with refrigerator energy consumption data to obtain a user energy-saving tendency index; performing refrigerator food storage cycle correlation analysis on the user energy-saving tendency index to obtain a dietary regularity characteristic; weighting the dietary regularity characteristic by the refrigerator door opening and closing frequency to obtain user life and work schedule data, and performing refrigerator internal light change pattern recognition on the user life and work schedule data to obtain a day and night activity characteristic; The refrigerator defrost cycle synchronization analysis is performed on the diurnal activity characteristics to obtain the user's home time estimation data, and the refrigerator peak usage overlap is calculated for the user's home time estimation data to obtain a family member interaction intensity map; the refrigerator intelligent scene pattern matching is performed on the family member interaction intensity map to obtain the access permission table.

[0011] In combination with the first aspect, in a sixth implementation of the first aspect of the present application, based on the access permission table, a time series feature analysis is performed on the real-time collected user refrigerator interaction behavior sequence to obtain abnormal behavior data and the refrigerator safety risk level, including: performing a refrigerator door switch frequency analysis on the user refrigerator interaction behavior sequence to obtain a door opening mode feature, and performing a refrigerator storage area temperature fluctuation correlation calculation on the door opening mode feature to obtain a temperature change trajectory; matching the temperature change trajectory with the refrigerator energy consumption peak time period to obtain an energy consumption behavior feature, and performing refrigerator lighting usage time statistics on the energy consumption behavior feature to obtain lighting demand data ; Analyze the change trend of humidity inside the refrigerator on the light demand data to obtain a storage environment index; calculate the refrigerator defrost cycle deviation on the storage environment index to obtain a maintenance demand warning, and perform a refrigerator food access sequence association analysis on the maintenance demand warning to obtain user habit characteristics; detect the change of contact pressure of the refrigerator door handle on the user habit characteristics to obtain operation force data; analyze the value fluctuation of the refrigerator internal odor sensor on the operation force data to obtain a storage abnormality index, and compare the storage abnormality index with the access permission table to the refrigerator safety threshold to obtain the abnormal behavior data and the refrigerator safety risk level.

[0012] In a second aspect, the present application provides an intelligent digital safety refrigerator face detection system, the intelligent digital safety refrigerator face detection system comprising:

[0013] A correction module, used for performing adaptive histogram equalization and geometric correction on a high frame rate image sequence collected from the door front area of ​​the target refrigerator to obtain a standardized image;

[0014] A detection module, used for performing face detection on the standardized image based on multi-level sliding window scanning to obtain a face bounding box and coordinates of five facial key points;

[0015] A segmentation module, used for performing regional segmentation on the standardized image based on the face bounding box and the coordinates of the five facial key points to obtain a target face region, and performing multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector;

[0016] A calculation module, used to perform similarity calculation on the high-dimensional facial feature vector, and perform liveness detection through a preset multimodal information library to obtain a verification token including a user ID, a confidence score and a timestamp;

[0017] An analysis module, configured to perform access permission analysis on the verification token based on pre-collected user physiological cycle data and family member interaction patterns, and obtain an access permission table;

[0018] The early warning module is used to perform time series feature analysis on the user refrigerator interaction behavior sequence collected in real time based on the access permission table to obtain abnormal behavior data and refrigerator safety risk level.

[0019] A third aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned intelligent digital safety refrigerator face detection method.

[0020] In the technical solution provided by the present application, by performing adaptive histogram equalization and geometric correction on the high-frame rate image sequence collected in the area in front of the refrigerator door, the problems of unstable light and image distortion in the refrigerator use environment are effectively solved, and the accuracy and robustness of subsequent face detection are significantly improved; secondly, a face detection method based on multi-level sliding window scanning is adopted, which can accurately locate the face boundary box and facial key points, laying the foundation for subsequent feature extraction; furthermore, by performing multi-angle sample generation and feature extraction on the target face area, the high-dimensional face feature vector obtained captures the user's facial features more comprehensively and enhances the reliability of identity authentication; and liveness detection is performed through a preset multimodal information library, which effectively prevents impersonation and deception and improves the security of the system; by introducing user physiological cycle data and family member interaction mode for access permission analysis, the method realizes more refined and personalized permission management and meets the differentiated needs of different family members for refrigerator use; finally, the real-time collected user refrigerator interaction behavior sequence is subjected to time series feature analysis to obtain abnormal behavior data and refrigerator safety risk level, which greatly improves the safety performance of smart refrigerators. The present invention can not only accurately identify abnormal usage behaviors, but also timely warn of potential safety risks, such as children's accidental ingestion and food spoilage, providing new protection for family safety. In short, by comprehensively considering the unique usage environment and user behavior characteristics of refrigerators, high-precision identity authentication and intelligent authority management are achieved, which improves the efficiency and accuracy of face detection in smart digital safety refrigerators. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 This is a schematic diagram of an embodiment of the face detection method of the intelligent digital safety refrigerator in the embodiment of the present application;

[0023] Figure 2This is a schematic diagram of an embodiment of the smart digital safety refrigerator face detection system in the embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiment of the present application provides a method and system for face detection of an intelligent digital safety refrigerator. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the face detection method of the intelligent digital safety refrigerator in the embodiment of the present application includes:

[0026] Step S101, performing adaptive histogram equalization and geometric correction on a high frame rate image sequence collected from the door front area of ​​a target refrigerator to obtain a standardized image;

[0027] It is understandable that the execution subject of the present application may be the intelligent digital safety refrigerator face detection system, or may be a terminal or a server, which is not limited here. The present application embodiment is described by taking the server as the execution subject as an example.

[0028] Specifically, the step of performing adaptive histogram equalization and geometric correction on the high frame rate image sequence collected in front of the door area of ​​the target refrigerator to obtain a standardized image is the basis of the face detection method for intelligent digital security refrigerators. The acquisition of high frame rate image sequences is achieved by installing a high-performance camera in front of the refrigerator door. The acquisition frame rate is usually not less than 60fps to capture the user's rapid movement. Adaptive histogram equalization is an image enhancement technology used to improve the contrast and brightness distribution of the image. In the refrigerator environment, due to the complex and changeable light conditions, such as the influence of factors such as the kitchen ceiling light, the internal lighting of the refrigerator, and the natural light outside the window, the collected original image may have the problem of being too dark or too bright in some areas. Adaptive histogram equalization divides the image into multiple small blocks, performs histogram equalization on each small block separately, and then uses bilinear interpolation to merge the processing results, thereby effectively enhancing the image details and avoiding the problem of over-enhancement that may be caused by traditional global histogram equalization. It can make the images collected under different light conditions have good contrast and clarity, providing high-quality input for subsequent face detection. Geometric correction is to solve the problem of image distortion. Due to the installation position and angle restrictions of the camera in front of the refrigerator door, the original image collected may have perspective deformation or lens distortion. Geometric correction corrects these distortions by performing affine transformation or nonlinear transformation on the image through pre-calibrated camera parameters, so that the straight lines in the image remain straight and the face shape will not be deformed due to the shooting angle. This step ensures the accuracy of subsequent face detection and feature extraction. Through the processing of adaptive histogram equalization and geometric correction, the original high frame rate image sequence is converted into standardized images with balanced brightness distribution, enhanced details and correct geometric shapes, which provide ideal input for subsequent face detection and recognition.

[0029] For example, when a user opens the refrigerator door at night, the original image may be severely underexposed or overexposed due to the sudden change in light. After adaptive histogram equalization, the brightness distribution of the image is optimized and the user's facial features become clearly identifiable. At the same time, if the user approaches the refrigerator from the side, causing perspective distortion in the image, the geometric correction step will correct this distortion to ensure that the face maintains the correct proportion and shape in the image.

[0030] Step S102, performing face detection based on multi-level sliding window scanning on the standardized image to obtain a face bounding box and coordinates of five facial key points;

[0031] Specifically, face detection based on multi-level sliding window scanning is performed on standardized images to obtain the face bounding box and the coordinates of five facial key points, which is one of the core steps of the face detection method for intelligent digital security refrigerators. Multi-level sliding window scanning is a commonly used target detection technology that locates faces by sliding detection windows at different scales and positions on the image. In this method, considering the particularity of refrigerator usage scenarios, the scale and step size of the sliding window need to be optimized according to the size of the refrigerator door and the possible distance range of the user.

[0032] An image pyramid is constructed for the standardized image, which usually contains 3 to 5 scale layers, and the size of each layer is 1.2 to 1.5 times smaller than the previous layer. In this way, faces of different sizes can be detected to adapt to the different distances between the user and the refrigerator. Then, a fixed-size sliding window (such as 24x24 pixels) is used for scanning at each scale layer. The sliding step size is usually set to 10% to 25% of the window size. In the refrigerator scenario, the step size can be appropriately increased to increase the detection speed. At each sliding window position, a pre-trained face detector (such as a Haar cascade classifier or a convolutional neural network) is used to determine whether a face is contained. In order to adapt to the refrigerator environment, the detector needs to be trained with a large number of face images collected in the refrigerator scenario to improve the detection accuracy under specific lighting conditions and backgrounds. For the detected face candidate area, the non-maximum suppression (NMS) algorithm is used to merge the overlapping detection boxes to obtain the final face bounding box. The NMS algorithm calculates the intersection over union (IoU) of overlapping detection boxes, retains the detection box with the highest confidence, and suppresses other detection boxes with an overlap degree higher than a threshold (usually 0.5). After obtaining the face bounding box, shape regression or deep learning methods are used to locate the five facial key points (eyes, nose tip, and mouth corners). The coordinates of these key points are crucial for subsequent face alignment and feature extraction. In the refrigerator scenario, the key point positioning algorithm needs to consider possible occlusions (such as users wearing masks or being partially blocked by refrigerator doors) and improve robustness by increasing the training of occluded samples.

[0033] For example, suppose a user opens the refrigerator door at night to get some water. The image size after normalization is 640x480 pixels, and a 3-layer image pyramid is constructed with a minimum detection window of 24x24 pixels. A sliding window of 48x48 is used with a step size of 12 pixels at the top layer (320x240 pixels); a window of 32x32 is used with a step size of 8 pixels at the middle layer (426x320 pixels); and a window of 24x24 is used with a step size of 6 pixels at the bottom layer (original size). With this multi-scale scanning, faces can be accurately detected even when the user is far or close to the refrigerator. The detector finds a face candidate with a high confidence score (0.95) at the coordinates (213, 160) in the middle layer, which is 96x96 pixels in size. After NMS processing, the final face bounding box is determined. Subsequently, the keypoint localization algorithm successfully locates five facial keypoints within this bounding box, maintaining high accuracy even under the special lighting conditions of the refrigerator interior.

[0034] Step S103, performing region segmentation on the standardized image based on the face bounding box and the coordinates of the five facial key points to obtain a target face region, and performing multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector;

[0035] Specifically, the standardized image is segmented based on the face bounding box and the coordinates of the five facial key points to obtain the target face region, and multi-angle sample generation and feature extraction are performed on the target face region to obtain a high-dimensional face feature vector, which is a key step in the face detection method of the intelligent digital security refrigerator. First, the standardized image is cropped using the face bounding box to obtain a preliminary face region. Then, the face is aligned based on the five facial key points (eyes, nose tip and mouth corners), and the face is rotated and scaled to a standard posture through affine transformation. This process ensures the consistency of subsequent feature extraction. The aligned face image is further cropped and scaled to a fixed size (such as 112x112 pixels) to obtain the final target face region. Multi-angle sample generation is to enhance the robustness of feature extraction. In the refrigerator usage scenario, users may approach the refrigerator at different angles, so various posture changes need to be considered. Multiple variant samples are generated by rotating the target face region by a small angle (such as ±5°, ±10°), slightly translating and scaling. These samples also include simulation of different lighting conditions (such as refrigerator interior lighting on / off) and possible occlusion situations (such as wearing a mask).

[0036] Feature extraction uses deep convolutional neural networks, such as ResNet or MobileNet architecture. The network is pre-trained with a large-scale face dataset and fine-tuned using face images collected in the refrigerator scene to adapt to the specific environment. The input of the network is the generated multi-angle samples, and the output is a high-dimensional feature vector (such as 512 dimensions). After extracting features from each sample, the final face feature vector is obtained by fusion through weighted averaging or maximum pooling.

[0037] For example, suppose that at 9 pm, a user opens the refrigerator door to take a drink. Based on the face bounding box obtained in the previous step (coordinates (100, 150, 200, 250)) and the coordinates of the five key points (left eye (120, 180), right eye (180,180), nose tip (150, 210), left corner of mouth (130, 230), right corner of mouth (170, 230)), the standardized image is cropped and aligned. The aligned face image is scaled to 112x112 pixels. Then, 5 samples with different angles (0°, ±5°, ±10°) are generated, and 2 lighting conditions (refrigerator internal lighting on and off) are simulated, resulting in a total of 10 samples. These samples are extracted through the pre-trained ResNet-50 network, and a 2048-dimensional feature vector is obtained for each sample. Finally, the 10 feature vectors are weighted averaged to obtain the final 2048-dimensional face feature vector.

[0038] Step S104: perform similarity calculation on the high-dimensional facial feature vector, and perform liveness detection through a preset multimodal information library to obtain a verification token including a user ID, a confidence score and a timestamp;

[0039] Specifically, the key link in the face detection method of the intelligent digital security refrigerator is to calculate the similarity of the high-dimensional face feature vector and perform liveness detection through the preset multimodal information library to obtain a verification token containing the user ID, confidence score and timestamp. First, the similarity calculation is completed by comparing the currently extracted high-dimensional face feature vector with the registered user feature vector pre-stored in the refrigerator database. Commonly used similarity measurement methods include cosine similarity or Euclidean distance, among which cosine similarity is widely used due to its insensitivity to the amplitude of the feature vector. The calculated similarity score is used to determine whether the current user is a registered user and to determine the specific user identity.

[0040] Liveness detection is an important means to prevent deceptive attacks (such as using photos, videos or masks). The pre-set multimodal information library contains a variety of liveness features, such as blinking frequency, micro-expression changes, head movement patterns, etc. By analyzing these features, it can be determined whether the currently detected face is a real live person. In the refrigerator scenario, liveness detection also needs to consider special factors, such as light changes when the refrigerator door is opened and closed, and the impact of cold air flow on facial expressions.

[0041] Based on the results of similarity calculation and liveness detection, a verification token is generated. The user ID is determined by the identifier corresponding to the registered user feature vector with the highest matching degree. The confidence score is a comprehensive score after considering the similarity score and liveness detection results, reflecting the credibility of the identity authentication. The timestamp records the specific time when the verification occurred, which is used for subsequent access control and behavior analysis.

[0042] For example, suppose a user opens the refrigerator door at 10 pm. The 2048-dimensional facial feature vector extracted by the system is compared with the feature vectors of five registered users stored in the database. Using cosine similarity calculation, the results are 0.92, 0.75, 0.68, 0.81, and 0.79, respectively. The highest similarity of 0.92 corresponds to the user ID "User001". At the same time, the liveness detection module detects that the user has two natural blinks within 3 seconds, the facial temperature distribution conforms to normal human characteristics, and the head also has slight natural movements. Based on this information, the system determines that it is a real live body and the liveness detection score is 0.95. Combining the similarity and liveness detection scores, the final confidence score is calculated as 0.92 × 0.95 = 0.874. The verification token generated by the system contains the following information: user ID "User001", confidence score 0.874, timestamp 2023-07-31 22:00:15.

[0043] Step S105: Analyze the access rights of the verification token based on the pre-collected user's physiological cycle data and family member interaction patterns to obtain an access rights table;

[0044] Specifically, the access rights of the verification token are analyzed through the pre-collected physiological cycle data of the user and the interaction mode of family members. The access rights table is an innovation in the face detection method of the smart digital security refrigerator. This step aims to dynamically adjust the access rights of the refrigerator according to the user's personal characteristics and family environment, thereby providing more personalized and intelligent services.

[0045] User physiological cycle data refers to periodic change information related to personal physiological status, such as women's menstrual cycle, personal work and rest patterns, etc. These data are collected through active user input or long-term collection of smart devices (such as smart bracelets). The family member interaction pattern is a family behavior model constructed by analyzing the time pattern, frequency, and sequence of family members using the refrigerator. The process of access permission analysis for the verification token begins with parsing the user ID, confidence score, and timestamp in the token. Then, the system queries the user's physiological cycle data and determines the user's physiological cycle stage in combination with the current timestamp. At the same time, the system analyzes the characteristics of the current time point in the family member interaction pattern, such as whether it is the family's common meal time, whether it is the regular use time of a specific member, etc.

[0046] Based on this information, the system dynamically generates an access rights table. This table contains the access rights levels for different refrigerator functions and areas, such as full access, limited access, or no access. The allocation of permissions takes into account multiple factors, including the user's identity, current physiological state, family interaction patterns, and the type of food stored in the refrigerator.

[0047] For example, suppose a family of four uses a smart refrigerator system, and user A (the mother) opens the refrigerator door at 9 pm. The system generates a verification token through face recognition, which contains the user ID "UserA", a confidence score of 0.95, and a timestamp of 2023-07-31 21:00:00. The system queries user A's physiological cycle data and shows that she is on the second day of her menstrual period. At the same time, the family interaction pattern analysis shows that this time period is usually a time for family members to relax together. Based on this information, the system generates the following access permission table: refrigerator main storage area (full access), freezer area (full access), wine storage area (restricted access, considering the physiological cycle stage), children's dedicated area (full access, but record behavior for subsequent analysis). This access permission table takes into account both user A's personal needs (such as the need for specific foods during special physiological periods) and the overall family usage pattern (such as preparing a midnight snack for the family to share).

[0048] Step S106: Based on the access permission table, a time series feature analysis is performed on the user refrigerator interaction behavior sequence collected in real time to obtain abnormal behavior data and refrigerator safety risk level.

[0049] Specifically, based on the access permission table, the time series feature analysis of the real-time collected user refrigerator interaction behavior sequence is performed to obtain abnormal behavior data and refrigerator security risk level, which is the last key step of the intelligent digital security refrigerator face detection method. This process first involves real-time collection of user interaction behaviors with the refrigerator, including door opening and closing frequency, dwelling time, storage areas visited, type and quantity of food taken and placed, etc. These data are collected in real time through various sensors built into the refrigerator (such as door magnetic switches, weight sensors, temperature sensors, etc.). Time series feature analysis refers to pattern recognition and anomaly detection on these interactive behavior data in time series. This process uses machine learning algorithms such as long short-term memory networks (LSTM) or time convolution networks (TCN) to learn the normal behavior patterns of users. The analysis process considers multi-dimensional information such as the user's historical behavior patterns, the current access permission table, and time factors (such as time of day, weekends / weekdays). Abnormal behavior data refers to behaviors that deviate significantly from the user's normal behavior patterns, such as frequent opening and closing of refrigerator doors, long stays, and access to unauthorized areas. Each detected abnormal behavior will be recorded, including detailed information such as behavior type, occurrence time, and duration.

[0050] The refrigerator security risk level is based on a comprehensive assessment of the abnormal behavior detected, its severity and frequency. The risk level is usually divided into three levels: low, medium and high, and each level corresponds to a different response strategy, such as issuing a warning, restricting access to specific areas, notifying family members, etc.

[0051] For example, suppose that at 2 a.m., the system detects that user B (a 14-year-old child) has opened the refrigerator. According to the access permission table, user B is only allowed to access the main storage area during this time period, and there are restrictions on food. The system records the following behavior sequence: open the door - access the freezer for 15 seconds - access the wine storage area for 10 seconds - take out an item weighing about 500g - close the door. The total process lasts 45 seconds. The time series feature analysis algorithm compares this behavior sequence with the normal pattern of user B (usually accessing the refrigerator between 4 and 5 p.m., with an average stay of 30 seconds, and mainly accessing the main storage area), and finds multiple anomalies: unusual time, access to unauthorized areas, and taking out abnormal weight of food. The system generates abnormal behavior data, recording the detailed information of this behavior. At the same time, considering the time when the behavior occurred and the areas involved (especially the wine storage area), the system rates the security risk level of this behavior as "high".

[0052] In the embodiment of the present application, by performing adaptive histogram equalization and geometric correction on the high-frame rate image sequence collected in the area in front of the refrigerator door, the problems of unstable light and image distortion in the refrigerator use environment are effectively solved, and the accuracy and robustness of subsequent face detection are significantly improved; secondly, a face detection method based on multi-level sliding window scanning is adopted, which can accurately locate the face boundary box and facial key points, laying the foundation for subsequent feature extraction; furthermore, by performing multi-angle sample generation and feature extraction on the target face area, the high-dimensional face feature vector obtained captures the user's facial features more comprehensively and enhances the reliability of identity authentication; and liveness detection is performed through a preset multimodal information library, which effectively prevents impersonation and deception and improves the security of the system; by introducing user physiological cycle data and family member interaction mode to perform access permission analysis, the method realizes more refined and personalized permission management and meets the differentiated needs of different family members for the use of refrigerators; finally, the real-time collected user refrigerator interaction behavior sequence is subjected to time series feature analysis to obtain abnormal behavior data and refrigerator safety risk level, which greatly improves the safety performance of smart refrigerators. The present invention can not only accurately identify abnormal usage behaviors, but also timely warn of potential safety risks, such as children's accidental ingestion and food spoilage, providing new protection for family safety. In short, by comprehensively considering the unique usage environment and user behavior characteristics of refrigerators, high-precision identity authentication and intelligent authority management are achieved, which improves the efficiency and accuracy of face detection in smart digital safety refrigerators.

[0053] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0054] (1) Adaptively sample the refrigerator door opening frequency of the high frame rate image sequence to obtain a dynamic key frame set, and eliminate the light interference of the refrigerator room from the dynamic key frame set to obtain an initial denoised image set;

[0055] (2) Apply the temperature difference compensation algorithm between the inside and outside of the refrigerator door to the initial denoised image set to perform fog correction to obtain a definition-enhanced image, and perform refrigerator door frame surface distortion calibration on the definition-enhanced image to obtain a geometrically corrected image;

[0056] (3) Compensating the effect of freezer frost on the geometrically corrected image to obtain a contrast-enhanced image, and removing the shadow of the refrigerator door handle from the contrast-enhanced image to obtain a shadow-balanced image;

[0057] (4) Suppressing the color interference of food stored in the refrigerator on the shadow-equalized image to obtain a color-pure image, and performing refrigerator-feature-preserving edge enhancement on the color-pure image to obtain an edge-sharpened image;

[0058] (5) Adaptively correct the uneven lighting of the refrigerator for the edge-sharpened image to obtain a balanced lighting image;

[0059] (6) The illumination-balanced image is adaptively cropped according to the refrigerator usage scenario to obtain a standardized image.

[0060] Specifically, in the face detection method of intelligent digital security refrigerator, the processing of high frame rate image sequences is a complex and delicate process involving multiple key steps. The adaptive sampling of refrigerator door opening frequency is designed for different users' door opening habits. By analyzing the historical door opening frequency data, the optimal sampling interval is determined, for example, a shorter interval (such as 0.1 second) is used when the door is opened frequently, and a longer interval (such as 0.5 second) is used when the door is opened infrequently. This adaptive strategy can capture fast movements and reduce redundant data, and the obtained dynamic key frame set is more representative. The dynamic key frame set is eliminated from the light interference of the refrigerator room. The LED lights in the refrigerator room will cause strong light changes at the moment of opening the door, affecting the image quality. By analyzing the image brightness histogram, the abnormal brightness area is identified, and the local adaptive histogram equalization technology is used to correct it to obtain the initial denoised image set.

[0061] The temperature difference compensation algorithm between the inside and outside of the refrigerator door is used to correct fogging. When cold air encounters warm and humid outside air, slight fogging will form in front of the lens. By establishing a temperature-fogging model, the image is deconvolved according to the real-time temperature sensor data to eliminate the fogging effect and obtain an image with improved clarity. The refrigerator door frame surface distortion calibration is aimed at the image distortion caused by the arc design of the refrigerator door. Using the pre-calibrated distortion parameters, the image is corrected by polynomial transformation to obtain a geometrically corrected image. The freezer frost compensation of the geometrically corrected image is to solve the reflection problem caused by frost. By analyzing the high-frequency components of the image, the frosted area is identified and processed using the local contrast enhancement algorithm to obtain a contrast-enhanced image. The refrigerator door handle shadow elimination processing uses morphological operations and local brightness adjustment to remove the shadow cast by the handle to obtain a shadow-balanced image.

[0062] The purpose of suppressing the color interference of food stored in the refrigerator is to reduce the impact of background food on face detection. Through color clustering and skin color model, non-skin color areas are identified and suppressed to obtain a pure color image. The refrigerator feature preservation edge enhancement uses an improved Canny edge detection algorithm to strengthen key features such as face contours to obtain an edge-sharpened image. The refrigerator lighting non-uniformity adaptive correction analyzes the image brightness distribution, establishes a lighting model, and makes targeted brightness adjustments to different areas to obtain a balanced lighting image. Finally, the refrigerator uses scene adaptive cropping to dynamically adjust the cropping range according to the preset area of ​​interest and real-time face detection results to obtain the final standardized image.

[0063] For example, suppose that at 9 pm, a user opens the refrigerator door to get a drink. The high frame rate camera captures images at 120fps. Door opening frequency analysis shows that users open the door once every 5 seconds on average, so the sampling interval is set to 0.2 seconds. In 2 seconds, 10 key frames are collected. The first frame is overexposed because the LED light in the refrigerator is suddenly turned on. After brightness histogram analysis and local equalization, the pixel value of the overexposed area is reduced from 255 to about 180. The temperature sensor shows that the temperature difference between the inside and outside of the refrigerator is 15°C. The fog correction algorithm is applied to improve the mean sharpness index (MSE) of the image by about 20%. Door frame distortion calibration corrects the curved straight lines at the edge of the image, and the distortion rate is reduced from 5% to less than 0.5%. Frost compensation improves the contrast of the freezer area by 30%. The shadow area of ​​the door handle is reduced by 90%, and the difference between the remaining shadow and the surrounding brightness does not exceed 10%. Color interference suppression reduces the saturation of non-skin areas by 50%. Edge enhancement increases the gradient value of the face contour by 40%. After illumination equalization, the standard deviation of brightness in each area of ​​the image is reduced to 1 / 3 of the original. Finally, according to the detected face position, the image is cropped to a standard ratio of 3:4 and a size of 1080x1440 pixels.

[0064] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0065] (1) Adaptively scale the standardized image according to the refrigerator door opening angle to obtain a multi-scale image pyramid, and then compensate for the changes in the refrigerator ambient light on the multi-scale image pyramid to obtain a lighting-invariant feature map;

[0066] (2) The refrigerator door handle area is shielded in the illumination-invariant feature map to obtain an optimized candidate area, and a sliding window scan weighted by the refrigerator usage frequency is performed on the optimized candidate area to obtain a preliminary face position estimate;

[0067] (3) Eliminate the interference of the refrigerator door frame reflection on the preliminary face position estimate to obtain a fine face region, and correct the temperature gradient inside the refrigerator for the fine face region to obtain a temperature-balanced face image;

[0068] (4) Adaptive boundary adjustment of the refrigerator door opening and closing speed is performed on the temperature-balanced face image to obtain a dynamic boundary frame, and the dynamic boundary frame is subjected to color interference suppression of the food stored in the refrigerator to obtain a pure face area;

[0069] (5) The shadow of the refrigerator-specific light source is compensated for the pure face area to obtain a shadow-balanced face image, and the refrigerator user's height-adaptive key point positioning is performed on the shadow-balanced face image to obtain the face bounding box and the coordinates of the five facial key points.

[0070] Specifically, the standardized image is adaptively scaled according to the refrigerator door opening angle to adapt to the changes in face size at different door opening angles. The door angle sensor is used to obtain the door opening angle in real time, and an angle-scaling ratio mapping relationship is established. For example, the scaling ratio is 1.2 when the door is opened to 30 degrees, 1.5 when it is opened to 60 degrees, and 1.8 when it is opened to 90 degrees. Based on this mapping relationship, the original image is scaled accordingly, and a 3 to 5-layer image pyramid is constructed. The image size of each layer is reduced by 0.8 times compared to the previous layer, and a multi-scale image pyramid is obtained.

[0071] The multi-scale image pyramid is used to compensate for the changes in ambient light in the refrigerator. The changes in ambient light in the refrigerator mainly come from the internal lighting of the refrigerator and the external ambient light. By analyzing the change trend of the image brightness histogram, a light change model is established. Adaptive gamma correction is applied to each layer of the image to adjust the overall brightness and contrast of the image to obtain the illumination invariant feature map. The refrigerator door handle area is shielded in the illumination invariant feature map. The door handle area often causes false detection. Through the pre-calibrated door handle position information, the pixel value of this area in the image is set to 0 or the average background color to obtain the optimized candidate area. Then, the optimized candidate area is scanned by a sliding window weighted by the frequency of refrigerator use. According to historical data, the high-frequency time periods when users use the refrigerator are analyzed, and the scanning density is increased during these time periods, and the scanning density is reduced during low-frequency time periods. The improved Viola-Jones algorithm is used for face detection to obtain a preliminary face position estimate.

[0072] The preliminary face position estimation is used to eliminate the interference of the refrigerator door frame reflection. Door frame reflection often leads to false detection. By analyzing the edge information of the image, possible reflection areas are identified. Local contrast enhancement and edge smoothing techniques are used to eliminate the reflection effect and obtain a fine face area. Subsequently, the fine face area is corrected for the temperature gradient inside the refrigerator. The temperature gradient inside the refrigerator may cause local blurring of the image. The temperature distribution model is constructed based on the temperature sensor data, and the sharpening degree of different areas of the image is adjusted accordingly to obtain a temperature-balanced face image. The temperature-balanced face image is adaptively adjusted to the opening and closing speed of the refrigerator door. The opening and closing speed of the door will affect the degree of motion blur of the face in the image. By analyzing the difference between consecutive frames, the opening and closing speed of the door is estimated, and the size and shape of the face bounding box are dynamically adjusted accordingly to obtain a dynamic bounding box. Then, the dynamic bounding box is subjected to the color interference suppression of the food stored in the refrigerator. The food in the refrigerator may be similar to the skin color, causing interference. The color clustering algorithm is used to identify the non-skin color area and reduce its significance to obtain a pure face area.

[0073] The refrigerator-specific light source shadow compensation is performed on the pure face area. The LED lights inside the refrigerator may cast special shadows on the face. By establishing a light source-shadow model and applying a local brightness adjustment algorithm, the influence of these shadows is eliminated to obtain a shadow-balanced face image. Finally, the refrigerator user's height adaptive key point positioning is performed on the shadow-balanced face image. According to the height information recorded in the user profile, the initial position and range of the facial feature point search are adjusted, and the improved ASM (ActiveShape Model) algorithm is used to accurately locate the five facial key points (eyes, nose tip, and mouth corners) to obtain the final face bounding box and the coordinates of the five facial key points.

[0074] For example, suppose a user with a height of 175cm opens the refrigerator door at 8pm. The door angle sensor shows that the door is open at an angle of 45 degrees. The system scales the original 1080x1440 pixel image by 1.3 times and builds a 3-layer image pyramid. Ambient light compensation increases the average brightness of the image from 120 to 180. The door handle area (200x300 pixel area in the lower right corner of the image) is blocked. Considering that 8pm is a high-frequency usage period for users, the sliding window scanning step size is reduced from the usual 16 pixels to 12 pixels. Three face candidate areas are initially detected, one of which is located in the door frame reflection area and is removed after reflection elimination. Temperature gradient correction improves the clarity of the face area by 15%. Door opening and closing speed detection shows that the door is opened at a fast speed (0.5m / s), and the system enlarges the face bounding box by 10% to cope with possible motion blur. Color interference suppression reduces the saturation of areas in the background that are similar to skin color (such as orange) by 40%. The shadow compensation algorithm corrected the shadow under the brow bone caused by the top LED light, increasing the brightness of this area by 20%. Finally, based on the user's height, the initial position of the eye search was set at the upper third of the image, and five facial key points were successfully located. The final face bounding box size was 240x320 pixels, located slightly above the center of the image. This series of processing ensures that the user's face can still be accurately detected and located in a complex refrigerator usage environment.

[0075] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0076] (1) The face boundary frame is adaptively adjusted to the opening and closing angle of the refrigerator door to obtain a dynamic face area, and the reflection of the light source inside the refrigerator is eliminated in the dynamic face area to obtain a de-reflected face image;

[0077] (2) Compensating the de-reflected face image for refrigerator temperature changes to obtain a temperature-balanced face image, and correcting the temperature-balanced face image for interference from the odor of food stored in the refrigerator to obtain a pure face area;

[0078] (3) Perform multi-angle rotation on the pure face area weighted by the frequency of refrigerator use to obtain a multi-view face sample set, and simulate the occlusion of the refrigerator door handle on the multi-view face sample set to obtain an occlusion-enhanced sample;

[0079] (4) Enhance the temperature gradient feature inside the refrigerator for the occlusion enhancement sample to obtain a temperature feature map, and perform adaptive feature extraction of the refrigerator door opening frequency on the temperature feature map to obtain a time-varying feature set;

[0080] (5) The correlation between the time-varying feature set and the refrigerator user habits is analyzed to obtain the user feature map. The user feature map is then enhanced to be robust to the refrigerator environment noise to obtain a high-dimensional face feature vector.

[0081] Specifically, the facial feature extraction process in the intelligent digital safety refrigerator face detection method is a complex and delicate step involving multiple key links. First, the face bounding box is adaptively adjusted to the refrigerator door opening and closing angle to adapt to the changes in face size and position at different door opening angles. The door angle sensor is used to obtain the door opening angle in real time, and establish an angle-adjustment parameter mapping relationship. For example, when the door is opened to 30 degrees, the bounding box is expanded by 10%, when it is opened to 60 degrees, it is expanded by 20%, and when it is opened to 90 degrees, it is expanded by 30%. According to this mapping relationship, the size and position of the face bounding box are dynamically adjusted to obtain a dynamic face area.

[0082] The dynamic face area is de-reflected by the light source inside the refrigerator. The metal surface and glass partition inside the refrigerator often produce reflections, which affect the quality of the face image. By analyzing the distribution and intensity of the highlight area of ​​the image and combining the internal structure information of the refrigerator, a reflection model is constructed. The improved polarization filtering algorithm is used to reduce the brightness of the reflection area in a targeted manner while maintaining the details of the non-reflection area to obtain a de-reflected face image. The de-reflected face image is compensated for the change in refrigerator temperature. The temperature difference between the inside and outside of the refrigerator will cause local blur or distortion of the image. The temperature distribution model is constructed based on the temperature sensor data, and the sharpness and contrast of different areas of the image are adjusted accordingly to obtain a temperature-balanced face image. Then, the temperature-balanced face image is corrected for the interference of the odor of food stored in the refrigerator. The volatile substances of some foods will form tiny particles on the lens, affecting the image clarity. By analyzing the distribution of high-frequency components of the image, the area disturbed by the odor is identified, and the adaptive Wiener filtering algorithm is applied to perform local deblurring processing to obtain a pure face area. The pure face area is rotated at multiple angles weighted by the frequency of refrigerator use. According to historical data, the common postures of users when using the refrigerator are analyzed, and higher weights are given to angles that appear frequently. Usually 5 to 7 rotation samples are generated in the range of -15° to +15°, and the weights are dynamically assigned according to the frequency of occurrence. The multi-view face sample set obtained in this way is more in line with the actual usage scenario. Subsequently, the refrigerator door handle occlusion simulation is performed on the multi-view face sample set. Door handle occlusion is a common interference factor. By analyzing the position of the door handle at different door opening angles, the occlusion effect on the face is simulated, the diversity of the samples is enhanced, and the occlusion enhanced samples are obtained.

[0083] The temperature gradient feature inside the refrigerator is enhanced for the occlusion enhancement sample. There is an obvious temperature gradient inside the refrigerator, which will affect the thermal imaging features of the face. The temperature gradient model is constructed through the temperature data collected by the infrared sensor, and the temperature-related features in the enhanced image, such as the hot spots such as the nose tip and the corner of the eye, are obtained to obtain the temperature feature map. Then, the temperature feature map is adaptively extracted based on the refrigerator door opening frequency. According to the statistical data of the user's door opening frequency, the density and range of feature extraction are dynamically adjusted. More detailed features are extracted during high-frequency use periods, and the main features are focused on during low-frequency periods to obtain a time-varying feature set. The time-varying feature set is analyzed for the correlation between refrigerator user habits. By analyzing the user's usage habits, such as commonly used storage areas and door opening times, a feature-habit mapping relationship is established. These habit information is integrated into the feature representation to obtain a more personalized user feature map. Finally, the user feature map is enhanced for robustness to refrigerator environmental noise. The vibration and noise during the operation of the refrigerator will affect the image quality. By simulating the feature changes under different noise levels, the feature's anti-noise ability is enhanced, and finally a high-dimensional face feature vector is obtained.

[0084] For example, suppose a user opens the refrigerator door at 10 pm to get a drink. The door angle sensor shows that the door is open at 60 degrees, and the system expands the original 240x320 pixel face bounding box by 20% to 288x384 pixels. The internal light source reflection elimination algorithm identifies the strong reflection of the 40x60 pixel area in the upper right corner and reduces its brightness from 250 to 180. The temperature sensor shows that the temperature difference between the inside and outside of the refrigerator is 18°C, and the temperature compensation algorithm improves the overall image clarity by 15%. The odor interference correction identifies the slight blur in the lower half, and the average gradient value of the blurred area is increased from 0.4 to 0.6 through the Wiener filter. According to user habits, five rotation samples of -10°, -5°, 0°, +5°, and +10° are generated, of which 0° and +5° have a weight of 0.3 and other angles have a weight of 0.2. The door handle occlusion simulation adds a 30x50 pixel occlusion area in the lower left corner of the -10° sample. The temperature gradient feature enhancement increases the feature weight of the nose tip area by 20%. The door opening frequency analysis shows that 10 pm is a high-frequency usage period, and the feature extraction density is increased by 30%, resulting in a 2048-dimensional feature vector. User habit analysis shows that the user often uses the middle storage area, and the feature weight of the corresponding area is increased by 15%. Finally, by adding 5% Gaussian noise for robustness enhancement, the final 2048-dimensional high-dimensional face feature vector is obtained.

[0085] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0086] (1) The high-dimensional face feature vector is weighted by the frequency of refrigerator door opening to obtain the time series enhanced feature, and the time series enhanced feature is compensated for the temperature change inside the refrigerator to obtain the temperature invariant feature;

[0087] (2) The temperature-invariant features are compensated for the airflow disturbance caused by the opening and closing of the refrigerator door to obtain stable facial features. The stable facial features are then correlated with the refrigerator usage period to obtain time-sensitive features.

[0088] (3) The similarity of time-sensitive features is calculated using the additive angle margin loss function to obtain the initial matching result;

[0089] (4) The initial matching result is verified by the refrigerator door handle contact pattern to obtain the tactile enhancement matching score, and the tactile enhancement matching score is adaptively adjusted to the refrigerator light changes to obtain the lighting robust matching value;

[0090] (5) Perform consistency test on the lighting robust matching value and refrigerator usage habits to obtain behavior verification results;

[0091] (6) The behavior verification result is compensated for the change in humidity inside the refrigerator to obtain the user ID, and the user ID is fused with the refrigerator door opening force feature to obtain a verification token, where the verification token includes: user ID, confidence score and timestamp.

[0092] Specifically, the feature processing and identity authentication process in the face detection method of the intelligent digital security refrigerator is a complex and delicate step involving multiple key links. First, the high-dimensional face feature vector is weighted by the frequency of refrigerator door opening in order to highlight the characteristics of users using the refrigerator in different time periods. By analyzing the user's historical door opening records, a time-frequency mapping table is constructed, for example, the weight is 1.5 for 7-8 am and 1.2 for 10-11 pm. These weights are applied to the corresponding dimensions of the feature vector to obtain the time series enhanced features. The time series enhanced features are compensated for the temperature changes inside the refrigerator. The fluctuation of the temperature inside the refrigerator will affect the stability of the facial features. The temperature-feature influence model is established through the data collected by the temperature sensor. For example, when the temperature rises from 4°C to 8°C, the value of a specific feature dimension may increase by 5%. According to the real-time temperature data, the feature vector is adjusted accordingly to obtain the temperature invariant feature.

[0093] The temperature-invariant features are compensated for the airflow disturbance caused by the opening and closing of the refrigerator door. The opening and closing of the door will cause the exchange of hot and cold air, affecting the clarity of the face image. By analyzing the speed and frequency of the door opening and closing, the degree of airflow disturbance is estimated, and the feature vector is smoothed. For example, when the door is opened quickly, the high-frequency feature components are attenuated by 10% to obtain stable face features. The stable face features are analyzed for correlation with the refrigerator usage period to obtain time-sensitive features. This step takes into account the user's behavior patterns when using the refrigerator in different time periods. For example, in the early morning, the facial fatigue features may be more concerned, while in the evening, the overall facial contour is more concerned. According to the current time, the weights of each dimension in the feature vector are dynamically adjusted to highlight the time-related features. The time-sensitive features are similarity calculated using the additive angular margin loss function to obtain the initial matching results. This step uses cosine similarity to measure the distance between feature vectors and introduces angular margins to increase inter-class differences. The calculation formula is: ;

[0094] in, is the angle between the eigenvectors, m is a margin parameter, usually set to 0.5. As the final similarity score, its value range is [-1, 1]. The closer the value is to 1, the more similar the two feature vectors are; the closer the value is to -1, the less similar the two feature vectors are.

[0095] The initial match result is verified by the refrigerator door handle contact pattern to obtain the tactile enhancement match score. This step combines the touch sensor data on the door handle to verify the user's door opening habits. For example, if the user is accustomed to opening the door with the index and middle fingers, the match score will increase by 10% when this pattern is detected. The tactile enhancement match score is adaptively adjusted to the refrigerator light changes to obtain the lighting robust match value. The change in light inside and outside the refrigerator can affect the accuracy of face recognition. The match score is corrected by analyzing the difference between the current lighting conditions and the lighting conditions when the user registered. For example, in low light conditions, the matching threshold may be reduced by 15%. The lighting robust match value is checked for consistency with refrigerator usage habits to obtain the behavior verification result. This step compares the consistency of the current user's behavior (such as door opening time, storage area visited) with the historical behavior pattern. If the consistency is higher than 80%, the behavior verification is considered to have passed.

[0096] The behavior verification result is compensated for the humidity change inside the refrigerator to obtain the user ID. Humidity changes will affect the quality of the face image. The verification result is fine-tuned through the humidity sensor data. For example, when the humidity rises from 50% to 70%, the verification threshold may be reduced by 5%. After determining the user ID, the final identity confirmation is performed in combination with the refrigerator door opening force characteristics (such as door opening speed and force), and a verification token containing the user ID, confidence score and timestamp is generated.

[0097] For example, suppose user A opens the refrigerator at 9 pm. The 2048-dimensional facial feature vector extracted by the system is first weighted according to the frequency of door opening. The weight at 9 pm is 1.3, and the overall feature value is increased by 30%. The temperature sensor shows that the temperature inside the refrigerator is 6°C, which is 2°C higher than the average temperature. The system compensates for the 3% reduction in the feature value of a specific dimension. The door opening and closing speed detection shows that it is a gentle opening, and the airflow disturbance compensation value is 5%. The time period analysis shows that the facial contour features are more concerned at 9 pm, and the weight of the relevant dimensions is increased by 20%. The initial matching score calculated by the additive angular margin loss function is 0.85. The door handle contact pattern is 90% consistent with the historical record of user A, and the matching score is improved to 0.93. The current lighting conditions are 30% lower than the brightness at registration, and the system reduces the matching threshold from 0.9 to 0.85. The user behavior consistency test shows that the current behavior is 85% consistent with the historical pattern, and the verification is passed. The humidity sensor shows that the current humidity is 65%, which is 15% higher than the average, and the system further reduces the verification threshold by 2%. Finally, the system generates a verification token containing the user ID "UserA", a confidence score of 0.91, and a timestamp of 2023-07-31 21:00:15.

[0098] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0099] (1) Perform refrigerator door opening time period distribution statistics on the verification token to obtain user time preference data, and perform refrigerator storage area access frequency correlation analysis on the user time preference data to obtain a regional access tendency map;

[0100] (2) Calculate the correlation between the temperature change inside the refrigerator and the regional access tendency map to obtain the temperature control habit characteristics, and match the temperature control habit characteristics with the refrigerator energy consumption data to obtain the user energy-saving tendency index;

[0101] (3) Conduct correlation analysis on the user's energy-saving tendency index and the refrigerator food storage cycle to obtain the dietary pattern characteristics;

[0102] (4) Weighting the eating pattern characteristics by the refrigerator door opening and closing frequency to obtain the user's daily routine data, and identifying the light change pattern inside the refrigerator based on the user's daily routine data to obtain the day and night activity characteristics;

[0103] (5) Analyze the synchronization of refrigerator defrosting cycles based on diurnal activity characteristics to obtain the estimated time users spend at home. Calculate the overlap of refrigerator usage peak hours based on the estimated time users spend at home to obtain a family member interaction intensity map.

[0104] The family members' interaction intensity graph is matched with the refrigerator intelligent scene pattern to obtain the access permission table.

[0105] Specifically, the verification token is used to perform distribution statistics on the refrigerator door opening time period. This step analyzes the user's historical door opening records, divides the 24 hours of a day into multiple time periods (such as one hour per hour), and counts the number of door openings in each time period to obtain the user's time preference data. For example, the statistical results may show that the user opens the door most frequently during the three time periods of 7:00-8:00, 12:00-13:00, and 18:00-19:00.

[0106] The user time preference data is analyzed for the frequency of visits to the refrigerator storage area. This step combines the time preference with the frequency of visits to different storage areas of the refrigerator (such as the refrigerator, freezer, fruit and vegetable box, etc.) to generate a regional visit tendency map. For example, the fruit and vegetable box and milk storage area may be mainly visited in the morning, while the freezer is visited more in the evening. The correlation between the temperature change inside the refrigerator and the regional visit tendency map is calculated to obtain the temperature control habit characteristics. This step analyzes the change pattern of the temperature inside the refrigerator after the user visits different areas. For example, frequent visits to the freezer may lead to large temperature fluctuations, while users who mainly visit the refrigerator may have smaller temperature fluctuations. Through this analysis, the user's sensitivity and preference for temperature changes can be obtained.

[0107] The temperature control habit characteristics are matched with the refrigerator energy consumption data to obtain the user's energy-saving tendency index. This step combines the user's temperature control habits with the refrigerator's energy consumption data to calculate the user's energy-saving degree. For example, if the user is accustomed to setting the refrigerator temperature at a higher level and opening the door for a shorter time, then their energy-saving tendency index is higher. The user's energy-saving tendency index is analyzed in association with the refrigerator food storage cycle to obtain the dietary regularity characteristics. This step analyzes the storage time of different foods in the refrigerator and combines the user's energy-saving tendency to infer the user's dietary habits. For example, users who often store fresh fruits and vegetables for a short period of time may have the habit of healthy eating.

[0108] The eating pattern characteristics are weighted by the frequency of refrigerator door opening and closing to obtain the user's daily life data. This step combines eating patterns with the frequency of refrigerator use to more comprehensively reflect the user's daily life rhythm. For example, users who frequently open the door at night may have the habit of eating late-night snacks. Subsequently, the user's daily life data is used to identify the light change pattern inside the refrigerator to obtain the day and night activity characteristics. By analyzing the data of the light sensor inside the refrigerator, it is determined whether the user uses the refrigerator in a dark environment, thereby inferring the user's daily life pattern.

[0109] The synchronization of refrigerator defrost cycles is analyzed for diurnal activity characteristics to obtain the estimated time data of users at home. The defrost cycle of refrigerators is usually set during the period when users are not using the refrigerators. By analyzing the overlap between user activities and defrost cycles, the time period when users are at home can be estimated. The overlap of refrigerator peak usage is calculated for the estimated time data of users at home to obtain the family member interaction intensity map. This step analyzes whether the refrigerator usage peaks of multiple family members overlap. A high overlap indicates frequent interaction among family members. Finally, the family member interaction intensity map is matched with refrigerator intelligent scene patterns to obtain an access permission table. Different intelligent scenes (such as family dinners, personal solitude, etc.) are set according to the interaction patterns of family members, and corresponding access permissions are assigned to each scene.

[0110] For example, in a smart refrigerator system for a family of four, the verification token of user A (father) shows that he mainly uses the refrigerator between 7:00-8:00 in the morning and 18:00-19:00 in the evening. Statistics show that he mainly visits the milk area and fruit and vegetable box in the refrigerator in the morning, and visits the freezer more at night. Temperature analysis shows that user A is accustomed to setting the refrigerator temperature at a lower level, and energy consumption data shows that his energy-saving tendency index is 75 (out of 100). Food storage cycle analysis shows that user A prefers fresh ingredients, with an average storage cycle of 3 days. Door switch frequency data shows that user A opens the door an average of 10 times a day, mainly in the morning and evening. Light change pattern recognition shows that user A rarely uses the refrigerator at night. Defrost cycle synchronization analysis estimates that user A usually does not use the refrigerator between 21:00 and 6:00. Family member interaction intensity analysis shows that dinner time (18:00-19:00) is a period of high overlap in refrigerator use among family members. Based on this data, the system generates an access permission table for User A: full access to all areas during the morning hours (6:00-9:00) and evening hours (17:00-21:00), and restricted access to the freezer and wine storage area during other hours.

[0111] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0112] (1) Analyze the refrigerator door opening and closing frequency of the user's refrigerator interaction behavior sequence to obtain the door opening pattern characteristics, and calculate the correlation between the refrigerator storage area temperature fluctuation and the door opening pattern characteristics to obtain the temperature change trajectory;

[0113] (2) Match the temperature change trajectory with the peak time period of refrigerator energy consumption to obtain energy consumption behavior characteristics, and then count the refrigerator lighting usage time based on the energy consumption behavior characteristics to obtain lighting demand data;

[0114] (3) Analyze the humidity change trend inside the refrigerator based on the light demand data to obtain the storage environment index;

[0115] (4) Calculate the refrigerator defrost cycle deviation based on the storage environment indicators to obtain maintenance demand warnings, and perform refrigerator food storage and retrieval sequence correlation analysis on the maintenance demand warnings to obtain user habit characteristics;

[0116] (5) Detect changes in contact pressure of refrigerator door handles based on user habit characteristics to obtain operation force data;

[0117] (6) The operation force data is analyzed for fluctuations in the value of the refrigerator's internal odor sensor to obtain storage anomaly indicators. The storage anomaly indicators are compared with the access permission table to determine the refrigerator's safety threshold to obtain abnormal behavior data and the refrigerator's safety risk level.

[0118] Specifically, the abnormal behavior detection and security risk assessment in the face detection method of the intelligent digital safety refrigerator is a complex and delicate process involving multiple key links. First, the refrigerator door opening and closing frequency analysis is performed on the user's refrigerator interaction behavior sequence. This step records the timestamp of the user opening and closing the refrigerator door through the door magnetic sensor, calculates the number of door openings and duration per hour, and obtains the door opening mode characteristics. For example, the record shows that the user opens the door an average of 5 times from 7:00 to 8:00 in the morning, each time lasting 10 seconds, and an average of 8 times from 18:00 to 19:00 in the evening, each time lasting 15 seconds. Subsequently, the door opening mode characteristics are correlated with the temperature fluctuation of the refrigerator storage area to obtain the temperature change trajectory. This step combines the door opening behavior with the temperature sensor data of each storage area (such as the refrigerator and freezer) to analyze the temperature change amplitude and recovery time after the door is opened. For example, the temperature of the refrigerator rises by 2°C after the door is opened, and it takes 5 minutes to return to the set temperature, while the temperature of the freezer rises by 1°C and it takes 8 minutes to recover. Next, the temperature change trajectory is matched with the refrigerator energy consumption peak time period to obtain the energy consumption behavior characteristics. This step correlates temperature changes with the operating status of the refrigerator compressor and identifies peak energy consumption periods. For example, it was found that during the 18:00-19:00 period when the door is frequently opened, the compressor operating time increases by 30%, and energy consumption increases significantly.

[0119] Then, the energy behavior characteristics are used to count the duration of refrigerator lighting use to obtain lighting demand data. By analyzing the working time of the lights inside the refrigerator, the lighting needs of users at different times are calculated. For example, statistics show that users use lighting for an average of 5 seconds each time they open the door in the morning, while they use it for an average of 10 seconds in the evening. The lighting demand data is used to analyze the trend of humidity changes inside the refrigerator to obtain storage environment indicators. This step combines the humidity sensor data to analyze the relationship between lighting use and humidity changes. For example, it is found that after opening the door and using lighting for a long time, the humidity in the refrigerator increases by 5%, and it takes 2 hours to return to normal. Next, the storage environment indicators are used to calculate the deviation of the refrigerator defrost cycle to obtain maintenance demand warnings. The operating status of the refrigerator is evaluated by comparing the difference between the actual defrost interval and the preset cycle. For example, if the actual defrost interval is 20% shorter than the preset cycle, a maintenance warning signal is generated. Subsequently, the maintenance demand warning is analyzed in association with the refrigerator food storage and retrieval sequence to obtain user habit characteristics. This step uses weight sensors and cameras to identify the food storage and retrieval sequence and analyzes the user's usage habits. For example, it is found that users tend to take out vegetables first, then store cooked food, and finally take out drinks. Then, the contact pressure change of the refrigerator door handle is detected based on the user's habit characteristics to obtain the operation force data. The pressure sensor on the door handle records the change in the user's door opening force. For example, it is detected that the user usually opens the door with a force of 3N, but the force increases to 4.5N when tired. The operation force data is analyzed for the fluctuation of the value of the refrigerator's internal odor sensor to obtain the storage anomaly index. This step associates the door opening force with the internal odor sensor data to identify potential storage anomalies. For example, when it is detected that the door opening force suddenly increases and the odor sensor value rises sharply by 50%, it is determined that food may be spoiled.

[0120] Finally, the storage abnormality index is compared with the access permission table to the refrigerator security threshold to obtain abnormal behavior data and refrigerator security risk level. This step compares the detected abnormal indicators with the preset security threshold, while taking into account the user's access rights. For example, if it is detected that an unauthorized user frequently opens the refrigerator late at night with abnormal operation force, and the storage abnormality index exceeds the security threshold by 30%, the risk level will be assessed as "high" and a detailed abnormal behavior data report will be generated.

[0121] For example, suppose that at 8 pm on a weekday, the smart refrigerator detects that user A (father) has opened the refrigerator door. The door magnetic sensor records that this is the 7th time the door has been opened within an hour, which is significantly higher than the average frequency (usually 3-4 times / hour). The temperature sensor data shows that the temperature of the refrigerator compartment has risen by 2.5°C after the door was opened, which is slightly higher than the normal situation (2°C). The energy consumption analysis shows that the compressor has increased its working time by 25% in the past hour. The lighting usage time is 15 seconds each time, which is longer than the average (10 seconds). The humidity sensor shows that the humidity in the refrigerator compartment has risen by 7%, which exceeds the normal fluctuation range (5%). The defrost cycle analysis shows that the most recent defrost interval is 15% shorter than the normal cycle. The food access sequence analysis found that the user took out and put back frozen food abnormally frequently. The door handle pressure sensor recorded a door opening force of 4.8N, which is significantly higher than the normal level of user A (3.2N). The odor sensor value rose by 40% in a short period of time. After comparing these data with the access permission table of user A, the system determines that the current behavior is abnormal, assesses the risk level as "medium", and generates an abnormal behavior report, recommending attention to the use of the refrigerator and the storage status of food.

[0122] The above describes the face detection method of the intelligent digital safety refrigerator in the embodiment of the present application. The following describes the face detection system of the intelligent digital safety refrigerator in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the intelligent digital safety refrigerator face detection system includes:

[0123] The correction module 201 is used to perform adaptive histogram equalization and geometric correction on the high frame rate image sequence collected from the door front area of ​​the target refrigerator to obtain a standardized image;

[0124] A detection module 202 is used to perform face detection on the standardized image based on multi-level sliding window scanning to obtain a face bounding box and coordinates of five facial key points;

[0125] A segmentation module 203 is used to perform region segmentation on the standardized image based on the face boundary box and the coordinates of the five facial key points to obtain a target face region, and to perform multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector;

[0126] The calculation module 204 is used to perform similarity calculation on the high-dimensional facial feature vector and perform liveness detection through a preset multimodal information library to obtain a verification token including a user ID, a confidence score and a timestamp;

[0127] The analysis module 205 is used to analyze the access rights of the verification token based on the pre-collected user physiological cycle data and family member interaction mode to obtain an access rights table;

[0128] The early warning module 206 is used to perform time series feature analysis on the user refrigerator interaction behavior sequence collected in real time based on the access permission table to obtain abnormal behavior data and refrigerator safety risk level.

[0129] Through the coordinated cooperation of the above components, the high-frame rate image sequence collected in the area in front of the refrigerator door is adaptively histogram equalized and geometrically corrected, which effectively solves the problems of unstable light and image distortion in the refrigerator use environment, and significantly improves the accuracy and robustness of subsequent face detection; secondly, the face detection method based on multi-level sliding window scanning can accurately locate the face boundary box and facial key points, laying the foundation for subsequent feature extraction; thirdly, by generating multi-angle samples and extracting features of the target face area, the high-dimensional face feature vector obtained captures the user's facial features more comprehensively and enhances the reliability of identity authentication; and liveness detection is performed through the preset multimodal information library, which effectively prevents impersonation and deception and improves the security of the system; by introducing user physiological cycle data and family member interaction mode for access permission analysis, this method realizes more refined and personalized permission management and meets the differentiated needs of different family members for refrigerator use; finally, the real-time collected user refrigerator interaction behavior sequence is analyzed for time series characteristics to obtain abnormal behavior data and refrigerator safety risk level, which greatly improves the safety performance of smart refrigerators. The present invention can not only accurately identify abnormal usage behaviors, but also timely warn of potential safety risks, such as children's accidental ingestion and food spoilage, providing new protection for family safety. In short, by comprehensively considering the unique usage environment and user behavior characteristics of refrigerators, high-precision identity authentication and intelligent authority management are achieved, which improves the efficiency and accuracy of face detection in smart digital safety refrigerators.

[0130] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the smart digital safety refrigerator face detection method.

[0131] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0132] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for face detection in an intelligent digital safety refrigerator, characterized in that: The intelligent digital safety refrigerator face detection method comprises: Adaptive histogram equalization and geometric correction are performed on the high frame rate image sequence collected from the door front area of ​​the target refrigerator to obtain a standardized image; Performing face detection based on multi-level sliding window scanning on the standardized image to obtain a face bounding box and coordinates of five facial key points; Performing region segmentation on the standardized image based on the face bounding box and the coordinates of the five facial key points to obtain a target face region, and performing multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector; Calculate the similarity of the high-dimensional facial feature vector, and perform liveness detection through a preset multimodal information library to obtain a verification token containing a user ID, a confidence score and a timestamp; Through the pre-collected user physiological cycle data and family member interaction pattern, the verification token is analyzed for access rights to obtain an access rights table, including: performing refrigerator door opening time period distribution statistics on the verification token to obtain user time preference data, and performing refrigerator storage area access frequency correlation analysis on the user time preference data to obtain a regional access tendency map; performing refrigerator internal temperature change correlation calculation on the regional access tendency map to obtain temperature control habit characteristics, and matching the temperature control habit characteristics with refrigerator energy consumption data to obtain a user energy-saving tendency index; performing refrigerator food storage cycle correlation analysis on the user energy-saving tendency index to obtain dietary regularity characteristics; performing refrigerator door opening and closing frequency weighting on the dietary regularity characteristics to obtain user life and work schedule data, and performing refrigerator internal light change pattern recognition on the user life and work schedule data to obtain diurnal activity characteristics; performing refrigerator defrost cycle synchronization analysis on the diurnal activity characteristics to obtain user home time estimation data, and performing refrigerator usage peak overlap calculation on the user home time estimation data to obtain a family member interaction intensity map; performing refrigerator intelligent scene pattern matching on the family member interaction intensity map to obtain the access rights table; Based on the access permission table, a time series feature analysis is performed on the user's refrigerator interaction behavior sequence collected in real time to obtain abnormal behavior data and refrigerator safety risk level.

2. The intelligent digital safety refrigerator face detection method according to claim 1 is characterized in that: The method of performing adaptive histogram equalization and geometric correction on the high frame rate image sequence collected from the door front area of ​​the target refrigerator to obtain a standardized image includes: Adaptively sampling the refrigerator door opening frequency on the high frame rate image sequence to obtain a dynamic key frame set, and eliminating the light interference of the refrigerator compartment on the dynamic key frame set to obtain an initial denoised image set; Applying a refrigerator door internal and external temperature difference compensation algorithm to the initial noise reduction image set to perform fog correction to obtain a definition-enhanced image, and performing refrigerator door frame surface distortion calibration on the definition-enhanced image to obtain a geometrically corrected image; Performing freezer compartment frost compensation on the geometrically corrected image to obtain a contrast-enhanced image, and performing refrigerator door handle shadow elimination processing on the contrast-enhanced image to obtain a shadow-balanced image; Performing refrigerator-stored food color interference suppression on the shadow-balanced image to obtain a color-pure image, and performing refrigerator-feature-preserving edge enhancement on the color-pure image to obtain an edge-sharpened image; Performing refrigerator lighting non-uniformity adaptive correction on the edge-sharpened image to obtain a lighting balanced image; The illumination balanced image is adaptively cropped according to the refrigerator usage scenario to obtain the standardized image.

3. The intelligent digital safety refrigerator face detection method according to claim 1, characterized in that: The face detection based on multi-level sliding window scanning is performed on the standardized image to obtain a face bounding box and coordinates of five facial key points, including: Adaptively scaling the standardized image according to the refrigerator door opening angle to obtain a multi-scale image pyramid, and compensating the multi-scale image pyramid for changes in refrigerator ambient light to obtain an illumination invariant feature map; Shielding the refrigerator door handle area of ​​the illumination invariant feature map to obtain an optimized candidate area, and performing a sliding window scan weighted by refrigerator usage frequency on the optimized candidate area to obtain a preliminary face position estimate; Eliminating the interference of refrigerator door frame reflection on the preliminary face position estimation to obtain a fine face area, and correcting the temperature gradient inside the refrigerator on the fine face area to obtain a temperature-balanced face image; Performing adaptive boundary adjustment of the refrigerator door opening and closing speed on the temperature-balanced face image to obtain a dynamic boundary frame, and suppressing the color interference of food stored in the refrigerator on the dynamic boundary frame to obtain a pure face area; The shadow compensation of the refrigerator-specific light source is performed on the pure face area to obtain a shadow-balanced face image, and the refrigerator user height-adaptive key point positioning is performed on the shadow-balanced face image to obtain the face boundary box and the coordinates of five facial key points.

4. The intelligent digital safety refrigerator face detection method according to claim 1, characterized in that: The method of performing region segmentation on the standardized image based on the face bounding box and the coordinates of the five facial key points to obtain a target face region, and performing multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector includes: Adaptively adjusting the opening and closing angle of the refrigerator door to the face boundary frame to obtain a dynamic face area, and eliminating the reflection of the light source inside the refrigerator to obtain a de-reflected face image; Perform refrigerator temperature change compensation on the de-reflected face image to obtain a temperature-balanced face image, and perform refrigerator-stored food odor interference correction on the temperature-balanced face image to obtain a pure face area; Perform multi-angle rotation on the pure face area weighted by refrigerator usage frequency to obtain a multi-view face sample set, and perform refrigerator door handle occlusion simulation on the multi-view face sample set to obtain an occlusion enhanced sample; Performing temperature gradient feature enhancement on the occlusion enhancement sample to obtain a temperature feature map, and performing refrigerator door opening frequency adaptive feature extraction on the temperature feature map to obtain a time-varying feature set; The time-varying feature set is subjected to a refrigerator user habit correlation analysis to obtain a user feature map, and the user feature map is subjected to refrigerator environmental noise robustness enhancement to obtain the high-dimensional face feature vector.

5. The intelligent digital safety refrigerator face detection method according to claim 1 is characterized in that: The similarity calculation is performed on the high-dimensional facial feature vector, and liveness detection is performed through a preset multimodal information library to obtain a verification token containing a user ID, a confidence score and a timestamp, including: The high-dimensional face feature vector is weighted by the frequency of opening the refrigerator door to obtain a time series enhancement feature, and the time series enhancement feature is compensated for the temperature change inside the refrigerator to obtain a temperature invariant feature; Compensating the airflow disturbance caused by opening and closing the refrigerator door for the temperature-invariant feature to obtain a stable facial feature, and performing a refrigerator usage time period correlation analysis on the stable facial feature to obtain a time-sensitive feature; Performing similarity calculation on the time-sensitive feature by using an additive angular margin loss function to obtain an initial matching result; The initial matching result is verified by the refrigerator door handle contact pattern to obtain a tactile enhancement matching score, and the tactile enhancement matching score is adaptively adjusted to the refrigerator light change to obtain a lighting robust matching value; Performing a refrigerator usage habit consistency check on the illumination robust matching value to obtain a behavior verification result; The behavior verification result is compensated for the change of humidity inside the refrigerator to obtain the user ID, and the user ID is fused with the refrigerator door opening force feature to obtain the verification token, wherein the verification token includes: user ID, confidence score and timestamp.

6. The intelligent digital safety refrigerator face detection method according to claim 1, characterized in that: Based on the access permission table, the real-time collected user refrigerator interaction behavior sequence is subjected to time series feature analysis to obtain abnormal behavior data and refrigerator safety risk level, including: Performing a refrigerator door opening and closing frequency analysis on the user refrigerator interaction behavior sequence to obtain a door opening mode feature, and performing a refrigerator storage area temperature fluctuation correlation calculation on the door opening mode feature to obtain a temperature change trajectory; Matching the temperature change trajectory with the peak time period of refrigerator energy consumption to obtain energy consumption behavior characteristics, and performing refrigerator lighting usage statistics on the energy consumption behavior characteristics to obtain lighting demand data; Analyzing the humidity change trend inside the refrigerator based on the light demand data to obtain a storage environment index; Calculating the refrigerator defrosting cycle deviation based on the storage environment index to obtain a maintenance demand warning, and performing a refrigerator food access sequence correlation analysis on the maintenance demand warning to obtain user habit characteristics; Detecting changes in contact pressure of the refrigerator door handle based on the user's habit characteristics to obtain operation force data; The operation force data is subjected to a numerical fluctuation analysis of the refrigerator internal odor sensor to obtain a storage abnormality index, and the storage abnormality index is compared with the access permission table to perform a refrigerator safety threshold to obtain the abnormal behavior data and the refrigerator safety risk level.

7. An intelligent digital safety refrigerator face detection system, characterized in that: Used to perform the intelligent digital safety refrigerator face detection method according to any one of claims 1 to 6, the intelligent digital safety refrigerator face detection system comprises: A correction module, used for performing adaptive histogram equalization and geometric correction on a high frame rate image sequence collected from the door front area of ​​the target refrigerator to obtain a standardized image; A detection module, used for performing face detection on the standardized image based on multi-level sliding window scanning to obtain a face bounding box and coordinates of five facial key points; A segmentation module, used for performing regional segmentation on the standardized image based on the face bounding box and the coordinates of the five facial key points to obtain a target face region, and performing multi-angle sample generation and feature extraction on the target face region to obtain a high-dimensional face feature vector; A calculation module, used to perform similarity calculation on the high-dimensional facial feature vector, and perform liveness detection through a preset multimodal information library to obtain a verification token including a user ID, a confidence score and a timestamp; An analysis module, configured to perform access permission analysis on the verification token based on pre-collected user physiological cycle data and family member interaction patterns, and obtain an access permission table; The early warning module is used to perform time series feature analysis on the user refrigerator interaction behavior sequence collected in real time based on the access permission table to obtain abnormal behavior data and refrigerator safety risk level.

Images