An intelligent mirror cabinet control method and system based on user recognition

Abstract

An intelligent mirror cabinet control method and system based on user recognition provided by the present invention. The method includes determining an image of a user in the intelligent mirror using an image generation model based on an image when the user approaches the intelligent mirror; determining the user's name, the ambient light brightness, and the display range of the user's body in the intelligent mirror based on the image of the user in the intelligent mirror; determining the left display area of the application, the right display area of the application, and the upper display area of the application using a display area determination model based on the user's physiological information corresponding to the user's name, the ambient light brightness, and the display range of the user's body in the intelligent mirror; determining a plurality of target applications and the display positions of each target application in the plurality of target applications in the intelligent mirror based on a graph neural network model. This method can improve the comfort of the display of application icons during the user's use process and improve the user experience.

Description

[0001] Division Explanation

[0002] This application is a divisional application filed in China based on the Chinese application with an application date of November 21, 2023, an application number of 202311561732.3, and an invention title of "Intelligent Mirror Cabinet Control Method and System Based on Human Sensing". Technical Field

[0003] The present invention relates to the technical field of intelligent mirrors, and specifically relates to an intelligent mirror cabinet control method and system based on user recognition. Background Art

[0004] An intelligent mirror is a mirror integrated with various intelligent functions, such as displaying weather information, time information, news, applications, etc. These functions make the intelligent mirror an indispensable part of modern home and office environments.

[0005] However, when the user looks at the mirror and the intelligent mirror needs to display application icons simultaneously, the user's image in the mirror is often blocked by the displayed application icons, resulting in an unclear or blocked display of the user's image in the mirror. Existing solutions usually display the application icons at the edge or corner of the mirror. However, since the application icons are displayed in the corner, which is usually not the main area of concern for the user, it is difficult for the user to find them, and the corner position of the mirror is not easily accessible, which may lead to difficult user operation and poor user experience.

[0006] Therefore, how to improve the comfort of the application icon display during user use and improve the user experience is an urgent problem to be solved currently. Summary of the Invention

[0007] The main technical problem to be solved by the present invention is how to improve the comfort of the application icon display during user use and improve the user experience.

[0008] According to a first aspect, the present invention provides an intelligent mirror cabinet control method based on user recognition, including: obtaining an image when a user approaches the intelligent mirror; using an image generation model to determine an image of the user in the intelligent mirror based on the image when the user approaches the intelligent mirror; determining the user's name, ambient light brightness, and the display range of the user's body in the intelligent mirror based on the image of the user in the intelligent mirror; obtaining the user's physiological information corresponding to the user's name; using a display area determination model to determine the left display area of the application, the right display area of the application, and the upper display area of the application based on the user's physiological information corresponding to the user's name, the ambient light brightness, and the display range of the user's body in the intelligent mirror; determining the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness; determining the number of applications to be displayed based on the display size of the application icon, the left display area of the application, the right display area of the application, and the upper display area of the application; obtaining multiple application information in the intelligent mirror, where each application information in the multiple application information in the intelligent mirror includes the user's historical behavior data, category information, usage frequency, user rating, update time, and application size in the application; constructing multiple nodes and multiple edges between the multiple nodes based on the multiple application information in the intelligent mirror, where each node in the multiple nodes represents an application, each node includes multiple node features, and the node features of each node include the number of applications to be displayed, the left display area of the application, the right display area of the application, the upper display area of the application, category information, usage frequency, user rating, update time, and application size, and the edge features of each edge in the multiple edges include the number of co-occurrences of two applications in user operations and the similarity between two applications; processing the multiple nodes and the multiple edges between the multiple nodes using a graph neural network model to determine multiple target applications and the display positions of each target application in the intelligent mirror.

[0009] Further, the user's physiological information corresponding to the user's name includes arm length, eyesight, age, and gender.

[0010] Further, the image generation model is a convolutional neural network model, the input of the image generation model is the image when the user approaches the intelligent mirror, and the output of the image generation model is the image of the user in the intelligent mirror.

[0011] Further, the method further includes: if the ambient light brightness is less than a threshold, reminding the user to turn on the light.

[0012] Further, the input of the graph neural network model is the multiple nodes and multiple edges between the multiple nodes, and the output of the graph neural network model is the multiple target applications and the display positions of each target application in the intelligent mirror.

[0013] According to a second aspect, the present invention provides an intelligent mirror cabinet control system based on user recognition, including: a first acquisition module, configured to acquire an image when a user approaches the intelligent mirror;

[0014] An image generation module, configured to determine an image of the user in the intelligent mirror based on the image when the user approaches the intelligent mirror using an image generation model;

[0015] A display range determination module, configured to determine the user name, the ambient light brightness, and the display range of the user's body in the intelligent mirror based on the image of the user in the intelligent mirror;

[0016] A second acquisition module, configured to acquire the user's physiological information corresponding to the user name;

[0017] A display area determination module, configured to determine the left display area, the right display area, and the upper display area of the application based on the user's physiological information corresponding to the user name, the ambient light brightness, and the display range of the user's body in the intelligent mirror using a display area determination model;

[0018] An icon determination module, configured to determine the display size of the application icon based on the left display area, the right display area, and the upper display area of the application, the user's physiological information corresponding to the user name, and the ambient light brightness;

[0019] A display quantity determination module, configured to determine the display quantity of the application based on the display size of the application icon, the left display area, the right display area, and the upper display area of the application;

[0020] A third acquisition module, configured to acquire multiple application information in the intelligent mirror, where each application information in the multiple application information in the intelligent mirror includes the user's historical behavior data, category information, usage frequency, user rating, update time, and application size;

[0021] A graph structure module, configured to construct multiple nodes and multiple edges between the multiple nodes based on multiple application information in the smart mirror. Each node in the multiple nodes represents an application, and each node includes multiple node features. The node features of each node include the number of application displays, the left display area of the application, the right display area of the application, the upper display area of the application, category information, usage frequency, user rating, update time, and application size. The edge features of each edge in the multiple edges include the number of co-occurrences of two applications in user operations and the similarity between two applications.

[0022] A graph neural network model processing module, configured to process the multiple nodes and the multiple edges between the multiple nodes based on a graph neural network model to determine multiple target applications and the display positions of each target application in the multiple target applications in the smart mirror.

[0023] Furthermore, the user physiological information corresponding to the user name includes arm length, eyesight, age, and gender.

[0024] Furthermore, the image generation model is a convolutional neural network model. The input of the image generation model is the image of the user when approaching the smart mirror, and the output of the image generation model is the image of the user in the smart mirror.

[0025] Furthermore, the system is further configured to: if the environmental light brightness is less than a threshold, remind the user to turn on the light.

[0026] Furthermore, the input of the graph neural network model is the multiple nodes and the multiple edges between the multiple nodes, and the output of the graph neural network model is the multiple target applications and the display positions of each target application in the multiple target applications in the smart mirror.

[0027] An intelligent mirror cabinet control method and system based on user recognition provided by the present invention. The method includes obtaining an image when a user approaches the intelligent mirror; using an image generation model to determine the image of the user in the intelligent mirror based on the image when the user approaches the intelligent mirror; determining the user's name, ambient light brightness, and the display range of the user's body in the intelligent mirror based on the image of the user in the intelligent mirror; obtaining the user's physiological information corresponding to the user's name; using a display area determination model to determine the left display area of the application, the right display area of the application, and the upper display area of the application based on the user's physiological information corresponding to the user's name, the ambient light brightness, and the display range of the user's body in the intelligent mirror; determining the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness; determining the number of applications to be displayed based on the display size of the application icon, the left display area of the application, the right display area of the application, and the upper display area of the application; obtaining multiple application information in the intelligent mirror, where each application information in the multiple application information in the intelligent mirror includes the user's historical behavior data, category information, usage frequency, user rating, update time, and application size in the application; constructing multiple nodes and multiple edges between the multiple nodes based on the multiple application information in the intelligent mirror, where each node in the multiple nodes represents an application, each node includes multiple node features, and the node features of each node include the number of applications to be displayed, the left display area of the application, the right display area of the application, the upper display area of the application, category information, usage frequency, user rating, update time, and application size, and the edge feature of each edge in the multiple edges includes the number of co-occurrences of two applications in user operations and the similarity between two applications; processing the multiple nodes and the multiple edges between the multiple nodes using a graph neural network model to determine multiple target applications and the display positions of each target application in the intelligent mirror. This method can improve the comfort of application icon display during user use and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic flowchart of an intelligent mirror cabinet control method based on user recognition provided by an embodiment of the present invention;

[0029] Figure 2 It is a schematic diagram of the display area of the application icon provided by an embodiment of the present invention;

[0030] Figure 3 It is a schematic diagram of an intelligent mirror cabinet control system based on user recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] In an embodiment of the present invention, it is provided asFigure 1 An intelligent mirror cabinet control method based on user recognition as shown, the intelligent mirror cabinet control method based on user recognition includes steps S1 to S10:

[0032] Step S1, obtain an image of the user when approaching the intelligent mirror.

[0033] When the user approaches the intelligent mirror, the camera installed in the intelligent mirror will automatically trigger the photographing function to obtain the user's image. For example, when the user approaches the intelligent mirror and stands directly in front of it, the camera will take a photo of the user as the image of the user when approaching the intelligent mirror. As an example, the intelligent mirror can be equipped with an infrared sensor. When the user approaches the intelligent mirror, the infrared sensor will detect the user's approach, thereby triggering the automatic trigger of the photographing function.

[0034] Step S2, use an image generation model to determine the user's image in the intelligent mirror based on the image of the user when approaching the intelligent mirror.

[0035] The user's image in the intelligent mirror is the image of the user himself seen in the mirror simulated by the image generation model according to the photographed image of the user when approaching the intelligent mirror. That is, the image of the user when approaching the intelligent mirror is a real existing image, while the user's image in the intelligent mirror is the image of the user seen in the intelligent mirror simulated from the user's perspective. The user's image in the intelligent mirror is the reflected image observed by the user when using the intelligent mirror.

[0036] The image generation model is a convolutional neural network model. The convolutional neural network (CNN) can be a multi-layer neural network (for example, including at least two layers). The at least two layers can include at least one of a convolutional layer (CONV), a rectified linear unit (ReLU) layer, a pooling layer (POOL), or a fully connected layer (FC). The convolutional neural network can extract useful features from the image and gradually understand and learn the context information of the image. The image generation model is a convolutional neural network model. The input of the image generation model is the image of the user when approaching the intelligent mirror, and the output of the image generation model is the user's image in the intelligent mirror. The convolutional neural network model learns the statistical laws and feature representations of the image through the training process and performs image reconstruction. Through a large number of training samples and optimization algorithms, the model can learn a more accurate and realistic image generation ability to generate the user's image in the intelligent mirror.

[0037] Step S3, determine the user's name, the ambient light brightness, and the display range of the user's body in the intelligent mirror based on the user's image in the intelligent mirror.

[0038] In some embodiments, the user name, the ambient light brightness, and the display range of the user's body in the smart mirror can be determined based on a user recognition model. The input of the user recognition model is the image of the user in the smart mirror, and the output of the user recognition model is the user name, the ambient light brightness, and the display range of the user's body in the smart mirror. The user recognition model is a convolutional neural network model. The user recognition model is a convolutional neural network model. The image of the user in the smart mirror includes ambient light brightness information and display range information of the user's body in the smart mirror. The display range of the user's body in the smart mirror can be obtained by recognizing the image of the user in the smart mirror through the user recognition model.

[0039] In some embodiments, the user recognition model includes a background segmentation layer, a brightness determination layer, and a user information determination layer. The background segmentation layer, the brightness determination layer, and the user information determination layer are all convolutional neural networks. The input of the background segmentation layer is the image of the user in the smart mirror, the output of the background segmentation layer is the segmented background image and the segmented user body image, the input of the brightness determination layer is the segmented background image, the output of the brightness determination layer is the ambient light brightness, the input of the user information determination layer is the segmented user body image, and the output of the user information determination layer is the user name and the display range of the user's body in the smart mirror.

[0040] The display range of the user's body in the smart mirror refers to the area where the user can see their own body in the smart mirror. By determining the display range of the user's body in the smart mirror, the display area of the application can be accurately distributed beside the display range of the user's body in the smart mirror when determining the display area of the application subsequently.

[0041] In some embodiments, the method further includes: if the ambient light brightness is less than a threshold value, reminding the user to turn on the light.

[0042] Step S4, obtaining the user's physiological information corresponding to the user name.

[0043] The user's physiological information corresponding to the user name includes arm length, eyesight, age, and gender.

[0044] The arm length can help determine the display range on the smart mirror. As an example, the longer the user's arm length, the wider the display range can be, enabling the user to reach more applications during operation.

[0045] The user's eyesight condition has an important impact on the display range of the smart mirror. As an example, the worse the user's eyesight, the smaller the display range of the smart mirror, the fewer the number of displayed applications, and the larger the display size of the application icons can be.

[0046] Age and gender can also affect the display range and content of the smart mirror to a certain extent. For example, for young users, a larger display range can be adopted to display more content; while for elderly users, a smaller display range and less content can be adopted.

[0047] Step S5: Based on the user's physiological information corresponding to the user name, the ambient light brightness, and the display range of the user's body in the smart mirror, use the display area determination model to determine the left display area of the application, the right display area of the application, and the upper display area of the application.

[0048] The left display area of the application refers to the display area on the smart mirror screen that is located on the left side of the display range of the user's body in the smart mirror and is used to display application icons.

[0049] The right display area of the application refers to the display area on the smart mirror screen that is located on the right side of the display range of the user's body in the smart mirror and is used to display application icons.

[0050] The upper display area of the application refers to the display area on the smart mirror screen that is located above the display range of the user's body in the smart mirror and is used to display application icons.

[0051] The display area determination model is a convolutional neural network model. The input of the display area determination model is the user's physiological information corresponding to the user name, the ambient light brightness, and the display range of the user's body in the smart mirror, and the output of the display area determination model is the left display area of the application, the right display area of the application, and the upper display area of the application.

[0052] The ambient light brightness also affects the display area of the application. For example, if the ambient light brightness is greater, the display area can be larger to display more content. Another example is that if the ambient light brightness is lower, the display area can be smaller and the content displayed can be less, so that the user can accurately identify the content in the display area.

[0053] As an example, Figure 2 is a schematic diagram of the display area of the application icon provided by the embodiment of the present invention. As Figure 2 shown, Figure 2 represents the smart mirror. Through Figure 2 it can be seen that the left display area of the application, the right display area of the application, and the upper display area of the application are respectively displayed beside the user, with a good display effect and also avoiding the occlusion of the application icons to the user himself.

[0054] The display area determination model can comprehensively consider the user's physiological information corresponding to the user name, the ambient light brightness, the display range of the user's body in the smart mirror, and effectively process the image data. Finally, it helps to determine the display area of the application by learning and extracting features. In some embodiments, the display area determination model can be trained by the gradient descent method to obtain the display area determination model.

[0055] Step S6: Determine the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user name, and the ambient light brightness.

[0056] In some embodiments, the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user name, and the ambient light brightness can be constructed into a vector to be matched. By calculating the distance between this vector to be matched and each reference vector in the database, the display size of the application icon corresponding to the reference vector with a distance less than the threshold is determined as the display size of the application icon. The database is pre-constructed and includes reference vectors and the display sizes of the application icons corresponding to the reference vectors. The reference vectors are constructed based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user name, and the ambient light brightness in the historical data. The display size of the application icon corresponding to the reference vector is the display size of the application icon determined in the historical data.

[0057] In some embodiments, a deep neural network model can be used to determine the display size of the application icon. The input of the deep neural network model is the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user name, and the ambient light brightness, and the output of the deep neural network model is the display size of the application icon.

[0058] Step S6 can adjust the display size of the application icon according to the user's physiological characteristics and environmental conditions, so as to provide a more personalized and comfortable user experience.

[0059] Step S7: Determine the number of applications to be displayed based on the display size of the application icon, the left display area of the application, the right display area of the application, and the upper display area of the application.

[0060] In some embodiments, the sum of the display areas of the left display area of the application, the right display area of the application, and the upper display area of the application can be added up and then divided by the display size of the application icon to obtain the number of applications to be displayed. The number of applications to be displayed is the total number of applications displayed in the display area of the application.

[0061] In some embodiments, the display areas of the left display area, the right display area, and the upper display area of the application can be added together to obtain the total display area. The number of applications to be displayed can be determined by querying a preset table of the number of applications to be displayed. The preset table of the number of applications to be displayed includes the total display area of each type and the corresponding number of applications to be displayed for each total display area, and the preset table of the number of applications to be displayed can be constructed manually based on historical data.

[0062] Step S8: Obtain multiple pieces of application information in the smart mirror. Each piece of application information in the multiple pieces of application information in the smart mirror includes the user's historical behavior data, category information, usage frequency, user rating, update time, and application size in the application.

[0063] The user's historical behavior data in the application refers to the operation records and behavior data of the user in a specific application, such as browsing history, click behavior, purchase records, etc.

[0064] The category information indicates the category or classification to which the application belongs, such as social, entertainment, tools, etc.

[0065] The usage frequency refers to the frequency of the user's use of a specific application, usually measured by the number of times or time periods.

[0066] The user rating reflects the user's satisfaction and evaluation of the application, usually presented in the form of stars or scores.

[0067] The update time refers to the timestamp or date of the most recent update of the application program.

[0068] The application size represents the size of the storage space occupied by the application program, usually in MB or GB.

[0069] By collecting this information, the smart mirror can better understand the user's preferences and behavior habits, and at the same time can also provide more personalized and efficient application recommendation and management services for the user.

[0070] In some embodiments, the relevant information of each application can be extracted from the application management system or database of the smart mirror.

[0071] Step S9: Based on the multiple pieces of application information in the smart mirror, construct multiple nodes and multiple edges between the multiple nodes. Each node in the multiple nodes represents an application, and each node includes multiple node features. The node features of each node include the number of applications to be displayed, the left display area of the application, the right display area of the application, the upper display area of the application, category information, usage frequency, user rating, update time, and application size. The edge features of each edge in the multiple edges include the number of times two applications co-occur in user operations and the similarity between the two applications.

[0072] The co-occurrence count of two applications in user operations represents the number of times the two applications are simultaneously opened and operated during the user's use of the smart mirror. As an example, assume that the user has performed 10 operations in the past week, and in 3 of these operations, both the music player and the weather forecast application were opened. Then the co-occurrence count of the two applications is 3.

[0073] In some embodiments, the co-occurrence count of two applications in user operations can be determined through the following steps, which include: collecting user operation data, including the applications used by the user and the usage times. For each user, count the number of times the two applications appear simultaneously. If the two applications appear simultaneously in the same user operation, increment the count by one. Finally, obtain the co-occurrence count of the two applications in user operations.

[0074] The similarity between two applications represents the degree of similarity between the two applications. In some embodiments, the SimHash values of the two application information can be calculated separately, and then the similarity of the SimHash values of the two application information can be calculated through the Hamming distance. The smaller the Hamming distance, the higher the similarity. The application information includes the user's historical behavior data, category information, usage frequency, user ratings, update time, and application size.

[0075] The steps for calculating the SimHash value of an application can include word segmentation, hash calculation, weighting, merging, dimensionality reduction, etc. For example, word segmentation: First, segment the application information to extract the feature vectors. And set the weights for the feature vectors; hash calculation: Calculate the hash values of each feature vector through a hash function, and the hash value is an n-bit signature composed of binary numbers 0 and 1; weighting: On the basis of the hash value, weight all the feature vectors, that is, W = Hash * weight, and when encountering 1, the hash value and the weight are multiplied positively, and when encountering 0, the hash value and the weight are multiplied negatively; merging: Accumulate the weighted results of the above feature vectors into a sequence string; dimensionality reduction: For the accumulated result, if it is greater than 0, set it to 1, otherwise set it to 0, so as to obtain the SimHash value of the statement.

[0076] By calculating the co-occurrence count of two applications and the similarity of the application feature vectors, the degree of association and similarity between them can be quantified. These metrics can be used as inputs to a graph neural network model to recommend applications suitable for the user, providing a better experience and decision-making support for the user.

[0077] The purpose of performing this step is to achieve an in-depth understanding of the relationship between user behavior and applications by constructing an association network between applications, so as to provide more accurate application recommendations for the smart mirror.

[0078] Step S10, process multiple nodes and multiple edges between the multiple nodes based on a graph neural network model to determine multiple target applications and the display position of each target application in the intelligent mirror.

[0079] In graph-structured data, a node represents an entity or concept, and each node contains a set of features used to describe the attributes or status of the entity. Here, a node refers to an application in the intelligent mirror, and each node includes various feature information of the application. An edge in graph-structured data represents the relationship between nodes.

[0080] The graph neural network model includes a Graph Neural Network (GNN) and a fully connected layer. The graph neural network is a neural network that directly acts on graph-structured data, which is a data structure composed of nodes and edges. The graph neural network is a deep learning model designed for graph-structured data, and it can effectively capture the features of nodes and edges in the graph and the relationships between them. Multiple application nodes and multiple edges in the intelligent mirror can form a graph structure, and the graph neural network is applied for processing on this structure.

[0081] The input of the graph neural network model is the multiple nodes and multiple edges between the multiple nodes, and the output of the graph neural network model is the multiple target applications and the display position of each target application in the intelligent mirror.

[0082] Each node represents an application and contains multiple node features, such as the number of application displays, the left display area, the right display area, the upper display area, category information, usage frequency, user rating, update time, and application size, etc. These node features provide detailed information about the application and can help the model understand the attributes and features of the application. Each edge represents the relationship between two applications, and the edge features include the number of co-occurrences of the two applications in user operations and the similarity between the two applications. These edge features provide interaction and similarity information between applications and help the model understand the correlation and connection method between applications. Through the learning and reasoning process of the graph neural network model, multiple target applications and their display positions in the intelligent mirror can be determined based on the node embedding vectors and edge weight information. The graph neural network model can sort, filter, and locate multiple applications by considering the relationships, similarities, and other context information between nodes, thereby determining the target applications and ultimately achieving the purpose of optimizing the display effect.

[0083] Based on the same inventive concept, Figure 3 A schematic diagram of an intelligent mirror cabinet control system based on user recognition provided by an embodiment of the present invention. The intelligent mirror cabinet control system based on user recognition includes:

[0084] The first acquisition module 31 is configured to acquire an image when the user approaches the smart mirror;

[0085] The image generation module 32 is configured to determine the user's image in the smart mirror based on the image when the user approaches the smart mirror using an image generation model;

[0086] The display range determination module 33 is configured to determine the user's name, the ambient light brightness, and the display range of the user's body in the smart mirror based on the user's image in the smart mirror;

[0087] The second acquisition module 34 is configured to acquire the user's physiological information corresponding to the user's name;

[0088] The display area determination module 35 is configured to determine the left display area, the right display area, and the upper display area of the application based on the user's physiological information corresponding to the user's name, the ambient light brightness, and the display range of the user's body in the smart mirror using a display area determination model;

[0089] The icon determination module 36 is configured to determine the display size of the application icon based on the left display area, the right display area, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness;

[0090] The display quantity determination module 37 is configured to determine the display quantity of the application based on the display size of the application icon, the left display area, the right display area, and the upper display area of the application;

[0091] The third acquisition module 38 is configured to acquire multiple application information in the smart mirror, and each application information in the multiple application information in the smart mirror includes the user's historical behavior data, category information, usage frequency, user rating, update time, and application size in the application;

[0092] The graph structure module 39 is configured to construct multiple nodes and multiple edges between the multiple nodes based on the multiple application information in the smart mirror. Each node in the multiple nodes represents an application, and each node includes multiple node features. The node features of each node include the application display quantity, the left display area, the right display area, the upper display area of the application, category information, usage frequency, user rating, update time, and application size. The edge feature of each edge in the multiple edges includes the co-occurrence times of two applications in the user's operation and the similarity between two applications;

[0093] The graph neural network model processing module 40 is used to process multiple nodes and multiple edges between the multiple nodes based on the graph neural network model to determine multiple target applications and the display positions of each target application in the multiple target applications in the smart mirror.

Claims

1. An intelligent mirror cabinet control method based on user identification, characterized in that Including: Obtain an image of the user when approaching the smart mirror; Determine the image of the user in the smart mirror based on the image of the user when approaching the smart mirror using an image generation model; Determine the user's name, the ambient light brightness, and the display range of the user's body in the smart mirror based on the image of the user in the smart mirror; Obtain the user's physiological information corresponding to the user's name; Determine the left display area of the application, the right display area of the application, and the upper display area of the application based on the user's physiological information corresponding to the user's name, the ambient light brightness, and the display range of the user's body in the smart mirror using a display area determination model; Determine the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness. Determining the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness includes: using a deep neural network model to determine the display size of the application icon, where the input of the deep neural network model is the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness, and the output of the deep neural network model is the display size of the application icon; Determine the number of applications to be displayed based on the display size of the application icon, the left display area of the application, the right display area of the application, and the upper display area of the application; Obtain multiple application information in the smart mirror, where each application information in the multiple application information in the smart mirror includes the user's historical behavior data in the application, category information, usage frequency, user rating, update time, and application size; Construct multiple nodes and multiple edges between the multiple nodes based on the multiple application information in the smart mirror. Each node in the multiple nodes represents an application, and each node includes multiple node features. The node features of each node include the number of applications to be displayed, the left display area of the application, the right display area of the application, the upper display area of the application, category information, usage frequency, user rating, update time, and application size. The edge features of each edge in the multiple edges include the number of co-occurrences of two applications in the user's operation and the similarity between the two applications; Process the multiple nodes and the multiple edges between the multiple nodes using a graph neural network model to determine multiple target applications and the display positions of each target application in the smart mirror.

2. The intelligent mirror cabinet control method based on user identification according to claim 1, wherein The user's physiological information corresponding to the user's name includes arm length, eyesight, age, and gender.

3. The intelligent mirror cabinet control method based on user identification according to claim 1, wherein, The image generation model is a convolutional neural network model. The input of the image generation model is the image of the user when approaching the smart mirror, and the output of the image generation model is the image of the user in the smart mirror.

4. The intelligent mirror cabinet control method based on user identification according to claim 1, wherein, The method further includes: if the ambient light brightness is less than a threshold, remind the user to turn on the light.

5. The intelligent mirror cabinet control method based on user identification according to claim 1, wherein The input of the graph neural network model is the multiple nodes and multiple edges between the multiple nodes, and the output of the graph neural network model is the multiple target applications and the display positions of each target application in the smart mirror.

6. An intelligent mirror cabinet control system based on user identification, characterized in that, Including: A first acquisition module, configured to acquire an image when the user approaches the smart mirror; An image generation module, configured to determine the user's image in the smart mirror based on the image when the user approaches the smart mirror using an image generation model; A display range determination module, configured to determine the user's name, the ambient light brightness, and the display range of the user's body in the smart mirror based on the user's image in the smart mirror; A second acquisition module, configured to acquire the user's physiological information corresponding to the user's name; A display area determination module, configured to determine the left display area of the application, the right display area of the application, and the upper display area of the application based on the user's physiological information corresponding to the user's name, the ambient light brightness, and the display range of the user's body in the smart mirror using a display area determination model; An icon determination module, configured to determine the display size of the application icon based on the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness. The icon determination module is further configured to: use a deep neural network model to determine the display size of the application icon. The input of the deep neural network model is the left display area of the application, the right display area of the application, the upper display area of the application, the user's physiological information corresponding to the user's name, and the ambient light brightness, and the output of the deep neural network model is the display size of the application icon; A display quantity determination module, configured to determine the display quantity of the application based on the display size of the application icon, the left display area of the application, the right display area of the application, and the upper display area of the application; A third acquisition module, configured to acquire multiple application information in the smart mirror. Each application information in the multiple application information in the smart mirror includes the user's historical behavior data in the application, category information, usage frequency, user rating, update time, and application size; A graph structure module, configured to construct multiple nodes and multiple edges between the multiple nodes based on the multiple application information in the smart mirror. Each node in the multiple nodes represents an application, and each node includes multiple node features. The node features of each node include the display quantity of the application, the left display area of the application, the right display area of the application, the upper display area of the application, category information, usage frequency, user rating, update time, and application size. The edge feature of each edge in the multiple edges includes the co-occurrence times of two applications in the user's operation and the similarity between two applications; A graph neural network model processing module, configured to process the multiple nodes and the multiple edges between the multiple nodes based on the graph neural network model to determine multiple target applications and the display positions of each target application in the smart mirror.

7. The intelligent mirror cabinet control system based on user identification according to claim 6, wherein The user's physiological information corresponding to the user's name includes arm length, eyesight, age, and gender.

8. The intelligent mirror cabinet control system based on user identification according to claim 6, characterized in that, The image generation model is a convolutional neural network model. The input of the image generation model is the image when the user approaches the smart mirror, and the output of the image generation model is the image of the user in the smart mirror.

9. The intelligent mirror cabinet control system based on user identification according to claim 6, wherein, The system is further configured to: if the environmental light brightness is less than a threshold value, remind the user to turn on the light.

10. The intelligent mirror cabinet control system based on user identification according to claim 6, characterized in that, The input of the graph neural network model is the multiple nodes and multiple edges between the multiple nodes, and the output of the graph neural network model is the multiple target applications and the display positions of each target application in the multiple target applications in the smart mirror.

Images