Health state assessment method and device, storage medium and computer equipment

Abstract

According to the health state assessment method and device, the storage medium and the computer equipment provided by the invention, when the target person is subjected to attendance checking and card punching, the face image of the target person collected by the access control equipment can be obtained, and after the multi-modal features in the face image are extracted, the predicted symptoms associated with the multi-modal features are determined; then, detail features and physiological features of the face image can be extracted, and fusion features are obtained after the detail features and the physiological features are fused, so that the reliability and accuracy of the features are improved; basic information, corresponding to the target person, in the face image can be recognized, and the basic information can assist in accurate judgment; therefore, the health state of the target person can be comprehensively evaluated according to the predicted symptoms, the fusion features and the basic information, and then a comprehensive and accurate comprehensive evaluation result is obtained.

Description

Technical Field

[0001] The present application relates to the field of smart access control technology, and in particular to a health status assessment method, device, storage medium and computer equipment. Background Art

[0002] Access control devices, as core components of modern security systems, are used in a variety of scenarios, such as office buildings, residential communities, industrial plants, schools, hospitals, shopping malls, and subways. The usage and functional design of access control devices vary depending on the scenario.

[0003] For example, when using access control devices for attendance, in order to improve the company's health management capabilities and pay timely attention to the health of employees, access control devices are generally used to monitor the health status of employees while they are clocking in, with the employees' knowledge and consent.

[0004] Currently, when using access control equipment for health monitoring, generally only basic indicators such as employees' body temperature, heart rate, and blood oxygen are monitored, and it is impossible to conduct a comprehensive and accurate assessment of the employees' health status. Summary of the Invention

[0005] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect in the prior art that when access control equipment is used for health monitoring, the health status of employees cannot be comprehensively and accurately assessed.

[0006] The present application provides a health status assessment method, the method comprising:

[0007] Obtain the facial image of the target person collected by the access control device;

[0008] extracting multimodal features of the facial image and determining predicted symptoms associated with the multimodal features;

[0009] Extracting detail features and physiological features of the face image, and fusing the detail features and the physiological features to obtain fused features;

[0010] Identifying basic information corresponding to the target person in the facial image;

[0011] A comprehensive assessment is performed on the health status of the target person based on the predicted symptoms, the fusion features, and the basic information to obtain a comprehensive assessment result.

[0012] Optionally, extracting multimodal features of the facial image includes:

[0013] Inputting the facial image into a pre-built facial feature recognition model to obtain 2D geometric features, 3D geometric features, and visual features of various facial regions output by the facial feature recognition model;

[0014] The 2D geometric features, the 3D geometric features, and the visual features are used as multimodal features.

[0015] Optionally, determining the predicted symptoms associated with the multimodal features comprises:

[0016] The multimodal features are associated and mapped with a preset TCM visual diagnosis knowledge graph, and the predicted symptoms corresponding to the multimodal features are determined based on the mapping results.

[0017] Optionally, extracting detailed features and physiological features of the facial image includes:

[0018] performing a multi-scale analysis on the facial image based on a generative adversarial network, and determining detailed features of the facial image according to a first analysis result;

[0019] Performing spectral imaging analysis on the facial image based on spectral analysis technology, and determining the physiological characteristics of the facial image according to the second analysis result.

[0020] Optionally, the basic information corresponding to the target person in the identified facial image includes:

[0021] The facial image is input into a multi-task learning model combined with an attention mechanism. When the multi-task learning model is used to extract facial features in the facial image, the attention mechanism is used to automatically focus on key features related to each task, thereby obtaining basic information corresponding to the target person output by the multi-task learning model.

[0022] Optionally, the comprehensive assessment of the health status of the target person based on the predicted symptoms, the fusion features, and the basic information to obtain a comprehensive assessment result includes:

[0023] Classifying the health status of the fused features to obtain a classification result;

[0024] The predicted symptoms, the classification results, and the basic information are input into a pre-configured target health status assessment model to obtain a comprehensive assessment result output by the target health status assessment model.

[0025] Optionally, the configuration process of the target health status assessment model includes:

[0026] Obtaining a training data set, wherein the training data set includes multiple training data, each training data includes sample facial features of multiple dimensions and sample health status corresponding to the sample facial features of each dimension;

[0027] Inputting the sample facial features of each dimension in each training data into the initial health status assessment model to obtain the predicted health status output by the initial health status assessment model;

[0028] With the goal of making the predicted health state approach the corresponding sample health state, iteratively training the initial health state assessment model until the training goal is reached;

[0029] The trained initial health status assessment model is used as the target health status assessment model.

[0030] The present application also provides a health status assessment device, comprising:

[0031] An image acquisition module is used to obtain the face image of the target person collected by the access control device;

[0032] a symptom prediction module, configured to extract multimodal features of the facial image and determine predicted symptoms associated with the multimodal features;

[0033] A feature extraction module is used to extract detail features and physiological features of the face image, and fuse the detail features and the physiological features to obtain fused features;

[0034] An information recognition module, configured to recognize basic information corresponding to the target person in the facial image;

[0035] The comprehensive evaluation module is used to comprehensively evaluate the health status of the target person based on the predicted symptoms, the fusion features and the basic information to obtain a comprehensive evaluation result.

[0036] The present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the health status assessment method as described in any of the above embodiments.

[0037] The present application also provides a computer device, comprising: one or more processors, and a memory;

[0038] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the health status assessment method as described in any one of the above embodiments are performed.

[0039] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0040] The health status assessment method, device, storage medium and computer equipment provided by the present application can obtain the facial image of the target person collected by the access control device when the target person clocks in for attendance, and after extracting the multimodal features in the facial image, determine the predicted symptoms associated with the multimodal features; then, the present application can extract the detailed features and physiological features of the facial image, and fuse the detailed features and physiological features to obtain fused features to improve the reliability and accuracy of the features; the present application can also identify the basic information corresponding to the target person in the facial image, and the basic information can assist in accurate judgment; in this way, the health status of the target person can be comprehensively assessed based on the predicted symptoms, fused features and basic information, thereby obtaining a comprehensive and accurate comprehensive assessment result. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 A flowchart of a health status assessment method provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the process of identifying basic information of a target person using a multi-task learning model provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the health status assessment process provided in an embodiment of the present application;

[0045] Figure 4 A diagram of the network architecture for selecting a Bayesian network for health status assessment according to an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of a health status assessment device provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] In one embodiment, Figure 1 As shown, Figure 1 A flow chart of a health status assessment method provided in an embodiment of the present application; the present application provides a health status assessment method, which may include:

[0050] S110: Acquire a facial image of the target person collected by the access control device.

[0051] In this step, when it is detected that the target person uses the access control device to clock in and out, the facial image of the target person captured by the access control device when clocking in can be obtained, and the health status of the target person can be evaluated based on the facial image. In this way, the target person can make a comprehensive and accurate assessment of his or her health status through the facial image captured when clocking in and out without performing any other operations.

[0052] Among them, the target person of this application refers to a person who has the authority to punch in and agrees to undergo a health status assessment. When obtaining the facial image of the target person collected by the access control device, this application can first determine whether the target person has the relevant authority and whether he agrees to undergo a health status assessment. If he does not have the authority, he cannot punch in and out for attendance, nor can he undergo a health status assessment; if he has the authority but does not agree to undergo a health status assessment, he can punch in and out for attendance; if he has the authority and agrees to undergo a health status assessment, he can punch in and out for attendance and undergo a health status assessment at the same time.

[0053] Furthermore, to obtain clearer facial images, the access control device of this application can be equipped with an intelligent camera module with high dynamic range (HDR) imaging capabilities and adopts an advanced CMOS (Complementary Metal Oxide Semiconductor) image sensor, which can accurately capture facial images within an extremely wide dynamic range of illumination. In addition, this application can also collect facial images through a multi-frame exposure fusion algorithm. In this way, in low-light environments as low as 0.1 lux and high-light environments as high as 100,000 lux, facial images with rich details and high color reproduction can be obtained, thereby providing a high-quality data source for subsequent in-depth analysis.

[0054] S120: Extracting multimodal features of the facial image and determining predicted symptoms associated with the multimodal features.

[0055] In this step, after obtaining the facial image of the target person captured by the access control device through S110, the multimodal features of the facial image can be extracted, and the predicted symptoms associated with the multimodal features can be determined.

[0056] Specifically, after acquiring the target person's facial image, the present application can first extract the multimodal features of the facial image in order to conduct a comprehensive and accurate health status assessment of the target person. The multimodal features refer to a set of complementary features of different dimensions extracted from the facial image. By fusing these features, the health status of the target person can be more comprehensively characterized. The present application can utilize the advantages of different modal data to make up for the limitations of a single modality, thereby effectively improving the robustness and accuracy of the assessment results.

[0057] Furthermore, after the present application determines the multimodal features corresponding to the facial image of the target person, it can also determine the predicted symptoms associated with the multimodal features, which can lay the foundation for subsequent health status assessment. Among them, when determining the predicted symptoms associated with the multimodal features, the present application can first determine the symptoms corresponding to each single modality, and then fuse the symptoms under each modality to determine the predicted symptoms associated with the multimodal features; it can also directly determine the predicted symptoms associated with it based on the multimodal features. The specific determination process can be selected according to the actual situation and is not limited here.

[0058] S130: extracting detail features and physiological features of the face image, and fusing the detail features and the physiological features to obtain fused features.

[0059] In this step, after obtaining the facial image of the target person captured by the access control device through S110, the detailed features and physiological features of the facial image can also be extracted, and the detailed features and physiological features are fused to obtain fused features. Using the fused features to perform health status assessment can effectively improve the accuracy of the assessment results.

[0060] In one specific implementation, the present application can perform multi-scale analysis on collected facial images. For example, when performing multi-scale analysis on facial images, the present application can analyze everything from the macroscopic overall facial morphology to the microscopic skin texture details, and extract the corresponding detailed features. Simultaneously, the present application can also perform multi-physiological analysis on skin color, lip color, and other features in facial images, obtaining physiological indicators such as blood oxygen saturation in subcutaneous microvessels, thereby enabling a comprehensive assessment of the health status of the human body, including qi and blood, and organ function.

[0061] Furthermore, when fusing detail features and physiological features, the present application can select any fusion method, such as splicing fusion, weighted fusion, or graph neural network fusion to fuse detail features and physiological features, and then obtain fusion features.

[0062] S140: Identify basic information corresponding to the target person in the facial image.

[0063] In this step, after obtaining the facial image of the target person collected by the access control device through S110, the basic information corresponding to the target person in the facial image can also be identified, which can help improve the accuracy of health status assessment.

[0064] Among them, the basic information corresponding to the target person in the facial image recognized by this application includes but is not limited to age, gender, height, weight, etc., which can be set according to the actual situation and is not limited here. In addition, this application can retrieve the attendance database to obtain the basic information of the target person, or it can identify the corresponding basic information through the basic information recognition model, or obtain the basic information of the target person through other means. When this application obtains the basic information of the target person by other means other than retrieving the attendance database, it can also correct the outdated information of the system (such as appearance changes that have not been updated, etc.) based on the recognized basic information, and can also detect whether it is an unauthorized person or an employee whose appearance has suddenly changed due to illness based on the recognized basic information. After this application recognizes the basic information of the target person, it can also count the age / gender distribution based on the basic information, thereby effectively supporting human resources and health management (such as customized physical examinations, health activities).

[0065] S150: Based on the predicted symptoms, fusion features and basic information, a comprehensive assessment is performed on the health status of the target person to obtain a comprehensive assessment result.

[0066] In this step, the multimodal features of the facial image are extracted through S120~S140, and the predicted symptoms associated with the multimodal features are determined, the detailed features and physiological features of the facial image are extracted, and the fusion features are obtained by fusing them, and the basic information corresponding to the target person in the facial image is identified. After that, this application can conduct a comprehensive assessment of the health status of the target person based on the predicted symptoms, fusion features and basic information obtained above, and thus obtain the final comprehensive assessment results.

[0067] Specifically, this application can make comprehensive inferences about the health status of the target person based on the dependencies between the different features obtained above, and generate a visual preliminary health report. This report can not only provide a quantitative assessment of the current health status, but also give potential health risk predictions and personalized health recommendations based on probabilistic reasoning, thereby fully reflecting the diversification and intelligence of access control equipment, as well as the company's humanistic care. And this process does not require additional health check equipment, so that the target person can quickly understand his or her own health status while clocking in for daily attendance, which helps users detect potential health problems as early as possible. When long-term health data records are formed, it can also provide data support for personal health management and medical diagnosis.

[0068] For example, this application can use the large amount of health data collected after desensitization for medical research, public health monitoring and other fields, thereby providing rich data resources for health-related scientific research and promoting the development of medicine and health. At the same time, this health data can also help companies to timely understand the health status of their employees, optimize their human resources management, and improve overall operational efficiency.

[0069] In the above embodiment, when the target person clocks in for attendance, the facial image of the target person collected by the access control device can be obtained, and after extracting the multimodal features in the facial image, the predicted symptoms associated with the multimodal features are determined; then, the present application can extract the detail features and physiological features of the facial image, and fuse the detail features and physiological features to obtain fused features to improve the reliability and accuracy of the features; the present application can also identify the basic information corresponding to the target person in the facial image, and the basic information can assist in accurate judgment; in this way, the health status of the target person can be comprehensively evaluated based on the predicted symptoms, fused features and basic information, thereby obtaining a comprehensive and accurate comprehensive evaluation result.

[0070] In one embodiment, extracting the multimodal features of the facial image in S120 may include:

[0071] S121: Input the facial image into a pre-built facial feature recognition model to obtain the 2D geometric features, 3D geometric features, and visual features of each facial region output by the facial feature recognition model.

[0072] S122: Taking the 2D geometric features, the 3D geometric features, and the visual features as multimodal features.

[0073] In this embodiment, when extracting multimodal features from facial images, a deep convolutional neural network (CNN) architecture can be used, combined with transfer learning techniques, to build a large-scale facial feature recognition model. The specific model construction process can be configured based on existing technologies and will not be detailed here.

[0074] Furthermore, after the application constructs a facial feature recognition model, the model is pre-trained using a large-scale image dataset and then retrained on facial images. Therefore, the model can accurately extract key facial features from complex backgrounds. These key features include but are not limited to 2D and 3D geometric features of areas such as the eyes, lips, and nose, as well as visual features such as skin color and texture, laying the foundation for subsequent health status analysis.

[0075] In one embodiment, determining the predicted symptoms associated with the multimodal features in S120 may include:

[0076] S123: Associating and mapping the multimodal features with a preset TCM visual diagnosis knowledge graph, and determining predicted symptoms corresponding to the multimodal features based on the mapping results.

[0077] In this embodiment, after extracting the multimodal features of the facial image, the present application can also determine the predicted symptoms associated with the multimodal features. For example, the present application can use knowledge graph technology to structure the knowledge about "inspection" in classical Chinese medicine literature and establish a semantic network containing multi-dimensional information such as complexion, tongue image, and facial features. Next, the present application can use the graph neural network (GNN) algorithm to associate and map the features obtained from the facial image analysis with the knowledge graph, thereby exploring the potential relationship between image features and Chinese medicine health diagnostic indicators, and realizing intelligent reasoning from image features to health status.

[0078] The specific process of constructing the knowledge graph of TCM diagnosis by observation in this application may include the following:

[0079] 1. Data collection and organization

[0080] Collect multi-source data: Collect data related to TCM visual diagnosis from ancient TCM books, clinical medical records, academic literature, TCM textbooks, etc., including symptom descriptions, diagnostic results, and relevant TCM theories.

[0081] Organize and label data: Clean and remove duplicates from the collected data, and label them according to unified standards. For example, label symptoms into different categories, such as complexion, tongue condition, pulse condition, etc.

[0082] 2. Knowledge Extraction

[0083] Entity recognition: Leveraging the natural language processing capabilities of large AI (artificial intelligence) models, we can identify entities related to TCM visual diagnosis in text, such as various complexions (e.g., pale, rosy), tongue images (e.g., red tongue, thin white tongue coating), and physical signs (e.g., dry hair, pale nails).

[0084] Relationship extraction: Extract the relationship between entities, such as the relationship between "pale complexion" and "insufficient qi and blood", the relationship between "red tongue" and "heat syndrome", etc.

[0085] 3. Knowledge graph construction

[0086] Define the graph structure: determine the nodes and edges of the knowledge graph. Nodes can be various visual diagnosis entities, TCM syndromes, diseases, etc. Edges represent the relationships between entities, such as "symptom-syndrome" and "syndrome-disease".

[0087] Filling the graph: Filling the extracted knowledge according to the graph structure to form a preliminary TCM visual diagnosis knowledge graph. Of course, in the actual application process, the TCM visual diagnosis knowledge graph can also be continuously updated and optimized.

[0088] After this application constructs the TCM visual diagnosis knowledge graph according to the above process, the multimodal features can be associated and mapped with the TCM visual diagnosis knowledge graph, so as to find nodes with similar facial features in the TCM visual diagnosis knowledge graph and match the corresponding symptoms. In this way, the predicted symptoms corresponding to the multimodal features can be obtained, thereby providing more accurate basic data for subsequent health status assessment.

[0089] In one embodiment, extracting the detailed features and physiological features of the facial image in S130 may include:

[0090] S131: Perform multi-scale analysis on the facial image based on a generative adversarial network, and determine detailed features of the facial image according to a first analysis result.

[0091] S132: Performing spectral imaging analysis on the facial image based on spectral analysis technology, and determining physiological features of the facial image according to a second analysis result.

[0092] In this embodiment, in order to obtain the detailed features and physiological characteristics of a facial image and improve the accuracy of the health status assessment results, the present application can perform multi-scale analysis on the facial image based on a generative adversarial network, and after obtaining a first analysis result, determine the detailed features of the facial image based on the first analysis result; then, the present application can perform spectral imaging analysis on the facial image based on spectral analysis technology, and after obtaining a second analysis result, determine the physiological characteristics of the facial image based on the second analysis result.

[0093] In one specific implementation, the generative adversarial network of this application may include a generator and a discriminator. The generator can use large and small convolution kernels for multi-scale extraction, with large kernels capturing facial contours and small kernels refining skin texture. When training the generative adversarial network, this application can input a large number of facial images to enable the generator to learn features of complexion (hue, brightness), eyes (shape, color), and lips (morphology, lip color), while the discriminator provides feedback for optimization. This allows the generator to extract detailed features of facial images after multiple iterations of training.

[0094] Furthermore, to improve the accuracy of health status assessment, this application can also use multispectral equipment, such as a multispectral camera, to collect reflection data of different wavelengths on the face (such as skin color and lip color spectra). This reflection data can reflect physiological indicators such as microvascular blood oxygen. Of course, this application can also add 650nm / 850nm filters to the original smart camera module and obtain dual-band data through alternating shooting, and then use this data to determine the corresponding physiological characteristics. Through spectral imaging analysis, this application can obtain deep physiological information without contacting the human body.

[0095] In one embodiment, the basic information corresponding to the target person in the facial image identified in S140 may include:

[0096] S141: Input the facial image into a multi-task learning model combined with an attention mechanism, and when using the multi-task learning model to extract facial features in the facial image, automatically focus on key features related to each task through the attention mechanism, and obtain basic information corresponding to the target person output by the multi-task learning model.

[0097] In this embodiment, when identifying the basic information corresponding to the target person in the facial image, a multi-task learning model can be used for recognition. The model is combined with an attention mechanism, which can automatically focus on key features related to each task, such as facial bone structure, skin laxity, etc., thereby achieving high-precision recognition of people of different age groups and genders, and the recognition accuracy rate reaches more than 98% on public data sets.

[0098] Schematically, as Figure 2 As shown, Figure 2A schematic diagram of the process of using a multi-task learning model to identify the basic information of a target person provided in an embodiment of the present application; the present application can input a facial image into the multi-task learning model, and the multi-task learning module can pre-determine multiple tasks based on the basic information to be identified. For example, if the present application wants to identify the age and gender corresponding to the facial image, the age and gender can be used as a task respectively, so that the attention mechanism can automatically focus on the key features related to age and the key features related to gender. Finally, the present application can output the age probability distribution and gender probability through the multi-task learning model, so that the basic information corresponding to the target person can be quickly determined.

[0099] In one embodiment, in S150, a comprehensive assessment of the health status of the target person is performed based on the predicted symptoms, the fusion features, and the basic information to obtain a comprehensive assessment result, which may include:

[0100] S151: Classify the health status of the fused features to obtain a classification result.

[0101] S152: Input the predicted symptoms, the classification results, and the basic information into a pre-configured target health status assessment model to obtain a comprehensive assessment result output by the target health status assessment model.

[0102] In this embodiment, when comprehensively evaluating the health status of the target person based on the predicted symptoms, fusion characteristics and basic information, the present application can first classify the health status of the fusion status, and after obtaining the classification results, conduct a comprehensive evaluation in combination with the predicted symptoms and basic information, which can further improve the accuracy of the evaluation results.

[0103] Schematically, as Figure 3 As shown, Figure 3 A schematic diagram of the health status assessment process provided for an embodiment of the present application; after capturing the facial image of the target person through the attendance machine, the present application can perform multi-scale analysis on the facial image based on the generative adversarial network, and extract detailed features such as complexion and eyes. At the same time, the present application can also integrate spectral analysis technology to perform spectral imaging analysis on local features such as skin color and lip color, and after fusing the detailed features with the physiological features, classify the fused features into health status to associate the health status of qi and blood, internal organs, etc., and finally combine the predicted symptoms, classification results and basic information to perform health status assessment, and then comprehensively assess the health status from these perspectives. Among them, when classifying the fused features, the present application can use a health status classification model, or it can use other methods for classification. The specific selection can be made according to the actual situation and is not limited here.

[0104] Furthermore, after the application obtains the classification results, the predicted symptoms, classification results, and basic information can be input into a pre-configured target health status assessment model, and the target health status assessment model can be used to comprehensively assess the health status of the target person, and obtain the comprehensive assessment results output by the target health status assessment model. The application can generate a visual preliminary health report based on the comprehensive assessment results. The report can not only provide a quantitative assessment of the current health status, but also give predictions of potential health risks and personalized health recommendations based on probabilistic reasoning, thereby fully reflecting the intelligence and diversity of access control equipment.

[0105] In one embodiment, the configuration process of the target health status assessment model may include:

[0106] S210: Obtain a training data set, where the training data set includes multiple training data, each training data includes sample facial features of multiple dimensions and sample health status corresponding to the sample facial features of each dimension.

[0107] S211: Inputting the sample facial features of each dimension in each training data into the initial health status assessment model to obtain the predicted health status output by the initial health status assessment model.

[0108] S212: With the goal of making the predicted health state approach the corresponding sample health state, iteratively train the initial health state assessment model until the training goal is reached.

[0109] S213: Using the trained initial health status assessment model as the target health status assessment model.

[0110] In this embodiment, before using the target health status assessment model to comprehensively assess the health status of the target person, the present application can iteratively train the initial health status assessment model so that the initial health status assessment model can learn the correlation between sample facial features of different dimensions and sample health status.

[0111] Specifically, the present application can first obtain a training data set, which contains multiple training data, and each training data can include sample facial features of multiple dimensions and sample health status corresponding to the sample facial features of each dimension. In this way, the sample facial features of each dimension in each training data can be input into the initial health status assessment model during each iterative training, and after obtaining the predicted health status output by the initial health status assessment model, the gap between the predicted health status and the sample health status is calculated, and the initial health status assessment model is iteratively trained with the goal of making the predicted health status approach the corresponding sample health status until the training goal is reached. In this way, the final target health status assessment model can be obtained.

[0112] Among them, the initial health status assessment model of this application can use a Bayesian network, which can make a comprehensive inference on the health status of the target person based on the probabilistic dependency relationship between different features. Figure 4 As shown, Figure 4 The network architecture diagram for selecting Bayesian network for health status assessment provided in the embodiment of this application; this application can be constructed as follows Figure 4 The Bayesian network shown in the figure has observation nodes representing facial features (continuous or discretized), which represent the predicted symptoms, classification results, and basic information in this application. Hidden nodes represent potential health states (such as "sub-health"), and decision nodes represent intervention recommendations (such as "hemoglobin check recommended"). This application achieves personalized and interpretable health status assessment by converting multi-dimensional facial features into observational evidence in a Bayesian network.

[0113] The following describes a health status assessment device provided in an embodiment of the present application. The health status assessment device described below and the health status assessment method described above can be referenced to each other.

[0114] In one embodiment, Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of a health status assessment device provided in an embodiment of the present application. The present application also provides a health status assessment device, which may include an image acquisition module 210, a symptom prediction module 220, a feature extraction module 230, an information recognition module 240, and a comprehensive assessment module 250, specifically including the following:

[0115] The image acquisition module 210 is used to acquire the facial image of the target person collected by the access control device.

[0116] The symptom prediction module 220 is used to extract multimodal features of the facial image and determine predicted symptoms associated with the multimodal features.

[0117] The feature extraction module 230 is used to extract detail features and physiological features of the face image, and fuse the detail features and the physiological features to obtain fused features.

[0118] The information recognition module 240 is used to recognize basic information corresponding to the target person in the facial image.

[0119] The comprehensive evaluation module 250 is used to perform a comprehensive evaluation on the health status of the target person according to the predicted symptoms, the fusion features and the basic information to obtain a comprehensive evaluation result.

[0120] In the above embodiment, when the target person clocks in for attendance, the facial image of the target person collected by the access control device can be obtained, and after extracting the multimodal features in the facial image, the predicted symptoms associated with the multimodal features are determined; then, the present application can extract the detail features and physiological features of the facial image, and fuse the detail features and physiological features to obtain fused features to improve the reliability and accuracy of the features; the present application can also identify the basic information corresponding to the target person in the facial image, and the basic information can assist in accurate judgment; in this way, the health status of the target person can be comprehensively evaluated based on the predicted symptoms, fused features and basic information, thereby obtaining a comprehensive and accurate comprehensive evaluation result.

[0121] The present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the health status assessment method as described in any of the above embodiments.

[0122] The present application also provides a computer device, comprising: one or more processors, and a memory;

[0123] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the health status assessment method as described in any one of the above embodiments are performed.

[0124] Schematically, as Figure 6 As shown, Figure 6 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 can be provided as a server. Figure 6Computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by memory 301 for storing instructions executable by processing component 302, such as an application. The application stored in memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, processing component 302 is configured to execute the instructions to perform the health status assessment method of any of the above embodiments.

[0125] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0126] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0127] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0128] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0129] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A health status assessment method, characterized in that: The method comprises: Obtain the facial image of the target person collected by the access control device; extracting multimodal features of the facial image and determining predicted symptoms associated with the multimodal features; Extracting detail features and physiological features of the face image, and fusing the detail features and the physiological features to obtain fused features; Identifying basic information corresponding to the target person in the facial image; A comprehensive assessment is performed on the health status of the target person based on the predicted symptoms, the fusion features, and the basic information to obtain a comprehensive assessment result.

2. The health status assessment method according to claim 1, characterized in that: The extracting multimodal features of the facial image includes: Inputting the facial image into a pre-built facial feature recognition model to obtain 2D geometric features, 3D geometric features, and visual features of various facial regions output by the facial feature recognition model; The 2D geometric features, the 3D geometric features, and the visual features are used as multimodal features.

3. The health status assessment method according to claim 1, characterized in that: Determining a predicted symptom associated with the multimodal feature comprises: The multimodal features are associated and mapped with a preset TCM visual diagnosis knowledge graph, and the predicted symptoms corresponding to the multimodal features are determined based on the mapping results.

4. The health status assessment method according to claim 1, characterized in that: The extracting of detail features and physiological features of the facial image includes: Performing a multi-scale analysis on the facial image based on a generative adversarial network, and determining detailed features of the facial image according to a first analysis result; Perform spectral imaging analysis on the facial image based on spectral analysis technology, and determine the physiological characteristics of the facial image according to the second analysis result.

5. The health status assessment method according to claim 1, characterized in that: The basic information corresponding to the target person in the identified facial image includes: The facial image is input into a multi-task learning model combined with an attention mechanism. When the multi-task learning model is used to extract facial features in the facial image, the attention mechanism is used to automatically focus on key features related to each task, thereby obtaining basic information corresponding to the target person output by the multi-task learning model.

6. The health status assessment method according to any one of claims 1 to 5, characterized in that: The comprehensive assessment of the health status of the target person is performed based on the predicted symptoms, the fusion features, and the basic information to obtain a comprehensive assessment result, including: Classifying the health status of the fused features to obtain a classification result; The predicted symptoms, the classification results, and the basic information are input into a pre-configured target health status assessment model to obtain a comprehensive assessment result output by the target health status assessment model.

7. The health status assessment method according to claim 6, characterized in that: The configuration process of the target health status assessment model includes: Obtaining a training data set, wherein the training data set includes multiple training data, each training data includes sample facial features of multiple dimensions and sample health status corresponding to the sample facial features of each dimension; Inputting the sample facial features of each dimension in each training data into the initial health status assessment model to obtain the predicted health status output by the initial health status assessment model; With the goal of making the predicted health state approach the corresponding sample health state, iteratively training the initial health state assessment model until the training goal is reached; The trained initial health status assessment model is used as the target health status assessment model.

8. A health status assessment device, characterized in that: include: An image acquisition module is used to obtain the face image of the target person collected by the access control device; a symptom prediction module, configured to extract multimodal features of the facial image and determine predicted symptoms associated with the multimodal features; A feature extraction module is used to extract detail features and physiological features of the face image, and fuse the detail features and the physiological features to obtain fused features; An information recognition module, configured to recognize basic information corresponding to the target person in the facial image; The comprehensive evaluation module is used to comprehensively evaluate the health status of the target person based on the predicted symptoms, the fusion features and the basic information to obtain a comprehensive evaluation result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the health status assessment method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the health status assessment method according to any one of claims 1 to 7.

Images