State recognition methods, devices, electronic equipment and storage media

Abstract

This application discloses a state recognition method, apparatus, electronic device, and storage medium. The state recognition method includes: obtaining a first dataset corresponding to a first user from the blockchain system; dividing the obtained first dataset to obtain at least two second datasets, and determining a first feature tensor corresponding to each second dataset; each of the at least two second datasets corresponds to a different data type; inputting the first feature tensor corresponding to each of the at least two second datasets into a set state recognition model, and outputting a first result; wherein the first dataset includes first data collected from the first user; the first data of the first user represents the state data of the first user; and the first result is used to generate a state evaluation report for the first user.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a state recognition method, device, electronic device, and storage medium. Background Technology

[0002] In related technologies, psychological assessment scales are usually used to identify users' psychological states. However, since users are largely influenced by subjective factors when filling out psychological assessment scales, the results of psychological state identification are not accurate enough. Summary of the Invention

[0003] In view of this, embodiments of this application provide a state recognition method, apparatus, electronic device, and storage medium to at least solve the problem that the psychological state recognition results of related technologies are not accurate enough.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a state recognition method, the method comprising:

[0006] Obtain the first dataset corresponding to the first user from the blockchain system;

[0007] The obtained first dataset is divided into at least two second datasets, and the first feature tensor corresponding to each second dataset is determined; each of the at least two second datasets corresponds to a different data type;

[0008] Input the first feature tensor corresponding to each of the at least two second datasets into the defined state recognition model, and output a first result; wherein...

[0009] The first dataset includes the first data collected from the first user; the first data of the first user represents the status data of the first user; the first result is used to generate a status assessment report for the first user.

[0010] In the above scheme, the step of dividing the obtained first dataset to obtain at least two second datasets includes:

[0011] Obtain the encrypted data of the first dataset corresponding to the first user from the blockchain system;

[0012] The first dataset is decrypted using the first key corresponding to the first user to obtain the third dataset;

[0013] The third dataset is divided to obtain at least two second datasets; wherein,

[0014] The first key is generated based on the information set by the first user.

[0015] In the above scheme, before obtaining the first dataset corresponding to the first user from the blockchain system, the method further includes:

[0016] Determine at least one first data point corresponding to the first user;

[0017] At least one first piece of data is determined and sent to the blockchain system.

[0018] In the above scheme, determining at least one piece of first data corresponding to the first user includes:

[0019] Invoke at least one acquisition module of the status acquisition device to acquire at least two of the following data from the first user:

[0020] First scale data;

[0021] First image data;

[0022] First audio data;

[0023] Primary vital signs data;

[0024] The first image data represents the user's facial data.

[0025] In the above scheme, the first data also includes the personal information of the first user.

[0026] In the above scheme, when sending at least one determined first data to the blockchain system, the method includes:

[0027] Predict the first data point at time i+1;

[0028] When the first data of the i-th time node and the predicted first data of the (i+1)-th time node satisfy the set relationship, the first data of the i-th time node is sent to the blockchain system.

[0029] The method in the above scheme further includes:

[0030] A state assessment report is generated based on the first result output by the defined state recognition model.

[0031] The generated status assessment report is sent to the blockchain system.

[0032] This application also provides a state recognition device, including:

[0033] An acquisition unit is used to acquire a first dataset corresponding to a first user from the blockchain system;

[0034] The first determining unit is used to divide the acquired first dataset into at least two second datasets, and determine the first feature tensor corresponding to each second dataset; each of the at least two second datasets corresponds to a different data type;

[0035] The output unit is used to input the first feature tensor corresponding to each of the at least two second datasets into a set state recognition model and output a first result; wherein,

[0036] The first dataset includes the first data collected from the first user; the first data of the first user represents the status data of the first user; the first result is used to generate a status assessment report for the first user.

[0037] This application also provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor.

[0038] When the processor runs the computer program, it executes the steps of the above-described state recognition method.

[0039] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described state recognition method.

[0040] In this embodiment, a first dataset representing the state data of a first user is obtained from a blockchain system. The first dataset is divided into at least two second datasets corresponding to different data types. A first feature tensor corresponding to each second dataset is determined. The first feature tensor corresponding to each of the at least two second datasets is input into a set state recognition model. A first result is output to generate a state evaluation report for the first user. In this way, by using the set state recognition model to analyze the feature tensors corresponding to at least two types of datasets representing psychological state data, the output result is more closely matched with the user's psychological state, thereby improving the accuracy of the user's psychological state recognition result. Attached Figure Description

[0041] Figure 1 A flowchart illustrating a state recognition method provided in an embodiment of this application;

[0042] Figure 2 A schematic diagram of a state recognition system provided for an application embodiment of this application;

[0043] Figure 3 A flowchart of face capture image encoding provided for an application embodiment of this application;

[0044] Figure 4A schematic diagram of the structure of a face acquisition module provided for an application embodiment of this application;

[0045] Figure 5 A schematic diagram of the structure of a filter rotation mechanism for a face acquisition module provided in the application embodiments of this application;

[0046] Figure 6 A schematic diagram of another state recognition system provided in an embodiment of this application;

[0047] Figure 7 A schematic diagram of the physical link of a blockchain system provided in this application embodiment;

[0048] Figure 8 This application provides a schematic diagram of a data storage layer for a blockchain system.

[0049] Figure 9 This is a schematic diagram of the structure of a state recognition device provided in an embodiment of this application;

[0050] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] For businesses, the stability of employees' mental state has a significant impact on their development. Currently, psychological states are typically identified through psychological assessment scales. However, because the completion of these scales is largely influenced by subjective factors, and the results only reflect the psychological state at the time of the assessment, the resulting identification of psychological states is not accurate enough.

[0052] Based on this, in the solution provided in this application embodiment, a first dataset representing the state data of the first user is obtained from the blockchain system. The first dataset is divided into at least two second datasets corresponding to different data types. A first feature tensor corresponding to each second dataset is determined. The first feature tensor corresponding to each of the at least two second datasets is input into a set state recognition model. A first result for generating a state evaluation report of the first user is output. In this way, by using the set state recognition model to analyze the feature tensors corresponding to at least two types of datasets representing psychological state data, the output result is more closely matched with the user's psychological state, thereby improving the accuracy of the user's psychological state recognition result.

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] Figure 1This is a flowchart illustrating the state recognition method provided in this application embodiment, applied to the first node in a blockchain system, such as... Figure 1 The state recognition method shown includes:

[0055] Step 101: Obtain the first dataset corresponding to the first user from the blockchain system.

[0056] The first dataset includes the first data collected from the first user; the first data of the first user represents the status data of the first user.

[0057] The first node obtains the first user's first dataset from the blockchain system at least once. The first dataset represents the set of first data collected at least once, and the first data represents the state data of the corresponding user. When the first dataset corresponding to the same first user is encapsulated in different blocks, the first node can obtain the first user's first dataset from the blockchain system at least once.

[0058] Alternatively, the data of the entire blockchain can be synchronized to the local machine of the first node. Based on the identifier of the first node, the data of the entire blockchain can be traversed to find the block in the blockchain that matches the identifier of the first node, thereby obtaining the first user's first dataset.

[0059] Here, the first node in the blockchain system represents any node in the blockchain system that can execute the state recognition method, such as a terminal or processor.

[0060] Step 102: Divide the obtained first dataset into at least two second datasets, and determine the first feature tensor corresponding to each second dataset.

[0061] Each of the at least two second datasets corresponds to a different data type.

[0062] The first dataset, acquired through at least one acquisition action of the first node, is partitioned according to predefined rules to obtain at least two second datasets, and a first feature tensor corresponding to each second dataset is determined. Here, the predefined rules can be determined based on the storage format and / or acquisition method of the first data, such as classifying the first data into text, audio, and image categories. By defining the rules, the first dataset is divided into at least two second datasets based on the first data within it, with each second dataset corresponding to a different data type. For example, if the first dataset includes two text data categories, one audio data category, and one image data category, partitioning it based on the first dataset yields three second datasets: second dataset 1 corresponding to the text data category, second dataset 2 corresponding to the audio data category, and second dataset 3 corresponding to the image data category.

[0063] Furthermore, at least one data point can be determined from each of the at least two data types based on a set time threshold, and used as the data collected during the same state recognition, to obtain at least two second datasets, and then the feature tensors corresponding to the at least two second datasets can be determined.

[0064] For example, the first set of data can be divided into two categories, A and B. Category A includes data A1 collected on June 1, 2020, and data A2 collected on June 1, 2021. Category B includes data B1 collected on June 4, 2020, and data B2 collected on April 29, 2021. A one-week time threshold is set. If the interval between A1 and B1 is less than the time threshold, A1 and B1 are identified as data collected during the same state identification period. If the intervals between A1 and B2, A1 and B2, and A2 and B2 are all greater than the set time threshold, they cannot be identified as data collected during the same state identification period.

[0065] Step 103: Input the first feature tensor corresponding to each of the at least two second datasets into the set state recognition model, and output the first result.

[0066] The first result is used to generate a status assessment report for the first user.

[0067] The first feature tensor corresponding to each determined second dataset is input into the established state recognition model, and the first result is output. Here, the established state recognition model can use deep learning-based deep neural networks (DNNs), long short-term memory networks (LSTMs), etc., as the basic model for state recognition. The model includes a classifier, which can be composed of fully connected layers, or a support vector machine (SVM) classifier, a softmax classifier, etc. The output first result represents the user's psychological state in the form of a score (out of 5, or a percentage, with higher scores indicating a healthier psychological state) and a label (whether healthy, whether there is a tendency towards depression), and generates a state assessment report for the first user based on the first result.

[0068] The established state recognition model is pre-trained based on state recognition data samples, which can be data from a constructed sample database. The electronic device backpropagates the total loss value of the state recognition model within the model. During the backpropagation of the total loss value to each layer of the model, the gradient of the loss function is calculated based on the total loss value, and the weight parameters backpropagated to the current layer are updated along the descent direction of the gradient.

[0069] In this embodiment, a first dataset representing the state data of a first user is obtained from a blockchain system. The first dataset is divided into at least two second datasets corresponding to different data types. A first feature tensor corresponding to each second dataset is determined. The first feature tensor corresponding to each of the at least two second datasets is input into a set state recognition model. A first result is output to generate a state evaluation report for the first user. In this way, by using the set state recognition model to analyze the feature tensors corresponding to at least two types of datasets representing psychological state data, the output result is more closely matched with the user's psychological state, thereby improving the accuracy of the user's psychological state recognition result.

[0070] Meanwhile, to ensure the privacy of collected user status data, it is typically stored using designated storage media, which suffers from low data storage reliability. In this embodiment, a blockchain system is used to achieve distributed storage of status data, ensuring the reliability of user psychological data storage.

[0071] In one embodiment, the step of dividing the acquired first dataset to obtain at least two second datasets includes:

[0072] Obtain the encrypted data of the first dataset corresponding to the first user from the blockchain system;

[0073] The first dataset is decrypted using the first key corresponding to the first user to obtain the third dataset;

[0074] The third dataset is divided to obtain at least two second datasets; wherein,

[0075] The first key is generated based on the information set by the first user.

[0076] The first dataset is pre-encrypted using the first key corresponding to the first user to obtain the ciphertext data of the first dataset. After the first node obtains the first dataset of the first user from the blockchain system, it decrypts the ciphertext data of the first dataset based on the first key corresponding to the first user to obtain the plaintext third dataset. The third dataset is then divided according to the set rules to obtain at least two second datasets.

[0077] Here, the first key is generated based on information set by the first user. It can be generated based on the first user's user identifier and a string set by the first user, where the user identifier includes the user ID. The setting rules can be based on the format and / or collection method of the first data. In this way, only a specific user can access the state dataset corresponding to that user, ensuring the reliability and security of user psychological data storage.

[0078] In practical applications, nodes in a blockchain can achieve data confidentiality during data transmission through the following steps:

[0079] Upload:

[0080] Step 1: For the third dataset that needs to be encrypted, use the first key to perform hash encryption. The first key is uniquely generated by the user's user ID and the set information.

[0081] Step 2: For the third dataset that needs to be encrypted, generate the ciphertext information of the hash string using the first key generated in step 1.

[0082] Step 3: Write the encrypted information of the hash string into the block, and append the user ID at the end of the block to realize the upload of user status data.

[0083] download:

[0084] Step 1: Synchronize the entire blockchain data to your local machine.

[0085] Step 2: Traverse the entire blockchain and find the last block in the blockchain that matches the user ID.

[0086] Step 3: For the matching block, decrypt it using the first key. Obtain the first user's status information.

[0087] Since the first key is generated and owned independently by each first user, even if each first user obtains all the ciphertext data, they can only obtain and decrypt their own state data and cannot obtain the state data of other users, thus maintaining data security.

[0088] In one embodiment, a third dataset of plaintext data can be encrypted and decrypted using attribute-based encryption (ABE). Only the first user whose user attributes meet the set policy can obtain the third dataset of plaintext data, thereby achieving the effect that only specific users can obtain the corresponding state dataset, ensuring the reliability and security of user psychological data storage.

[0089] In one embodiment, before obtaining the first dataset corresponding to the first user from the blockchain system, the method further includes:

[0090] Determine at least one first data point corresponding to the first user;

[0091] At least one first piece of data is determined and sent to the blockchain system.

[0092] The first node in the blockchain system determines at least one piece of first data corresponding to the first user and sends this data to the blockchain system. The first node then retrieves the first user's first dataset from the blockchain system. Here, the first node can determine all the first data or only a portion of it, with other nodes determining the remaining data. The at least one piece of first data sent by the first node to the blockchain system can be packaged together with the first data sent by other nodes and written into a block in the blockchain. The node that determines the first data can directly upload the data to the blockchain system and retrieve the first user's first dataset stored in the block when state recognition is required. This utilizes the blockchain system to achieve distributed storage of state data, ensuring the reliability of user psychological data storage. Simultaneously, each node can upload data independently; the nodes sending data to the blockchain system are decoupled, preventing incomplete information from preventing the entire data from being uploaded.

[0093] In one embodiment, determining at least one first piece of data corresponding to the first user includes:

[0094] Invoke at least one acquisition module of the status acquisition device to acquire at least two of the following data from the first user:

[0095] First scale data;

[0096] First image data;

[0097] First audio data;

[0098] Primary vital signs data;

[0099] The first image data represents the user's facial data.

[0100] The first node in the blockchain system collects at least two data points from the first user's first scale data, first image data, first audio data, and first vital sign data by calling at least one acquisition module of the state acquisition device. This first node then retrieves the first user's first dataset from the blockchain system. Here, the first image data represents the user's facial data, and the first vital sign data includes one or more physiological parameters such as heart rate, heart rate variability, pulse intensity, pulse rise time, pulse fall time, the ratio of pulse rise time to fall time, and skin resistance.

[0101] In this way, when identifying a user's psychological state based on the established state recognition model, analysis can be performed based on data from more dimensions, making the output results more consistent with the user's psychological state and thus improving the accuracy of the user's psychological state recognition results.

[0102] In one embodiment, the first node can perform at least one function of collecting first scale data, first image data, first audio data, and first vital sign data. If the first node can only collect some of the above data, the data that cannot be collected will be collected by at least one node in the blockchain system, ensuring that the first dataset includes at least the above four types of data.

[0103] In this way, the distributed storage of state data using a blockchain system ensures the reliability of user psychological data storage. Simultaneously, each node collecting data from the first user can independently upload data, and the nodes sending data to the blockchain system are decoupled, preventing incomplete information collection from causing all information to fail to upload.

[0104] In one embodiment, the first data further includes the personal information of the first user.

[0105] When determining at least one piece of first data corresponding to a first user, the first data also includes the first user's personal information. Here, the first user's personal information includes gender, age, family history of mental illness, and medical records. Medical records include the user's physical examination reports, medical records, test results, treatment outcomes, and prescriptions. This allows users to be categorized into defined user categories based on their personal information. Thus, when identifying a user's psychological state based on a defined state recognition model, the analysis combines the psychological state of the user's category with the data from multiple dimensions, making the output more closely match the user's psychological state and improving the accuracy of the user's psychological state identification results.

[0106] In one embodiment, when sending at least one determined first piece of data to the blockchain system, the method includes:

[0107] Predict the first data point at time i+1;

[0108] When the first data of the i-th time node and the predicted first data of the (i+1)-th time node satisfy the set relationship, the first data of the i-th time node is sent to the blockchain system.

[0109] Based on the first user's first dataset, predict the first data at the (i+1)th time node. Determine whether the first data at the ith time node and the predicted first data at the (i+1)th time node satisfy a set relationship: if the set relationship is satisfied, the first data at the ith time node is considered trustworthy, and the first data at the ith time node is sent to the blockchain system; if the set relationship is not satisfied, the first data at the ith time node is considered untrustworthy, and the first data at the ith time node is not sent to the blockchain system.

[0110] Here, the first user's first dataset includes data from the first time point to the i-th time point.

[0111] In this way, by analyzing data from historical time points, the reliability of the data is analyzed each time it is uploaded. Data that is deemed reliable is sent to the blockchain system, preventing a large amount of useless information from polluting the blocks and wasting block data.

[0112] Determining whether to send the first data at time node i to the blockchain system mainly includes the following steps:

[0113] Step 1: According to formula (1), fit the data using the first historically collected data:

[0114]

[0115] Among them, y t This is the first data collected at time point t, where μ is a constant term and γ is a variable. i ε is the autocorrelation coefficient at the i-th time point. t This is the error term.

[0116] Find a set of μ, γ i ,make Minimum.

[0117] Step 2: Predict the first data point for the future time node based on formula (2):

[0118]

[0119] Among them, y t+1 This is the first data collected at the (t+1)th time node.

[0120] Step 3: Determine whether the first data at time node t and the predicted first data at time node (t+1) satisfy the set relationship according to formula (3):

[0121]

[0122] Here, the judgment parameter α is set based on experience, usually 0.5. For different types of first data, machine learning models can also be used for training to obtain the optimal judgment parameters.

[0123] Step 4: If the set relationship is satisfied, the first data of the i-th time node is considered trustworthy data, and the first data of the i-th time node is sent to the blockchain system; if the set relationship is not satisfied, the first data of the i-th time node is considered untrustworthy data, and the first data of the i-th time node is not sent to the blockchain system.

[0124] In one embodiment, the method further includes:

[0125] A state assessment report is generated based on the first result output by the defined state recognition model.

[0126] The generated status assessment report is sent to the blockchain system.

[0127] A state assessment report is generated based on the first result output by the defined state recognition model and sent to the blockchain system. This report records the historical and current psychological state data and assessment results of at least one user. The historical and current psychological state data reflects changes in psychological state and predicts future psychological states. When the first result of at least one user meets defined conditions, such as a score below a defined threshold, these users are recorded in the state assessment report, and a corresponding state assessment report for that user is generated.

[0128] In this way, based on the status assessment report, the psychological state of at least one primary user can be determined so that timely intervention can be carried out.

[0129] In practical applications, all employees of a company can be considered as the primary users, and a status assessment report can be generated for the company to analyze and track changes in the psychological state of all employees.

[0130] like Figure 2 The schematic diagram of the state recognition system shown in the embodiments of this application includes a scale module, a face acquisition module, a voice acquisition module, a vital sign acquisition module, and a multi-dimensional data fusion analysis module. In various embodiments of this application, the first node of the state recognition method includes at least the multi-dimensional data fusion analysis module.

[0131] Scale Module: Used to collect data from the first scale. This module collects data through online user responses to the first scale. It includes different psychological test scales. The responses to these scales by the first user are encoded to obtain a two-dimensional first feature tensor.

[0132] One-hot encoding is used. The number of rows in the two-dimensional feature tensor is the number of questions, and the number of columns is the maximum number of options in the questions of the psychological test scale. When the user selects the y-th option for the x-th question, the element is set to 1.

[0133] For example, in question 1, the first user chooses C; in question 2, the first user chooses AC; and in question 3, the first user chooses B. The corresponding two-dimensional first feature tensor is:

[0134] The scale module includes one or more of the following: timer, single-question answering time, single-question hesitation time for each item in the test scale, total answering time for each item in the first scale data corresponding to each factor in the test scale, average answering time for each item in the first scale data corresponding to each factor, average single-question answering time for each item in the first scale data corresponding to each factor, and the ratio of the total answering time for each item in the first scale data corresponding to each factor to the total answering time for each item in the first scale data. Due to the scale test items, the user's existing knowledge, and concerns about the assessment results, the user experiences psychological conflict, which not only prolongs the answering time for the corresponding items in the first scale data but also causes fluctuations in bodily signs. By analyzing the collected first data, the user's psychological state can be identified.

[0135] Face capture module: Used to capture initial image data. It extracts each frame of video as an image through video capture, recognizes the user's facial expressions, and analyzes the user's emotions.

[0136] The encoding process of the image determined by the face acquisition module, such as... Figure 3 The flowchart shown below illustrates the face capture image encoding process, which includes the following steps:

[0137] Step 1: Turn on the camera. If it is on, continue to the next step; otherwise, go back and try again.

[0138] Step 2: Divide the captured video into frames, turning the continuous video into a single image.

[0139] Step 3: Convert a single image to grayscale.

[0140] Step 4: Use a detector to detect faces in each image. If a face is detected, continue the step; otherwise, output the message "no face". Commonly used face detectors include the Haarcascade detector.

[0141] Step 5: Use a convolutional neural network to detect key points of the face in the image, and obtain the results of the key point labeling in the image.

[0142] Step 6: Calculate key point parameters.

[0143] The relative distance between the vertical coordinates of the upper and lower edges of the mouth is mouth_hight. The interpolation of the average vertical coordinates of the two corners of the mouth with the average vertical coordinates of the upper and lower edges of the mouth is brow_k. The average interpolation of the upper and lower edges of the eyes is eye_hight.

[0144] Step 7: According to the flowchart, if the key point parameters meet the threshold conditions, it is a certain expression. The candidate expressions here can be natural, angry, surprised, or happy.

[0145] Define a feature tensor. When it is determined that an image corresponds to a certain expression, set the corresponding element of the feature tensor to 1.

[0146] The face capture module can use, for example, Figure 4 The schematic diagram shown includes an ambient light detection unit 1, an infrared light source 2, an image sensor 3, a control unit 4, a display screen 5, a main central processing unit (CPU) 6, a filter rotation mechanism 7, and a power module 8. The ambient light detection unit 1, image sensor 3, control unit 4, and display screen 5 are electrically connected to the main CPU 6; the filter rotation mechanism 7 and infrared light source 2 are electrically connected to the control unit 4; and all four components—ambient light detection unit 1, infrared light source 2, image sensor 3, control unit 4, display screen 5, main CPU 6, and filter rotation mechanism 7—are connected to the power module 8.

[0147] The filter rotation mechanism 7 consists of an infrared cut-off filter 9, a full-pass lens 10, and a rotary motor 11 (e.g., Figure 5 (As shown).

[0148] It should be noted that when the face acquisition module is working, the ambient light detection unit 1 detects the current ambient light illuminance, sends the detected value to the main CPU 6 for calculation, and compares the calculated light illuminance value with a preset value, which is a pre-set light illuminance value. When the calculated light intensity value is greater than the preset value, the main CPU 6 determines that the current light is sufficient and sends a control command to the control unit 4 to turn off the infrared light source 2. The control unit 4 controls the rotary motor 11 to rotate to the all-pass lens 10. At this time, the ambient visible light irradiates the face and the reflected light enters the image sensor 3 to form an image. The image data is processed by the main CPU 6 and displayed on the display screen 5. The image obtained in this case is a color image, which is convenient for judging a person's physiological characteristics and behavior. When the calculated light intensity value is less than the preset value, the main CPU 6 determines that the current light is weak and sends a control command to the control unit 4 to turn on the infrared light source 2. The control unit 4 controls the rotary motor 11 to rotate to the infrared cutoff filter 9. The infrared light irradiates the face area being tested. At this time, the infrared light reflected by the face enters the image sensor 3 to form an image. The image data is processed by the main CPU 6 and displayed on the display screen 5.

[0149] The audio acquisition module is used to collect primary audio data, enabling functions such as speech acquisition, text conversion, semantic analysis, and emotion recognition. It captures the user's speech audio using an array microphone. After converting the speech to text, semantic analysis is performed to determine the user's emotion. Speech acquisition is completed via the array microphone, while speech-to-text, semantic analysis, and emotion recognition are all implemented through a third-party AI API. This recognition method can provide an assessment of the user's emotion, yielding emotional tendency results: positive, neutral, and negative. The detected classification results include natural, angry, surprised, and happy.

[0150] The quantization coding method is similar to the encoding process of images determined by the face acquisition module. It uses prosodic features such as duration, pitch, and energy; sound quality features such as formant frequency and bandwidth, frequency perturbation and amplitude perturbation, and glottal parameter; and spectrum-based correlation features such as Mel Frequency Cepstrum Coefficient (MFCC), Linear Prediction Coefficients (LPC), Linear Predictive Cepstrum Coefficient (LPCC), or a combination of two or more of the above features.

[0151] Vital Signs Acquisition Module: Used to collect primary vital sign data. The module collects the user's vital signs indicators through acquisition devices, including one or more of the following: heart rate, heart rate variability, pulse intensity, pulse rise time, pulse fall time, the ratio of pulse rise time to fall time, and skin resistance. Based on the established vital sign-emotion model, the module analyzes the user's emotional and psychological state.

[0152] The multidimensional data fusion and analysis module analyzes point data from psychological assessment scales using the scale module, and combines this with line data from video and audio acquisition modules to analyze speech data, thereby extracting long-term psychological state data for the user. The model outputs a score assessing the user's psychological state.

[0153] like Figure 6 As shown, the status recognition system may also include a personal information processing module.

[0154] The user's personal information includes their age, gender, medical records, and family history of mental illness. Natural Language Processing (NLP) is used to process the unstructured data in this personal information to obtain the corresponding structured data.

[0155] This application proposes a multi-dimensional data fusion analysis model. By acquiring the user's facial images and voice information, combined with scale data, vital sign data, and personal information, the model is trained using multi-dimensional data. The model is then analyzed using a set state recognition model to analyze the feature tensors corresponding to at least two types of datasets representing psychological states, making the output results more closely match the user's psychological state, thereby improving the accuracy of the user's psychological state recognition results.

[0156] Based on the outputs of the scale module, face capture module, voice capture module, and personal information processing module, the state recognition model of the multidimensional data fusion analysis module is trained. The training includes the following steps:

[0157] Step 1: Obtain the training sample dataset. The training sample dataset includes psychological assessment scale samples, facial image samples, audio samples, and personal information for each user. The labeling information in the sample data consists of psychological recognition scores determined by experts based on scales, voice, video, and other information.

[0158] Step 2: Features representing different data have different weights in the model. Before the model starts training, the weight parameters are randomly initialized.

[0159] Step 3: The above four two-dimensional matrices are fused in the neural network DNN.

[0160] Step 4: During model training, adjust the weight parameters for each feature using backpropagation. Training is complete once the model's relevant metrics (ACC, Loss) reach their respective thresholds.

[0161] The model output is the first result for the first user, and the psychological state recognition result is represented by a score.

[0162] The state recognition system applies a six-layer structure of blockchain, consisting of data layer, network layer, consensus layer, incentive layer, contract layer and application layer, forming a decentralized platform.

[0163] Data Layer: The data layer encapsulates the chain structure of the underlying data blocks, involving related asymmetric public-private key encryption technologies, timestamps, hash functions, and Merkle trees, among other technical elements. It is the lowest-level modem in blockchain technology. The data layer constitutes the most basic yet most important information of this platform. At the data layer, information collected by modules such as the scale module, face capture module, voice capture module, and vital sign capture module is written to the blockchain. Nodes that obtain the right to record transactions must add a timestamp to indicate the time the data was written, ensuring that subsequent nodes cannot modify previous data and guaranteeing the immutability of information.

[0164] Network Layer: As a medium for data processing, the network layer propagates and verifies data from the data layer, automatically forming a network and utilizing information obtained through various channels to formally create a platform with decentralized characteristics. However, due to the privacy requirements of psychological data, the network layer of the multidimensional psychological assessment platform employs new encryption methods. Only specific users can access their corresponding state datasets, ensuring the reliability and security of user psychological data storage.

[0165] In practical applications, nodes in a blockchain can achieve data confidentiality during data transmission through the following steps:

[0166] Upload:

[0167] Step 1: For the third dataset that needs to be encrypted, use the first key to perform hash encryption. The first key is uniquely generated by the user's user ID and the set information.

[0168] Step 2: For the third dataset that needs to be encrypted, generate the ciphertext information of the hash string using the first key generated in step 1.

[0169] Step 3: Write the encrypted information of the hash string into the block, and append the user ID at the end of the block to realize the upload of user status data.

[0170] download:

[0171] Step 1: Synchronize the entire blockchain data to your local machine.

[0172] Step 2: Traverse the entire blockchain and find the last block in the blockchain that matches the user ID.

[0173] Step 3: For the matching block, decrypt it using the first key. Obtain the first user's status information.

[0174] Consensus Layer: The consensus layer primarily encapsulates various consensus mechanism algorithms for network nodes and is the core technology of the entire blockchain. Currently, the main consensus algorithms used include Proof of Work (PoW), Proof of Stake (PoS), and Delegated Proof of Stake (DPoS). Combining the characteristics of each mechanism, the Delegated Proof of Stake mechanism is applied to a multi-dimensional psychological assessment system. DPoS professionalizes the role of the ledger clerk in PoS, selecting the clerk through stake, and then the clerks take turns recording transactions. That is, the scale module, face capture module, voice capture module, and vital sign capture module can all upload data. The modules sending data to the blockchain system are decoupled, avoiding the inability to upload all information due to incomplete information.

[0175] Incentive Layer: The incentive layer integrates economic factors into the blockchain technology system, mainly including the issuance and distribution mechanisms of economic incentives. Within the system, the system will judge the credibility of uploaded data, incentivizing and recording correctly interpreted information, while rejecting forged information to prevent a large amount of useless information from polluting the blocks and wasting block data.

[0176] Determining the trustworthiness of uploaded data mainly involves the following steps:

[0177] Step 1: According to formula (1), fit the first data collected in history to find a set of μ, γ i ,make Minimum.

[0178] Step 2: Predict the first data for future time nodes based on formula (2).

[0179] Step 3: Determine whether the first data of the t-th time node and the predicted first data of the (t+1)-th time node satisfy the set relationship according to formula (3).

[0180] Step 4: If the set relationship is satisfied, the first data of the i-th time node is considered reliable data, and the first data of the i-th time node is written into the block; if the set relationship is not satisfied, the first data of the i-th time node is considered unreliable data, and the first data of the i-th time node is not written into the block.

[0181] The physical links and implementation of this blockchain system are described below:

[0182] like Figure 7As shown, the aforementioned distributed database system includes a server management layer, a request processing layer, and a data storage layer connected via an internal gigabit or higher network. The gigabit or higher network ensures the flow of data information and control information between worker nodes, as well as the management of the liveness and status of each worker node by the master server.

[0183] The server management layer is used to provide node management, resource management, fault handling, storage balancing, global lock management, and arbitration services. It includes two synchronously running master control servers with identical structures and synchronized data. Its structural model is a central service management model, which manages the global storage layout, various catalog information, and the running status information of each working node.

[0184] The request processing layer is used to receive user requests, plan tasks, and execute tasks. It includes several worker servers, which together form multiple worker nodes that can accept user requests. The worker nodes have the same structure and can be replaced by each other. The worker nodes form a network. Users only need to connect to any one worker node to connect to the entire database, without needing to connect to the master server, thus reducing the workload of the master server. Furthermore, if any worker node fails, it can be replaced by another worker node without affecting the operation of the entire database, achieving efficient, stable, and 24 / 7 availability of the system.

[0185] The data storage layer is used to store data and respond to operation commands from worker nodes. It includes several storage servers, which together form multiple storage nodes capable of storing data. The data storage layer adopts a Share Nothing architecture for distributed storage and a multi-replica fault-tolerant model. That is, data is stored in units of storage fragments, distributed across storage nodes. Each storage fragment is stored in multiple replicas on different storage nodes. Specifically, as shown... Figure 8 As shown in Table 1, the data storage consists of multiple storage segments distributed across different storage nodes (storage node 1, storage node 2, and storage node 3). Reading and writing can be performed simultaneously on multiple storage servers, improving efficiency. As the smallest storage unit, a storage segment can be distributed across any storage server, enabling load balancing when multiple storage operations are concurrent. Each storage segment has multiple replicas (tablet_5 is stored on storage nodes 1, 2, and 3). If any node fails and a replica is corrupted, a backup is available to switch over, achieving high availability. When the storage pressure on each storage node is uneven, the master server initiates storage migration to ensure balanced storage load. When there are too few data replicas, storage repair occurs, and replicas are created on the remaining available nodes to ensure the number of data replicas and improve data availability.

[0186] In this embodiment, multi-dimensional data analysis and emotion judgment are used to achieve a comprehensive approach combining point data and line data, thereby enabling long-term psychological assessment of users. Blockchain technology is employed to achieve distributed storage of user psychological data assets, thereby improving the reliability and security of user psychological data storage. By integrating multi-dimensional psychological assessment technology and blockchain data storage technology, both long-term assessment of users' psychological states and the reliability, security, and immutability of this massive monitoring data storage are achieved.

[0187] To implement the method of this application embodiment, this application embodiment also provides a state recognition device, which is set on the first node in the blockchain system. The first node is an electronic device such as a terminal or server. Figure 9 As shown, the device includes:

[0188] The acquisition unit 901 is used to acquire the first dataset corresponding to the first user from the blockchain system;

[0189] The first determining unit 902 is used to divide the acquired first dataset into at least two second datasets, and determine the first feature tensor corresponding to each second dataset; each of the at least two second datasets corresponds to a different data type;

[0190] Output unit 903 is used to input the first feature tensor corresponding to each of the at least two second datasets into a set state recognition model and output a first result; wherein,

[0191] The first dataset includes the first data collected from the first user; the first data of the first user represents the status data of the first user; the first result is used to generate a status assessment report for the first user.

[0192] In one embodiment, the first determining unit 902 is configured to:

[0193] Obtain the encrypted data of the first dataset corresponding to the first user from the blockchain system;

[0194] The first dataset is decrypted using the first key corresponding to the first user to obtain the third dataset;

[0195] The third dataset is divided to obtain at least two second datasets; wherein,

[0196] The first key is generated based on the information set by the first user.

[0197] In one embodiment, the apparatus further includes:

[0198] The second determining unit is used to determine at least one first data corresponding to the first user;

[0199] The first sending unit is used to send at least one determined first data to the blockchain system.

[0200] In one embodiment, the second determining unit is configured to:

[0201] Invoke at least one acquisition module of the status acquisition device to acquire at least two of the following data from the first user:

[0202] First scale data;

[0203] First image data;

[0204] First audio data;

[0205] Primary vital signs data;

[0206] The first image data represents the user's facial data.

[0207] In one embodiment, the first data may also include the personal information of the first user.

[0208] In one embodiment, the first transmitting unit is configured to:

[0209] Predict the first data point at time i+1;

[0210] When the first data of the i-th time node and the predicted first data of the (i+1)-th time node satisfy the set relationship, the first data of the i-th time node is sent to the blockchain system.

[0211] In one embodiment, the apparatus further includes:

[0212] The reporting unit is used to generate a state assessment report based on the first result output by the set state recognition model;

[0213] The second sending unit is used to send the generated status assessment report to the blockchain system.

[0214] In practical applications, the acquisition unit 901, the first sending unit, and the second sending unit can be implemented based on the communication interface in the state recognition device, and the first determining unit 902, the output unit 903, the second determining unit, and the reporting unit can be implemented based on the processor in the state recognition device, such as a CPU, a digital signal processor (DSP), a microcontroller unit (MCU), or a field-programmable gate array (FPGA).

[0215] It should be noted that the state recognition device provided in the above embodiments is only illustrated by the division of the above program modules when performing state recognition. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the state recognition device and the state recognition method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0216] Based on the hardware implementation of the above program modules, and in order to implement the state recognition method of this application embodiment, this application embodiment also provides an electronic device, such as... Figure 10 As shown, the electronic device 1000 includes:

[0217] The communication interface 1010 enables information exchange with other devices, such as network devices.

[0218] The processor 1020, connected to the communication interface 1010, enables information interaction with other devices and, when running a computer program, executes the methods provided by one or more technical solutions applied to the first node in the blockchain system. The computer program is stored in the memory 1030.

[0219] Of course, in practical applications, the various components in electronic device 1000 are coupled together through bus system 1040. It can be understood that bus system 1040 is used to realize the connection and communication between these components. In addition to a data bus, bus system 1040 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 10 The general labeled all buses as Bus System 1040.

[0220] The memory 1030 in this embodiment is used to store various types of data to support the operation of the electronic device 1000. Examples of such data include any computer program used to operate on the electronic device 1000.

[0221] It is understood that memory 1030 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 1030 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0222] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1020. The processor 1020 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 1020 or by instructions in the form of software. The processor 1020 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1020 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 1030. The processor 1020 reads the program in the memory 1030 and completes the steps of the aforementioned method in conjunction with its hardware.

[0223] Optionally, when the processor 1020 executes the program, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0224] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 1030 storing a computer program, which can be executed by a processor 1020 of an electronic device to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed apparatus, electronic devices, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0226] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0227] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0228] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0229] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0230] It should be noted that the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict. Unless otherwise stated and limited, the term "connection" should be interpreted broadly. For example, it can refer to an electrical connection, or the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above term according to the specific circumstances.

[0231] Furthermore, in the examples of this application, "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the objects distinguished by "first," "second," and "third" can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than those illustrated or described herein.

[0232] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0233] The specific technical features described in the various embodiments in the detailed implementation can be combined in various ways without contradiction. For example, different implementation methods can be formed by combining different specific technical features. In order to avoid unnecessary repetition, the various possible combinations of the specific technical features in this application will not be described separately.

Claims

1. A state recognition method, applied to the first node in a blockchain system, characterized in that, The method includes: Obtain the first dataset corresponding to the first user from the blockchain system; The first dataset is divided into at least two second datasets, and the first feature tensor corresponding to each second dataset is determined; each of the at least two second datasets corresponds to a different data type; data with a collection time interval of less than a set threshold are identified as data collected during the same psychological state recognition. Each of the at least two second datasets is input into the same set psychological state recognition model, and a first result is output; wherein, the first dataset includes the first data collected from the first user; the first data of the first user represents the state data of the first user; the first result is used to generate a psychological state assessment report of the first user; Before obtaining the first dataset corresponding to the first user from the blockchain system, the method further includes: determining at least one first data corresponding to the first user; and sending the determined at least one first data to the blockchain system. When sending at least one determined first data point to the blockchain system, the method includes: predicting the first data point at the (i+1)th time node; when the first data point at the i-th time node and the predicted first data point at the (i+1)th time node satisfy a set relationship, sending the first data point at the i-th time node to the blockchain system; the set relationship includes the predicted first data point at the (i+1)th time node having a change in degree relative to the first data point at the i-th time node that is less than a set threshold.

2. The state recognition method according to claim 1, characterized in that, The first dataset is divided to obtain at least two second datasets, including: Obtain the encrypted data of the first dataset corresponding to the first user from the blockchain system; The first dataset is decrypted using the first key corresponding to the first user to obtain the third dataset; The third dataset is divided to obtain at least two second datasets; wherein, The first key is generated based on the information set by the first user.

3. The state recognition method according to claim 1, characterized in that, Determining at least one piece of first data corresponding to the first user includes: Invoke at least one acquisition module of the status acquisition device to acquire at least two of the following data from the first user: First scale data; First image data; First audio data; Primary vital signs data; The first image data represents the user's facial data.

4. The state recognition method according to claim 1, characterized in that, The first data also includes the first user's personal information.

5. The state recognition method according to any one of claims 1 to 4, characterized in that, The method further includes: A psychological state assessment report is generated based on the first result output by the established psychological state recognition model. The generated psychological state assessment report is sent to the blockchain system.

6. A state recognition device, applied to the first node in a blockchain system, characterized in that, The device includes: An acquisition unit is used to acquire a first dataset corresponding to a first user from the blockchain system; The first determining unit is used to divide the acquired first dataset into at least two second datasets and determine the first feature tensor corresponding to each second dataset; each of the at least two second datasets corresponds to a different data type; data with a collection time interval of less than a set threshold are identified as data collected during the same psychological state recognition. The output unit is configured to input the first feature tensor corresponding to each of the at least two second datasets into the same predefined mental state recognition model and output a first result; wherein, The first dataset includes the first data collected from the first user; the first data of the first user represents the state data of the first user; the first result is used to generate a psychological state assessment report for the first user; The second determining unit is used to determine at least one first data corresponding to the first user; The first sending unit is used to send at least one determined first data to the blockchain system; The first sending unit is further configured to: predict the first data of the (i+1)th time node; and when the first data of the i-th time node and the predicted first data of the (i+1)th time node satisfy a set relationship, send the first data of the i-th time node to the blockchain system. The defined relationship includes the fact that the degree of change of the first data at the (i+1)th time node relative to the first data at the ith time node is less than a defined threshold.

7. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor runs the computer program, it performs the steps of the state recognition method according to any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the state recognition method as described in any one of claims 1 to 5.

Images