Emotional portrait tracking method and device based on multi-site galvanic skin technology

Abstract

The invention provides an emotion portrait tracking method and device based on a multi-site skin electric technology, and relates to the technical field of data processing, and the method comprises the steps: obtaining the multi-site skin electric signal features of a target individual; inputting the multi-site skin electric signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; wherein the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion tag data set comprises multi-dimensional emotion tags obtained by dynamically marking a plurality of individuals based on own emotion experiences in a preset scene. Through the method provided by the invention, accurate depiction of the emotion portrait is realized, and the dynamic depiction method has the characteristics of high efficiency, convenience and low cost.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an emotion portrait tracking method and device based on multi-site electrical skin technology. Background Art

[0002] Diverse and complex learning emotional experiences (for example, confusion, joy, and curiosity in learning) are not only an important part of the learning process, but also affect how students view and perform learning tasks, and are closely related to academic performance. At the same time, the complexity of the learning process also makes it inevitable for students to experience complex and multi-dimensional learning emotions. Learning emotions of different dimensions are intertwined and together form a learning emotion portrait (that is, the holistic and multi-dimensional learning emotions in the learning process). Real-time and accurate portrayal of learning emotion portraits is expected to provide theoretical and technical support for achieving student-centered teaching and providing personalized teaching feedback.

[0003] However, in traditional educational research, in order not to interfere with the learning process, researchers often use retrospective self-reporting and other methods to obtain students' learning emotion portraits, which reflect the overall learning emotion feelings during a period of learning, and cannot depict the dynamic changes of learning emotion portraits on a finer time scale, thus restricting the space for real-time feedback and intervention.

[0004] How to achieve dynamic and accurate portrayal of learners' emotional portraits is a technical problem that needs to be solved at present. Summary of the invention

[0005] The present invention provides an emotion portrait tracking method and device based on multi-site electrical skin technology, so as to solve the defects existing in the prior art.

[0006] The present invention provides an emotion portrait tracking method based on multi-site electrical skin technology, comprising the following steps: Acquire multi-site skin electrical signal characteristics of the target individual; Inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0007] According to the emotion portrait tracking method based on multi-site electrical skin technology provided by the present invention, the acquisition of multi-site electrical skin signal characteristics of the target individual includes: The multi-site skin electrical signals of the target individual are obtained, the skin electrical signals are processed through a sliding window, and the multi-site skin electrical signal features in a short-time window corresponding to each time point are calculated.

[0008] According to the emotion portrait tracking method based on multi-site skin electrical technology provided by the present invention, before inputting the multi-site skin electrical signal data of the target individual into the pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual, the method also includes: Under a preset scenario, scene video information of each of the multiple individuals is obtained; wherein the scene video information includes: facial expressions and body movements of the corresponding individual; A multidimensional emotion label is obtained by dynamically labeling each individual among the multiple individuals based on the corresponding scene video information and the individual's own emotion experience, so as to construct the multidimensional emotion label data set.

[0009] According to the emotion portrait tracking method based on multi-site electrical skin technology provided by the present invention, the training process of the emotion portrait prediction model includes: Acquire historical multi-site skin electrical signals of the multiple individuals, process the historical multi-site skin electrical signals through a sliding window, and calculate the historical multi-site skin electrical signal features within a short-time window corresponding to each time point; Time-matching the historical multi-site skin electrical signal features in the short-term window corresponding to each time point with the multi-dimensional emotion label data set to obtain a multi-dimensional emotion label feature vector set; The multi-label neural network model is trained based on the multi-dimensional emotion label feature vector set to obtain the emotion portrait prediction model.

[0010] According to an emotion portrait tracking method based on multi-site skin conductance technology provided by the present invention, the historical multi-site skin conductance signals of the multiple individuals are obtained, the historical multi-site skin conductance signals are processed through a sliding window, and the historical multi-site skin conductance signal features in a short-time window corresponding to each time point are calculated, including: Acquiring historical multi-site electrical skin signals of the multiple individuals; For each short-term window of each site, extracting the skin electrical response and skin conductance level in the historical multi-site skin electrical signal based on a continuous decomposition analysis technique; The skin conductance response is used to reflect the rapid sympathetic nerve activity caused by emotional events or stimuli, and the skin conductance level is used to reflect the overall arousal level.

[0011] According to the emotion portrait tracking method based on multi-site electrical skin technology provided by the present invention, after inputting the multi-site electrical skin signal characteristics of the target individual into a pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual, the method further includes: Obtaining emotional state feedback information of the target individual; The multi-dimensional emotion portrait prediction result of the target individual is adjusted based on the emotion state feedback information of the target individual to obtain a target multi-dimensional emotion portrait result of the target individual.

[0012] The present invention also provides an emotion portrait tracking device based on multi-site electrical skin technology, comprising the following modules: An acquisition module, used to acquire multi-site skin electrical signal characteristics of a target individual; A prediction module, used for inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an emotion portrait tracking method based on multi-site electrical skin technology as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for tracking emotion portraits based on multi-site electrical skin technology as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned emotion portrait tracking methods based on multi-site electrical skin technology.

[0016] The present invention provides an emotion portrait tracking method and device based on multi-site skin electrical technology, which obtains the multi-site skin electrical signal characteristics of the target individual; inputs the multi-site skin electrical signal characteristics of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; wherein the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences under preset scenarios. It can be seen that the present invention collects multi-site skin electrical signals of the target individual and inputs them into a pre-established emotion portrait prediction model to achieve accurate characterization of the emotion portrait, and this dynamic characterization method is characterized by high efficiency, convenience and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the emotion portrait tracking method based on multi-site galvanic skin technology provided by the present invention.

[0019] Figure 2 This is a schematic diagram of multi-site skin electricity collection provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of an emotion portrait tracking device based on multi-site electrical skin technology provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Combine the following Figure 1-Figure 4 The present invention describes an emotion portrait tracking method and device based on multi-site electrical skin technology.

[0024] It should be noted that human skin electrical activity may be one of the most promising physiological signals in the representation of human emotions. Compared with brain signal indicators such as EEG and near infrared, the acquisition cost of skin electrical activity is low, the equipment is simple to use and easy to wear for a long time, and it has better practical potential. Compared with commonly used physiological indicators such as heart rate and respiratory rate, skin electrical activity indicators represented by skin electrical response and skin conductance level can track the temporal dynamics of emotions more quickly on a sub-second time scale. More importantly, previous studies have demonstrated the body skin electrical representation of emotions and demonstrated the feasibility of using body skin electrical signals for emotion computing. However, existing studies have only explored the separability of skin electricity between high and low arousal emotional levels and the three emotional valences of positive, negative and neutral. There is no report on the recognition of different categories of emotions, especially different categories of learning emotions. Moreover, compared with directly dividing the emotion categories, directly considering the integrity and multi-dimensionality of learning emotions, and realizing the characterization of learners' emotional portraits has not been reported. Based on this, the present invention provides an emotion portrait tracking method based on multi-site skin electrical technology to solve the above problems.

[0025] Figure 1 : is a flow chart of the emotion portrait tracking method based on multi-site electrical skin technology provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 100: Acquire multi-site skin electrical signal characteristics of the target individual.

[0026] It should be noted that the present invention is applicable to task emotion profile tracking in multiple scenarios, and no special limitation is imposed on this. This embodiment takes a classroom learning scenario as an example, and the target individual is a student.

[0027] Specifically, step 100 obtains the multi-site skin electrical signal characteristics of the target individual, including: The multi-site skin electrical signals of the target individual are obtained, the skin electrical signals are processed through a sliding window, and the multi-site skin electrical signal features in a short-time window corresponding to each time point are calculated.

[0028] Before step 200, the construction process of the multidimensional emotion label data set provided by the embodiment of the present invention is first described, which specifically includes: Under a preset scenario, scene video information of each of the multiple individuals is obtained; wherein the scene video information includes: facial expressions and body movements of the corresponding individual; A multidimensional emotion label is obtained by dynamically labeling each individual among the multiple individuals based on the corresponding scene video information and the individual's own emotion experience, so as to construct the multidimensional emotion label data set.

[0029] In one embodiment, no less than 100 students from primary school, middle school and university are selected, and multi-point electrodermal recordings are performed on them in a real classroom. Figure 2 This is a schematic diagram of multi-site electrodermal acquisition provided by the present invention, and the recording sites are not limited to Figure 2 Skin electrical measurements were performed on nine body parts, including the wrists, legs, neck, chest, abdomen, and back, with a sampling rate of no less than 50 Hz. Classroom video information was recorded at the same time, and a high-resolution camera was used to record each student's facial expressions and body movements to provide supplementary information for the annotation and modeling of emotional states.

[0030] Furthermore, after the learning process is over, students are required to dynamically label their own emotional experiences during the class. Based on the classroom video and the playback of their own facial expressions and body movements, students will score learning emotion dimensions including but not limited to curiosity and confusion on the timeline to obtain multi-dimensional emotional labels of skin electrical signals during the class. These labels serve as the basis for supervised learning and are used to establish the association between skin electrical signals and emotional states.

[0031] Step 200: Input the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0032] Specifically, the training process of the emotion portrait prediction model provided by the embodiment of the present invention is described, which specifically includes: Step 210: Obtain historical multi-site skin electrical signals of the multiple individuals, process the historical multi-site skin electrical signals through a sliding window, and calculate the historical multi-site skin electrical signal features within a short-time window corresponding to each time point.

[0033] Step 210 specifically includes: Step 211: Acquire historical multi-site skin electrical signals of the multiple individuals.

[0034] Step 212: for each short-time window of each site, extract the skin electrical response and skin conductance level in the historical multi-site skin electrical signal based on the continuous decomposition analysis technology; The skin conductance response is used to reflect the rapid sympathetic nerve activity caused by emotional events or stimuli, and the skin conductance level is used to reflect the overall arousal level.

[0035] Step 220: Time-match the historical multi-site skin electrical signal features within the short-term window corresponding to each time point with the multi-dimensional emotion label data set to obtain a multi-dimensional emotion label feature vector set.

[0036] Step 230: training the multi-label neural network model based on the multi-dimensional emotion label feature vector set to obtain the emotion portrait prediction model.

[0037] In one embodiment, a 0.5 second sliding window is used to analyze the data to extract short-term features in dynamic emotional changes. Specifically, for each short-term window of each site, the phase (phasic) and base (tonic) components of the skin electricity, namely the skin electrical response (SCR) and the skin conductance level (SCL), are extracted based on the continuous decomposition analysis technology. SCR can reflect the rapid sympathetic nerve activity caused by emotional events or stimuli, while SCL is a slowly changing component related to the overall arousal level. The above process is carried out in parallel for different sites to obtain the characteristic vectors of the skin electricity of multiple sites in each short-term window, and time matching is performed with the emotion tags in the multi-dimensional emotion tag data set constructed in the above embodiment to obtain a set of characteristic vectors with multi-dimensional emotion tags.

[0038] In order to achieve a many-to-many mapping relationship from multi-site electrodermal data to multi-dimensional emotion indicators, a multi-label neural network (such as Multi-Label Perceptron) will be used to build the model. The network will include an input layer, a hidden layer, and an output layer. The input layer size = the number of sites The number of features; the hidden layer is a fully connected layer, which is used to learn the complex relationship of the input data; the size of the output layer should be equal to the number of emotion dimensions. The activation function will use the Sigmoid activation function to map each output to the range of [0, 1], indicating the probability of the existence of a certain emotion; the loss function uses the Binary Cross-Entropy Loss function to calculate the probability of the occurrence of the emotion. Then, the model training and performance testing are carried out based on the aforementioned set of skin electrical feature vectors with multi-dimensional emotion labels.

[0039] Furthermore, after step 200 inputs the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual, the method further includes: Obtaining emotional state feedback information of the target individual; The multi-dimensional emotion portrait prediction result of the target individual is adjusted based on the emotion state feedback information of the target individual to obtain a target multi-dimensional emotion portrait result of the target individual.

[0040] In one embodiment, for new audiences, the skin electrical signals at the same location are collected during their classroom learning, and the skin electrical signals are processed using a sliding window. At each time point, the skin electrical features within the sliding window are calculated and then input into the model to obtain the current multi-dimensional emotional portrait prediction. At the same time, this model will also add a feedback mechanism, which can collect feedback on the current emotional state of students through interaction with them (such as simple questionnaires or facial expression feedback) to adjust the model's prediction results, thereby ensuring the dynamics and accuracy of the model.

[0041] Accurate portrayal of learning emotion portraits is an important prerequisite for implementing learner-centered teaching practice programs. The emotion portrait tracking method based on multi-site electrodermal technology provided in the embodiment of the present invention is efficient, convenient and low-cost, and is expected to explore diverse and complex learning experiences from a holistic perspective, so as to have a deeper understanding of the external manifestations and internal mechanisms of the learning experience, and provide theoretical and technical support for the realization of personalized education.

[0042] The above is a description of the steps of the emotion portrait tracking method based on multi-site skin conductance technology provided by the present invention. From the description of the above steps, it can be seen that according to the emotion portrait tracking method based on multi-site skin conductance technology provided by the present invention, by obtaining the multi-site skin conductance signal characteristics of the target individual; the multi-site skin conductance signal characteristics of the target individual are input into the pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual; wherein the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: in a preset scenario, multiple individuals are dynamically labeled based on their own emotional experiences and obtained multi-dimensional emotion labels. It can be seen that the present invention collects the multi-site skin conductance signals of the target individual and inputs them into the pre-established emotion portrait prediction model to achieve accurate characterization of the emotion portrait, and this dynamic characterization method is characterized by high efficiency, convenience and low cost.

[0043] The emotion portrait tracking device based on multi-site galvanic skin technology provided by the present invention is described below. The emotion portrait tracking device based on multi-site galvanic skin technology described below and the emotion portrait tracking method based on multi-site galvanic skin technology described above can be referenced to each other.

[0044] Figure 3 is a schematic diagram of the structure of the emotion portrait tracking device based on multi-site skin conduction technology provided by the present invention. Figure 3 As shown, the emotion portrait tracking device based on multi-site skin conduction technology provided by the present invention includes: An acquisition module 301 is used to acquire multi-site skin electrical signal characteristics of a target individual; Prediction module 302, used for inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0045] The emotion portrait tracking device based on multi-site skin conductance technology provided by the present invention obtains the multi-site skin conductance signal characteristics of the target individual; inputs the multi-site skin conductance signal characteristics of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; wherein the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences under preset scenarios. It can be seen that the present invention collects the multi-site skin conductance signals of the target individual and inputs them into a pre-established emotion portrait prediction model to achieve accurate characterization of the emotion portrait, and this dynamic characterization method is characterized by high efficiency, convenience and low cost.

[0046] Based on the above embodiment, in this embodiment, the acquisition module 301 is specifically used for: The multi-site skin electrical signals of the target individual are obtained, the skin electrical signals are processed through a sliding window, and the multi-site skin electrical signal features in a short-time window corresponding to each time point are calculated.

[0047] Based on the above embodiment, in this embodiment, the device further includes a building module, which is specifically used for: Before inputting the multi-site skin electrical signal data of the target individual into the pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual, scene video information of each of the multiple individuals is obtained under a preset scene; wherein the scene video information includes: facial expressions and body movements of the corresponding individual; A multidimensional emotion label is obtained by dynamically labeling each individual among the multiple individuals based on the corresponding scene video information and the individual's own emotion experience, so as to construct the multidimensional emotion label data set.

[0048] Based on the above embodiment, in this embodiment, the device further includes a training module, which is specifically used for: Acquire historical multi-site skin electrical signals of the multiple individuals, process the historical multi-site skin electrical signals through a sliding window, and calculate the historical multi-site skin electrical signal features within a short-time window corresponding to each time point; Time-matching the historical multi-site skin electrical signal features in the short-term window corresponding to each time point with the multi-dimensional emotion label data set to obtain a multi-dimensional emotion label feature vector set; The multi-label neural network model is trained based on the multi-dimensional emotion label feature vector set to obtain the emotion portrait prediction model.

[0049] Based on the above embodiment, in this embodiment, the training module is specifically used for: Acquiring historical multi-site electrical skin signals of the multiple individuals; For each short-term window of each site, extracting the skin electrical response and skin conductance level in the historical multi-site skin electrical signal based on a continuous decomposition analysis technique; The skin conductance response is used to reflect the rapid sympathetic nerve activity caused by emotional events or stimuli, and the skin conductance level is used to reflect the overall arousal level.

[0050] Based on the above embodiment, in this embodiment, the device further includes an adjustment module, which is specifically used to: The step of inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual, and then obtaining the emotion state feedback information of the target individual; The multi-dimensional emotional portrait prediction result of the target individual is adjusted based on the emotional state feedback information of the target individual to obtain a target multi-dimensional emotional portrait result of the target individual.

[0051] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may be a robot or other electronic device, and the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 630 to execute the emotion portrait tracking method based on the multi-site electrical skin technology, including: Acquire multi-site skin electrical signal characteristics of the target individual; Inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0052] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0053] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the emotion portrait tracking method based on multi-site skin conductance technology provided by the above methods, including: Acquire multi-site skin electrical signal characteristics of the target individual; Inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0054] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the emotion portrait tracking method based on multi-site electrical skin technology provided by the above methods, including: Acquire multi-site skin electrical signal characteristics of the target individual; Inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

[0055] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0056] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for tracking emotional portraits based on multi-site electrical skin technology, characterized in that: include: Acquire multi-site skin electrical signal characteristics of the target individual; Inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

2. The emotion portrait tracking method based on multi-site electrical skin technology according to claim 1 is characterized in that: The step of obtaining the multi-site skin electrical signal characteristics of the target individual includes: The multi-site skin electrical signals of the target individual are obtained, the skin electrical signals are processed through a sliding window, and the multi-site skin electrical signal features in a short-time window corresponding to each time point are calculated.

3. The emotion portrait tracking method based on multi-site electrical skin technology according to claim 1 is characterized in that: Before inputting the multi-site skin electrical signal data of the target individual into the pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual, the method further includes: Under a preset scene, scene video information of each of the multiple individuals is obtained; wherein the scene video information includes: facial expressions and body movements of the corresponding individual; A multidimensional emotion label is obtained by dynamically labeling each individual among the multiple individuals based on the corresponding scene video information and the individual's own emotion experience, so as to construct the multidimensional emotion label data set.

4. The emotion portrait tracking method based on multi-site electrical skin technology according to claim 3 is characterized in that: The training process of the emotion portrait prediction model includes: Acquire historical multi-site skin electrical signals of the multiple individuals, process the historical multi-site skin electrical signals through a sliding window, and calculate the historical multi-site skin electrical signal features within a short-time window corresponding to each time point; Time-matching the historical multi-site skin electrical signal features within the short-term window corresponding to each time point with the multi-dimensional emotion label data set to obtain a multi-dimensional emotion label feature vector set; The multi-label neural network model is trained based on the multi-dimensional emotion label feature vector set to obtain the emotion portrait prediction model.

5. The emotion portrait tracking method based on multi-site electrical skin technology according to claim 4 is characterized in that: The step of obtaining the historical multi-site skin electrical signals of the multiple individuals, processing the historical multi-site skin electrical signals through a sliding window, and calculating the historical multi-site skin electrical signal features within a short-time window corresponding to each time point includes: Acquiring historical multi-site skin electrical signals of the multiple individuals; For each short-term window of each site, extracting the skin electrical response and skin conductance level in the historical multi-site skin electrical signal based on a continuous decomposition analysis technique; The galvanic skin response is used to reflect the rapid sympathetic nerve activity caused by emotional events or stimuli, and the skin conductance level is used to reflect the overall arousal level.

6. The emotion portrait tracking method based on multi-site electrical skin technology according to claim 1 is characterized in that: After inputting the multi-site skin electrical signal features of the target individual into the pre-trained emotion portrait prediction model to obtain the multi-dimensional emotion portrait prediction result of the target individual, the method further includes: Obtaining emotional state feedback information of the target individual; The multi-dimensional emotional portrait prediction result of the target individual is adjusted based on the emotional state feedback information of the target individual to obtain a target multi-dimensional emotional portrait result of the target individual.

7. An emotion portrait tracking device based on multi-site electrical skin technology, characterized in that: include: An acquisition module, used to acquire multi-site skin electrical signal characteristics of a target individual; A prediction module, used for inputting the multi-site skin electrical signal features of the target individual into a pre-trained emotion portrait prediction model to obtain a multi-dimensional emotion portrait prediction result of the target individual; Among them, the emotion portrait prediction model is obtained by training a multi-label neural network model based on a pre-established multi-dimensional emotion label data set; the multi-dimensional emotion label data set includes: multi-dimensional emotion labels obtained by dynamically labeling multiple individuals based on their own emotional experiences in a preset scenario.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the emotion portrait tracking method based on multi-site electrical skin technology as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for tracking emotion portraits based on multi-site electrical skin technology as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for tracking emotion portraits based on multi-site electrical skin technology as described in any one of claims 1 to 6 is implemented.

Images