Simulation system for preventing postpartum depression

Abstract

The invention discloses a simulation system for preventing postpartum depression, which belongs to the technical field of prenatal prevention and comprises a video data acquisition module, a video information extraction module, a character relationship analysis module, a character postpartum depression level analysis module, a VR animation production module, a classification storage module and a VR simulation module. According to the method, the historical video data are collected to construct the VR animation, the VR animation comprises various different scenes triggering postpartum depression, and a user can play a more comprehensive and better prevention effect through VR simulation of different scenes triggering postpartum depression.

Description

Technical Field

[0001] The present invention belongs to the technical field of prenatal prevention, and particularly relates to a simulation system for preventing postpartum depression. Background Art

[0002] Postpartum depression refers to an emotional disorder experienced by women after childbirth, usually manifested as symptoms such as persistent sadness, anxiety, loss of interest, irritability, exhaustion, and loss of interest in or inability to participate in daily activities.

[0003] Currently, the main cause of postpartum depression is the lack of understanding of the postpartum life situation by pregnant women before childbirth.

[0004] Chinese Patent Application No. 201910331353.2 discloses a simulation system for infant care scenarios to prevent postpartum depression, including a rubber doll in the shape of a simulated infant with a built-in chip, a voice module, a motion sensor module, and a magnetic sensor module, a liquid crystal display module, a reed switch, a baby bottle with a magnet at the nipple head, and a diaper with a magnet encapsulated therein. By simulating the care scenarios of pregnant and lying-in women for infants through the above modules, the understanding degree is increased to prevent the occurrence of postpartum depression.

[0005] The above technology has the following problems: The above system can only simulate the care scenarios of pregnant and lying-in women for infants, but postpartum depression is not only caused by taking care of infants. In some cases, it can also be caused by the interaction between family members. Therefore, the simulation is not comprehensive and the prevention effect still needs to be improved.

[0006] In view of this, a simulation system for preventing postpartum depression is designed to solve the above problems. Summary of the Invention

[0007] To solve the problems raised in the above background art, the present invention provides a simulation system for preventing postpartum depression, which has the characteristics of being able to achieve a more comprehensive and better prevention effect.

[0008] To achieve the above object, the present invention provides the following technical solution: A simulation system for preventing postpartum depression, comprising:

[0009] A video data acquisition module, which acquires video data showing postpartum depression manifestations;

[0010] A video information extraction module, which extracts video information based on the acquired video data. The video information includes an image of a person showing postpartum depression manifestations, an associated person image that causes the person to have postpartum depression manifestations, and an associated event that causes the person to have postpartum depression manifestations.

[0011] A character relationship analysis module determines the relationships between each other based on the extracted character images with postpartum depression manifestations and the associated character images that cause the character to have postpartum depression manifestations. The relationships include mother-daughter, mother-son, husband-wife, mother-in-law-daughter-in-law, father-in-law-daughter-in-law, mother-daughter, and father-daughter.

[0012] A postpartum depression level analysis module for characters determines the level of postpartum depression based on the extracted character images with postpartum depression manifestations.

[0013] A VR animation production module produces VR animations based on the extracted character images with postpartum depression manifestations, the associated character images that cause the character to have postpartum depression manifestations, the determined relationships between the character images, and the associated events that cause the character to have postpartum depression manifestations.

[0014] A classification and storage module classifies the produced VR animations once based on the determined relationships between the character images. The VR animations after the first classification are classified a second time based on the determined levels of postpartum depression. The VR animations between different levels of postpartum depression are sorted in ascending order according to the level and then stored.

[0015] A VR simulation module allows a user to enter a VR animation for simulation by wearing a VR device. Before the simulation, in the VR animations classified once, the user can freely choose to enter the VR animations with different character image relationships for simulation. In the VR animations classified a second time, the user can choose to enter the VR animations with a lower level of postpartum depression for simulation before choosing to enter the VR animations with a higher level of postpartum depression for simulation. During the simulation, the body coordinates of the user's actions are collected in real time, converted into VR animation coordinates, and the VR animation adjusts the data based on the VR animation coordinates of the user's actions to update the VR animation. At the same time, guiding language is triggered based on the updated VR animation to guide the user to perform correct interaction actions.

[0016] Furthermore, the specific steps for extracting the character images with postpartum depression manifestations in the video information extraction module include:

[0017] Decompose the collected video data into video frames.

[0018] Analyze each video frame one by one based on a preset postpartum depression extraction model. First, extract the video frames with postpartum depression manifestations, and then segment and extract the character images with postpartum depression manifestations in the video frames.

[0019] Furthermore, the specific steps for extracting the associated character images that cause the character to have postpartum depression manifestations in the video information extraction module include:

[0020] Decode the collected video data to obtain the original audio data.

[0021] Extract the audio data with the same timestamp as the video frames showing postpartum depression from the original audio data;

[0022] Extract the MFCC features of the audio data;

[0023] Convert the MFCC features of the audio data into text vectors based on a preset acoustic model;

[0024] Understand the context of the text vectors based on a preset large language model and output the associated audio data that prompts the person to show postpartum depression;

[0025] Predict the word-by-word lip images of the associated audio data that prompts the person to show postpartum depression based on a preset lip prediction model;

[0026] Decompose the collected video data into video frames;

[0027] Extract the lip images of the person in the video frames with the same timestamp as the associated audio data that prompts the person to show postpartum depression based on a preset lip extraction model;

[0028] Compare the similarity between the predicted lip images and the extracted lip images of the person to determine the associated lip images of the person that prompt the person to show postpartum depression;

[0029] Determine the associated person who prompts the person to show postpartum depression based on the associated lip images of the person, and segment and extract the associated person images of the person that prompt the person to show postpartum depression.

[0030] Further, the specific steps for extracting the associated events that prompt the person to show postpartum depression in the video information extraction module include:

[0031] Decode the collected video data to obtain the original audio data;

[0032] Extract the audio data with the same timestamp as the video frames showing postpartum depression from the original audio data;

[0033] Extract the MFCC features of the audio data;

[0034] Convert the MFCC features of the audio data into text vectors based on a preset acoustic model;

[0035] Understand the context of the text vectors based on a preset large language model and output the associated events that prompt the person to show postpartum depression.

[0036] Further, the specific steps of the person relationship analysis module include:

[0037] Determine the identity of the person image based on the comparison and extraction of the person image with postpartum depression manifestations from the face database and the associated person image that causes the person to have postpartum depression manifestations.

[0038] Based on the comparison of the identity of the person image in the identity blood relationship database, determine the relationship between the person images.

[0039] Furthermore, the specific steps of the person postpartum depression level analysis module include:

[0040] Based on the preset postpartum depression level judgment model, determine the postpartum depression level of the extracted person image with postpartum depression manifestations.

[0041] Compared with the prior art, the beneficial effects of the present invention are:

[0042] 1. The present invention collects historical video data to construct a VR animation, which contains various scenarios that trigger postpartum depression. Through the VR simulation of different scenarios that trigger postpartum depression, users can achieve a more comprehensive and better preventive effect.

[0043] 2. During the VR animation simulation process of the present invention, the body coordinates of the user's actions are collected in real time, converted into VR animation coordinates, and the VR animation coordinates of the user interaction are corrected in real time based on the VR animation coordinates, enabling users to produce a better simulation effect and a better preventive effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is the system framework diagram of the present invention;

[0045] In the figure: 1. Video data acquisition module; 2. Video information extraction module; 3. Person relationship analysis module; 4. Person postpartum depression level analysis module; 5. VR animation production module; 6. Classification and storage module; 7. VR simulation module. DETAILED DESCRIPTION OF THE INVENTION

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] The present invention provides the following technical solutions: A simulation system for preventing postpartum depression, including:

[0048] A video data acquisition module 1, which acquires video data with postpartum depression manifestations;

[0049] The video information extraction module 2 extracts video information based on the collected video data. The video information includes the images of the person with postpartum depression, the associated person images that cause the person to have postpartum depression, and the associated events that cause the person to have postpartum depression;

[0050] The person relationship analysis module 3 determines the relationship between each other based on the extracted images of the person with postpartum depression and the associated person images that cause the person to have postpartum depression. The relationships include mother-daughter, mother-son, husband-wife, mother-in-law / daughter-in-law, father-in-law / daughter-in-law, mother-daughter, and father-daughter;

[0051] The person postpartum depression level analysis module 4 determines the postpartum depression level based on the extracted images of the person with postpartum depression;

[0052] The VR animation production module 5 produces VR animations based on the extracted images of the person with postpartum depression, the associated person images that cause the person to have postpartum depression, the judged relationship between the person images, and the associated events that cause the person to have postpartum depression;

[0053] The classification and storage module 6 classifies the produced VR animations once based on the judged relationship between the person images. The VR animations after the first classification are classified twice based on the judged postpartum depression level. The VR animations between different postpartum depression levels are sorted in ascending order of level and saved;

[0054] The VR simulation module 7 allows the user to wear a VR device to enter the VR animation for simulation. Before the simulation, in the VR animations of the first classification, the user can freely choose to enter the VR animations with different person image relationships for simulation. In the VR animations of the second classification, the user can choose to enter the VR animations with a lower postpartum depression level for simulation before choosing to enter the VR animations with a higher postpartum depression level for simulation. During the simulation process, the body coordinates of the user's actions are collected in real time, and the body coordinates of the user's actions are converted into VR animation coordinates. The VR animation updates the data based on the VR animation coordinates of the user's actions and triggers guiding language based on the updated VR animation to guide the user's correct interaction actions.

[0055] The guiding voice is preset to be associated with different VR animation data, and when the VR animation is updated to trigger the VR animation data, the guiding voice is triggered.

[0056] Specifically, the specific steps for extracting the images of the person with postpartum depression in the video information extraction module 2 include:

[0057] Decompose the collected video data into video frames;

[0058] Analyze video frames one by one based on a preset postpartum depression extraction model. First, extract the video frames with postpartum depression manifestations, and then segment and extract the images of the figures with postpartum depression manifestations in the video frames.

[0059] The preset postpartum depression extraction model is a convolutional neural network model. The convolutional neural network model is trained with historical postpartum depression surface images so that the convolutional neural network model learns the image features of postpartum depression manifestations to achieve the function of the model for extracting postpartum depression images.

[0060] Specifically, the specific steps for extracting the associated figure images that prompt the figure to have postpartum depression manifestations in the video information extraction module 2 include:

[0061] Decode the collected video data to obtain the original audio data;

[0062] Extract the audio data with the same timestamp as the video frames with postpartum depression manifestations from the original audio data;

[0063] Extract the MFCC features of the audio data;

[0064] Based on a preset acoustic model, convert the MFCC features of the audio data into text vectors;

[0065] Train the acoustic model with historical audio data so that the acoustic model learns the conversion features to achieve the function of the model for converting text vectors;

[0066] Based on a preset large language model, understand the context of the text vectors and output the associated audio data that prompts the figure to have postpartum depression manifestations;

[0067] Train the large language model with historical text vectors so that the large language model learns the text vector features to achieve the function of the model for understanding text vectors;

[0068] Based on a preset lip prediction model, predict the word-by-word lip images of the associated audio data that prompts the figure to have postpartum depression manifestations;

[0069] The lip prediction model is the LLaVA model. The LLaVA model retrieves the lip images that match the text information from the text information in the associated audio data that prompts the figure to have postpartum depression manifestations. Among them, the lip images of the text information are preset;

[0070] Decompose the collected video data into video frames;

[0071] Based on a preset lip extraction model, extract the lip images of the figures in the video frames with the same timestamp as the associated audio data that prompts the figure to have postpartum depression manifestations;

[0072] The lip extraction model is a convolutional neural network model. The convolutional neural network model is trained with historical figure images so that the model can learn the lip image features, thereby realizing the function of extracting lip images from the input images.

[0073] Compare the similarity between the predicted lip image and the extracted lip image of the person, and determine the lip image of the associated person that contributes to the person's postpartum depression symptoms.

[0074] The similarity between images is based on cosine similarity. Two images are converted into feature vectors, and the similarity between the two image feature vectors is calculated to measure the similarity between the two images. The formula is:

[0075]

[0076] In the formula: A·B represents the dot product of vectors A and B, and ||A||||B|| represents the norms of vectors A and B.

[0077] Based on the lip image of the associated person, determine the associated person who contributes to the person's postpartum depression symptoms, and segment and extract the image of the associated person who contributes to the person's postpartum depression symptoms.

[0078] Specifically, the specific steps for extracting the associated events that contribute to the person's postpartum depression symptoms in the video information extraction module 2 are as follows:

[0079] Decode the collected video data to obtain the original audio data.

[0080] Extract the audio data with the same timestamp as the video frame with postpartum depression symptoms from the original audio data.

[0081] Extract the MFCC features of the audio data.

[0082] Based on a preset acoustic model, convert the MFCC features of the audio data into text vectors.

[0083] Based on a preset large language model, understand the context of the text vector and output the associated events that contribute to the person's postpartum depression symptoms.

[0084] Specifically, the specific steps of the person relationship analysis module 3 are as follows:

[0085] Based on the face database, compare the person image with postpartum depression symptoms and the lip image of the associated person that contributes to the person's postpartum depression symptoms, and determine the identity of the person image.

[0086] Based on the identity and blood relationship database, compare the identities of the person images and determine the relationship between the person images.

[0087] Specifically, the specific steps of the postpartum depression level analysis module 4 for characters include:

[0088] Determine the postpartum depression level of the extracted character images with postpartum depression manifestations based on a preset postpartum depression level judgment model;

[0089] The postpartum depression level judgment model is a convolutional neural network model. The convolutional neural network model is trained with historical character images so that the convolutional neural network model learns the image features of postpartum depression at different levels to achieve the function of the model for judging the postpartum depression level.

[0090] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A simulation system for preventing postpartum depression, characterized in that, Including: A video data acquisition module (1) for acquiring video data showing postpartum depression; A video information extraction module (2) for extracting video information based on the acquired video data. The video information includes images of people showing postpartum depression, associated person images that cause the person to show postpartum depression, and associated events that cause the person to show postpartum depression; A person relationship analysis module (3) for determining the relationship between each other based on the extracted images of people showing postpartum depression and the associated person images that cause the person to show postpartum depression. The relationships include mother-daughter, mother-son, husband-wife, mother-in-law-daughter-in-law, father-in-law-daughter-in-law, mother-daughter, and father-daughter; A person postpartum depression level analysis module (4) for determining the level of postpartum depression based on the extracted images of people showing postpartum depression; A VR animation production module (5) for producing VR animations based on the extracted images of people showing postpartum depression, the associated person images that cause the person to show postpartum depression, the determined relationship between the person images, and the associated events that cause the person to show postpartum depression; A classification and storage module (6) for classifying the produced VR animations once based on the determined relationship between the person images, and then classifying the VR animations after the first classification based on the determined level of postpartum depression. The VR animations between different levels of postpartum depression are sorted in ascending order of level and stored; A VR simulation module (7) for a user to wear a VR device to enter the VR animation for simulation. Before simulation, in the VR animations classified once, the user can randomly select to enter the VR animations with different person image relationships for simulation. In the VR animations classified twice, the user can select to enter the VR animations with a lower level of postpartum depression for simulation first, and then can select to enter the VR animations with a higher level of postpartum depression for simulation. During the simulation process, the body coordinates of the user's actions are collected in real time, and the body coordinates of the user's actions are converted into VR animation coordinates. The VR animation adjusts the data based on the VR animation coordinates of the user's actions to update the VR animation, and at the same time triggers guiding language based on the updated VR animation to guide the user's correct interaction actions.

2. The simulation system for preventing postpartum depression according to claim 1, characterized in that: The specific steps for extracting the images of people showing postpartum depression in the video information extraction module (2) include: Decomposing the acquired video data into video frames; Analyzing each video frame one by one based on a preset postpartum depression extraction model, first extracting the video frames showing postpartum depression, and then segmenting and extracting the images of people showing postpartum depression in the video frames.

3. The simulation system for preventing postpartum depression according to claim 2, wherein: The specific steps for extracting the associated person images that cause the person to show postpartum depression in the video information extraction module (2) include: Decoding the acquired video data to obtain the original audio data; Extracting the audio data with the same timestamp as the video frames showing postpartum depression from the original audio data; Extracting the MFCC features of the audio data; Converting the MFCC features of the audio data into text vectors based on a preset acoustic model; Understanding the context of the text vectors based on a preset large language model and outputting the associated audio data that causes the person to show postpartum depression. Predict the word-by-word lip image of the associated audio data that prompts the person to have postpartum depression manifestations based on a preset lip prediction model; Decompose the collected video data into video frames; Extract the lip image of the person in the video frame with the same timestamp as the associated audio data that prompts the person to have postpartum depression manifestations based on a preset lip extraction model; Compare the similarity between the predicted lip image and the extracted lip image of the person, and judge the lip image of the associated person that prompts the person to have postpartum depression manifestations; Determine the associated person who prompts the person to have postpartum depression manifestations based on the lip image of the associated person, and segment and extract the image of the associated person who prompts the person to have postpartum depression manifestations.

4. A simulation system for preventing postpartum depression according to claim 3, characterized in that: The specific steps for extracting the associated event that prompts the person to have postpartum depression manifestations in the video information extraction module (2) include: Decode the collected video data to obtain the original audio data; Extract the audio data with the same timestamp as the video frame with postpartum depression manifestations from the original audio data; Extract the MFCC features of the audio data; Convert the MFCC features of the audio data into text vectors based on a preset acoustic model; Understand the context of the text vector based on a preset large language model and output the associated event that prompts the person to have postpartum depression manifestations.

5. The simulation system for preventing postpartum depression according to claim 4, characterized in that: The specific steps of the person relationship analysis module (3) include: Compare the extracted person image with postpartum depression manifestations and the lip image of the associated person who prompts the person to have postpartum depression manifestations based on the face database to determine the identity of the person image; Compare the identity of the person image based on the identity blood relationship database to determine the relationship between the person images.

6. The simulation system for preventing postpartum depression according to claim 5, characterized in that: The specific steps of the person postpartum depression level analysis module (4) include: Determine the postpartum depression level of the extracted person image with postpartum depression manifestations based on a preset postpartum depression level judgment model.

Images