Early recognition and early warning system for acute ischemic stroke symptoms

Abstract

The invention discloses an early recognition and early warning system for acute ischemic stroke symptoms, and relates to the technical field of health detection. Comprising an image acquisition module, a voice acquisition module, a data processing module, a display module and an alarm module, the voice acquisition module is used for acquiring user audio signals; the data processing module is used for processing the collected signals to generate corresponding face judgment signals, limb judgment signals and audio judgment signals; the display module is used for displaying sentences needing to be read by the user; and the alarm module is used for judging whether the user has an acute ischemic stroke early symptom or not according to the face judgment signal, the limb judgment signal and the audio judgment signal, and if so, dialing 120 and a contact person through the mobile phone. The method can significantly improve the early warning and prevention efficiency of cerebral apoplexy, is expected to play a greater role in the field of cerebral apoplexy prevention and treatment, and improves the prognosis and life quality of patients.

Description

Technical Field

[0001] The present invention relates to the technical field of health detection, and particularly to an early recognition and warning system for acute ischemic stroke symptoms. Background Art

[0002] Stroke is an acute cerebrovascular disease, a group of diseases caused by sudden rupture of blood vessels in the brain or blockage of blood vessels leading to inability of blood to flow into the brain, resulting in brain tissue damage, including ischemic and hemorrhagic strokes. In severe cases, it can cause permanent nerve damage. If acute stroke is not diagnosed and treated in time, it can cause serious complications and even death, and has become an important public health problem seriously endangering the health of Chinese people, with the five characteristics of high incidence, high disability rate, high mortality rate, high recurrence rate, and high economic burden. Currently, commonly used pre-hospital stroke screening tools include the Cincinnati Prehospital Stroke Scale, Facial Arm Speech Test, Los Angeles Prehospital Stroke Scale, Melbourne Ambulance Stroke Screen, and Emergency Department Stroke Identification Scale. These tools usually require users to have a certain medical professional background, and ordinary people cannot use these scales to evaluate the incidence of stroke in the first place. China has proposed a rapid stroke recognition tool "Stroke 120" suitable for the Chinese population. However, when a solitary elderly person has a stroke or there is no one else beside them when the stroke occurs, the evaluation tool cannot be used either, resulting in the inability to provide treatment in the first place, and there are still deficiencies in achieving or reaching effective prediction, early intervention, and combination of prevention and treatment of stroke.

[0003] Therefore, how to provide an early recognition and warning system for acute ischemic stroke symptoms to solve the difficulties existing in the prior art is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides an early recognition and warning system for acute ischemic stroke symptoms, which combines facial information, limb information, and audio information for comprehensive recognition, effectively improving the accuracy of stroke risk recognition.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] An early recognition and warning system for acute ischemic stroke symptoms, including an image acquisition module, a voice acquisition module, a data processing module, a display module, and an alarm module.

[0007] The image acquisition module is used to acquire the user's facial image and limb image.

[0008] The voice acquisition module is connected to the second input end of the data processing module and is used to acquire the user's audio signal.

[0009] A data processing module, with its first input terminal connected to the output terminal of the image acquisition module, is used to process the user's facial image, limb image, and audio signal to generate corresponding facial judgment signals, limb judgment signals, and audio judgment signals.

[0010] A display module, connected to the first output terminal of the data processing unit, is used to display the sentences that the user needs to read aloud.

[0011] An alarm module, connected to the second output terminal of the data processing unit, is used to determine whether the user has early symptoms of acute ischemic stroke based on the facial judgment signal, limb judgment signal, and audio judgment signal. If so, it dials 120 through the mobile phone and notifies the contacts.

[0012] Optionally, collecting the user's facial image includes: using a camera to shoot a target video containing the user's face, reading the target video frame by frame, and extracting the user's facial image.

[0013] Optionally, collecting the user's limb image includes:

[0014] Obtaining a video file containing the user's limbs;

[0015] Obtaining clear images from randomly selected key frames of the video file and identifying the face region in the images;

[0016] Determining the user's limb position based on the face region to obtain the user's limb image.

[0017] Optionally, the data processing module includes a facial recognition unit, a limb recognition unit, and an audio detection unit;

[0018] The facial recognition unit is used to recognize the facial state and the state of the corners of the mouth, judge whether the face is symmetrical and whether the two corners of the mouth are on the same horizontal line, and obtain the facial judgment signal;

[0019] The limb recognition unit is used to recognize the limb state and judge whether the two arms are lifted parallel to each other to obtain the limb judgment signal;

[0020] The audio detection unit is used to detect whether the received audio information is consistent with the information of the display module and detect the clarity of the audio signal to obtain the audio judgment signal.

[0021] Optionally, the facial recognition unit includes:

[0022] Obtaining the user's facial image and the corresponding preset feature points;

[0023] Collecting all the preset feature points of the facial images as a feature point sequence and calculating the mean value of the feature point sequence to obtain a facial template;

[0024] Mapping the facial template to the image to be detected and comparing the mouth feature points to generate the facial judgment signal.

[0025] Optionally, the limb recognition unit includes:

[0026] Obtain the user's limb image and set the motion key points;

[0027] Perform action recognition on the motion key point detection data to obtain limb motion action recognition data;

[0028] Perform action classification mapping according to the limb motion action recognition data to obtain action classification mapping data;

[0029] Map the action classification mapping data to the image to be detected, and compare the limb motion action data to form a limb judgment signal.

[0030] Optionally, the audio detection unit includes:

[0031] Obtain the user's audio signal, analyze the audio signal to obtain the speech fluency index and the speech clarity index as the first quality index,

[0032] Input the audio signal into the speech content recognition model, output the predicted text of the audio signal, and compare and analyze it with the text displayed by the display module to obtain the second quality index;

[0033] Generate an audio judgment signal based on the first quality index and the second quality index.

[0034] As can be seen from the above technical solutions, compared with the prior art, the present invention provides an early recognition and warning system for acute ischemic stroke symptoms, and has the following beneficial effects: 1) The present invention can output a face area with a stable size under the input of pictures with different clarity and resolution, improving the compatibility and accuracy of mouth recognition; 2) By performing pose estimation and key point detection on the limb motion image data, the present invention can accurately obtain the user's motion state and key point information, automatically recognize the user's motion actions, and achieve precise detection of limb motion; 3) The present invention can automatically process and analyze speech data, comprehensively evaluate the speech from the accuracy and integrity of pronunciation, improving the efficiency and accuracy of speech evaluation; 4) The present invention can significantly improve the early warning and prevention efficiency of stroke, is expected to play a greater role in the field of stroke prevention and treatment, and improve the prognosis and quality of life of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0036] Figure 1 This is a block diagram of an early recognition and warning system for acute ischemic stroke symptoms disclosed by the present invention. Specific embodiments

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] Referring to Figure 1 As shown, the present invention discloses an early recognition and warning system for acute ischemic stroke symptoms, including an image acquisition module, a voice acquisition module, a data processing module, a display module, and an alarm module.

[0039] The image acquisition module is used to acquire the user's facial image and limb image.

[0040] The voice acquisition module is connected to the second input end of the data processing module and is used to acquire the user's audio signal.

[0041] The data processing module, with its first input end connected to the output end of the image acquisition module, is used to process the user's facial image, limb image, and audio signal to generate corresponding facial judgment signals, limb judgment signals, and audio judgment signals.

[0042] The display module is connected to the first output end of the data processing unit and is used to display the sentences that the user needs to read aloud.

[0043] The alarm module is connected to the second output end of the data processing unit and is used to judge whether the user has early symptoms of acute ischemic stroke according to the facial judgment signal, limb judgment signal, and audio judgment signal. If so, it will call 120 through the mobile phone and notify the contact person.

[0044] Furthermore, collecting the user's facial image includes: using a camera to shoot a target video containing the user's face, reading the target video frame by frame, and extracting the user's facial image.

[0045] Furthermore, collecting the user's limb image includes:

[0046] Obtaining a video file containing the user's limbs;

[0047] Obtaining a clear image from the randomly selected key frames of the video file and identifying the face area in the image;

[0048] Determining the user's limb position based on the face area to obtain the user's limb image.

[0049] Specifically, identifying the face region includes: using a trained deep learning model based on color information and depth information to detect whether there is a face in the main image of the scene. When the deep learning model is given a training set, the data in the training set includes the color information and depth information of the face. Therefore, the trained deep learning model can infer whether there is a face region in the current scene based on the color information and depth information of the current scene. Since the acquisition of the depth information of the face region is not easily affected by environmental factors such as illumination, the accuracy of face detection can be improved, and then the portrait region is determined according to the face region and the depth information of the face region.

[0050] Further, the data processing module includes a face recognition unit, a limb recognition unit, and an audio detection unit;

[0051] The face recognition unit is used to recognize the facial state and the state of the corners of the mouth, judge whether the face is symmetrical and whether the two corners of the mouth are on the same horizontal line, and obtain a facial judgment signal;

[0052] The limb recognition unit is used to recognize the limb state, judge whether the two arms are lifted parallel, and obtain a limb judgment signal;

[0053] The audio detection unit is used to detect whether the received audio information is consistent with the information of the display module, and detect the clarity of the audio signal, and obtain an audio judgment signal.

[0054] Further, the face recognition unit includes:

[0055] Obtain the user's facial image and the corresponding preset feature points;

[0056] Collect all the preset feature points of the facial images as a feature point sequence, and calculate the mean value of the feature point sequence to obtain a facial template;

[0057] Map the facial template to the image to be detected, and compare the feature points of the mouth to generate a facial judgment signal.

[0058] Specifically, when a human face is detected in each frame, a face key-point detection algorithm can be used to obtain facial feature points. Facial feature points usually include special positions such as eyes, nose, mouth, eyebrows, etc. After obtaining the facial feature points of each frame, they are combined into a sequence. Usually, techniques such as linear interpolation can be used to fill in the missing facial feature points. The average value of each feature point in the facial feature point sequence is calculated respectively to obtain the average value of each feature point, thereby obtaining a facial template. A mapping is established between the facial template and the image to be detected in the existing video to obtain a mapping matrix. The mapping matrix is used to map each of the to-be-processed video frames of the existing video frame by frame to obtain a sequence of video frames of the target size; the mapping matrix is used to perform mapping on each of the to-be-processed video frames for the facial feature points to obtain corresponding target feature points of the target size; the sequence of video frames and the target feature points are used as the alignment image information, and the mouth region is identified according to the feature point sequence to determine whether the corner feature points of the mouth are skewed.

[0059] Further, the limb recognition unit includes:

[0060] Obtain the user's limb image and set motion key points;

[0061] Perform action recognition on the motion key point detection data to obtain limb motion action recognition data;

[0062] Perform action classification mapping according to the limb motion action recognition data to obtain action classification mapping data;

[0063] Map the action classification mapping data to the image to be detected, and compare the limb motion action data to form a limb judgment signal.

[0064] Specifically, according to the preset first threshold angle data, the first limb motion image data is deformed, and then an edge detection algorithm or a feature point detection algorithm is used to extract the contour or key points of the limb, and the angle of the limb is calculated based on this information to obtain the first limb angle detection data. Similarly, according to the preset second threshold angle data, the angle of the second limb is detected to obtain the angle detection data of the second limb;

[0065] According to the first limb angle detection data and the second limb angle detection data, the first limb motion image data and the second limb motion image data are corrected to obtain the first limb motion image correction data and the second limb motion image correction data, and feature points are extracted from them to obtain the first limb motion image feature point data and the second limb motion image feature point data;

[0066] Perform feature matching on the feature point data of the first limb movement image and the feature point data of the second limb movement image to obtain the feature matching data of the limb movement image and perform image fusion on it to obtain the limb movement action recognition data.

[0067] Construct an action recognition model, input the limb movement action recognition data, and output the action classification result. According to the output result of the model, map each action to the corresponding category to obtain the action classification mapping data. Determine the start and end times of the action based on the key point detection data, and perform temporal division on the action. Use the time window-based method to divide the key point data into several segments in time, and each segment represents an action stage. Determine the specific action category of each action stage according to the action classification mapping data. For example, let the action of raising both arms parallel be divided into 2 stages. Stage 1 is that both arms hang naturally, and stage 2 is that both arms are raised. Record the time required from stage 1 to stage 2, whether the action in stage 2 is standard, and the duration of stage 2, and correspondingly generate a limb judgment signal.

[0068] Further, the audio detection unit includes:

[0069] Obtain the user audio signal, analyze the audio signal to obtain the speech fluency index and the speech clarity index as the first quality index.

[0070] Input the audio signal into the speech content recognition model, output the predicted text of the audio signal, and obtain the second quality index through comparative analysis with the text displayed by the display module.

[0071] Generate an audio judgment signal based on the first quality index and the second quality index.

[0072] Specifically, obtain the total number of syllables produced and the corpus duration of the speech data to be evaluated according to the speech data to be evaluated, and record the ratio of the total number of syllables produced to the corpus duration as the speech rate feature parameter of the speech data to be evaluated; obtain the pronunciation duration of the speech data to be evaluated according to the speech data to be evaluated, where the pronunciation duration is the duration of continuous pronunciation in the speech data to be evaluated, and record the ratio of the total number of syllables produced to the pronunciation duration as the pronunciation speed feature parameter of the speech data to be evaluated; obtain the number of silent pauses of the speech data to be evaluated according to the speech data to be evaluated, and record the ratio of the number of silent pauses to the corpus duration as the silent pause rate feature parameter of the speech data to be evaluated; use the speech rate feature parameter, the pronunciation speed feature parameter, and the silent pause rate feature parameter to obtain the speech fluency index.

[0073] Use a trained speech recognition model (such as a GMM-HMM or DNN-based model) to identify the phonemes in the speech data. For each phoneme, set a GOP threshold to determine whether the pronunciation of the phoneme is correct, count the number of correctly pronounced phonemes, and obtain the speech clarity index by combining the total number of phonemes in the speech data to be evaluated. Then, obtain the first speech quality index by combining the speech fluency index.

[0074] Judge whether the sum of the first speech quality index and the second speech quality index exceeds a preset threshold, and then obtain an audio judgment signal.

[0075] Furthermore, set risk thresholds for the facial judgment signal, the limb judgment signal, and the audio judgment signal respectively, and give an alarm to the user when any one of the signals exceeds the threshold.

[0076] Furthermore, the alarm module is completed through the mobile phone. The user can set emergency contacts and emergency contact voices inside the mobile phone by himself / herself. The emergency contact voice includes the user's name, residential location, system detection results, etc. When it is judged that the user is in the early symptoms of acute ischemic stroke, the mobile phone automatically dials 120 and the emergency contacts and broadcasts the emergency contact voice.

[0077] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An early recognition and warning system for symptoms of acute ischemic stroke, characterized in that, It includes an image acquisition module, a voice acquisition module, a data processing module, a display module and an alarm module. The image acquisition module is used to acquire the user's facial image and limb image. The voice acquisition module is connected to the second input end of the data processing module and is used to acquire the user's audio signal. The data processing module, with its first input end connected to the output end of the image acquisition module, is used to process the user's facial image, limb image and audio signal to generate corresponding facial judgment signals, limb judgment signals and audio judgment signals. The display module is connected to the first output end of the data processing unit and is used to display the sentences that the user needs to read aloud. The alarm module is connected to the second output end of the data processing unit and is used to judge whether the user has early symptoms of acute ischemic stroke according to the facial judgment signal, limb judgment signal and audio judgment signal. If so, it will call 120 through the mobile phone and notify the contacts.

2. The early recognition and warning system for acute ischemic stroke symptoms according to claim 1, characterized in that Collecting the user's facial image includes: shooting a target video containing the user's face with a camera, reading the target video frame by frame, and extracting the user's facial image.

3. The early recognition and warning system for acute ischemic stroke symptoms according to claim 1, characterized in that Collecting the user's limb image includes: Obtaining a video file containing the user's limbs; Obtaining clear images from randomly selected key frames of the video file and identifying the face area in the images; Determining the user's limb position based on the face area to obtain the user's limb image.

4. The early recognition and warning system for acute ischemic stroke symptoms according to claim 1, characterized in that The data processing module includes a face recognition unit, a limb recognition unit and an audio detection unit; The face recognition unit is used to recognize the facial state and the state of the corners of the mouth, judge whether the face is symmetrical and whether the two corners of the mouth are on the same horizontal line, and obtain a facial judgment signal; The limb recognition unit is used to recognize the limb state and judge whether the two arms are lifted parallel, and obtain a limb judgment signal; The audio detection unit is used to detect whether the received audio information is consistent with the information of the display module and detect the clarity of the audio signal, and obtain an audio judgment signal.

5. The early recognition and warning system for acute ischemic stroke symptoms according to claim 4, characterized in that The face recognition unit includes: Obtaining the user's facial image and corresponding preset feature points; Collecting all the preset feature points of the facial image as a feature point sequence and calculating the mean value of the feature point sequence to obtain a facial template; Mapping the facial template to the image to be detected and comparing the mouth feature points to generate a facial judgment signal.

6. The early recognition and warning system for acute ischemic stroke symptoms according to claim 4, characterized in that The limb recognition unit includes: Obtaining the user's limb image and setting motion key points; Performing action recognition on the detection data of the motion key points to obtain limb motion action recognition data; Performing action classification mapping according to the limb motion action recognition data to obtain action classification mapping data; Map the action classification mapping data to the image to be detected, and compare the limb movement action data to form a limb judgment signal.

7. An early recognition and warning system for acute ischemic stroke symptoms according to claim 4, characterized in that The audio detection unit includes: Obtain the user's audio signal, analyze the audio signal to obtain the speech fluency index and the speech clarity index as the first quality index, Input the audio signal into the speech content recognition model, output the predicted text of the audio signal, and compare and analyze it with the text displayed by the display module to obtain the second quality index; Generate an audio judgment signal based on the first quality index and the second quality index.

Images