Risk assessment method, device and equipment for stroke patient and medium

Abstract

The invention relates to the technical field of health management, in particular to a risk assessment method and device for a stroke patient, equipment and a medium, and the method comprises the steps: obtaining a video image and audio data of the stroke patient, the video image comprises a face reaction image made by the stroke patient based on the first guide instruction and a limb reaction image made by the stroke patient based on the second guide instruction, and the audio data is a reply made by the stroke patient based on the third guide instruction; determining a facial paralysis recognition result and an eye response recognition result based on the facial response image; determining a limb motion recognition result based on the limb reaction image; based on the audio data, determining a language pronunciation recognition result; according to the method, the risk assessment level of the stroke patient is determined based on the facial paralysis recognition result, the eye response recognition result, the limb movement recognition result and the language composition recognition result, and the risk of the stroke patient can be accurately and quickly assessed without considering medical environment conditions.

Description

Technical Field

[0001] The present invention relates to the field of health management technology, and in particular to a risk assessment method, device, equipment and medium for stroke patients. Background Art

[0002] When a patient has symptoms of a stroke, a professional is needed to assess the risk of a stroke. A standard assessment form is usually used, such as the NIHHS assessment scale for stroke patients. During the assessment, the professional will ask the patient to make corresponding actions and reactions one by one according to the indicators in the scale. Then, the score will be given item by item based on the patient's response. Finally, the patient's stroke risk level is determined by summarizing the scores. Different people may have different judgments based on their experience. Especially in more remote village hospitals or clinics, when faced with risk assessments of stroke patients, there is often a lack of sufficient professionalism and standardized processes, which can easily lead to incorrect assessments of their risk level and miss the best time for treatment.

[0003] Therefore, how to accurately and quickly assess the risk of stroke patients is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the above problems, the present invention provides a method, device, equipment and medium for risk assessment of stroke patients that overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect, the present invention provides a method for risk assessment of stroke patients, comprising: Acquire video images and audio data of a stroke patient, wherein the video images include facial reaction images of the stroke patient based on the first guidance instruction and limb reaction images based on the second guidance instruction, and the audio data are responses of the stroke patient based on the third guidance instruction; Determining a facial paralysis recognition result and an eye reaction recognition result based on the facial reaction image; Determining limb movement recognition results based on the limb reaction image; Determining a language pronunciation recognition result based on the audio data; Based on the facial paralysis recognition results, eye reaction recognition results, limb movement recognition results, and language articulation recognition results, the risk assessment level of the stroke patient is determined.

[0006] Preferably, before determining the facial paralysis recognition result and the eye reaction recognition result based on the facial reaction image, the method further comprises: Using a Haar feature classifier to identify the video image and locate the facial area of ​​the stroke patient; Based on the face region, a facial reaction image is determined.

[0007] Preferably, determining the facial paralysis recognition result and the eye reaction recognition result based on the facial reaction image comprises: Based on the facial reaction image, marking the position of each facial feature in the facial region; Based on the position of each facial feature, determine the key feature points of each facial feature; Connect the key feature points of each facial feature to form a point line; Based on the floating changes of the point line, the facial paralysis recognition results and the eye reaction recognition results are determined.

[0008] Preferably, based on the floating change of the point line, determining the facial paralysis recognition result and the eye reaction recognition result includes: Determine the change of the target area based on the floating change of the point line; Based on the changes in the target area, a facial paralysis recognition result and an eye expression reaction recognition result are determined.

[0009] Preferably, determining the facial paralysis recognition result and the eye reaction recognition result based on the change of the target area includes: When identifying facial paralysis, determining a facial paralysis identification result based on a first change in the nasolabial groove and a second change in the lower face; When recognizing the eye reaction situation, the eye reaction recognition result is determined based on the concentration of the eyeballs and the reaction of the eyeballs to the stimulus.

[0010] Preferably, determining the limb movement recognition result based on the limb reaction image includes: By providing an image guide frame, the image guide frame is used as a control; Determine whether the target limb in the limb reaction image is lifted into the image guidance frame and maintained for a preset time, and obtain a determination result; Based on the judgment result, the limb movement recognition result is determined.

[0011] Preferably, determining a language pronunciation recognition result based on the audio data includes: The audio data is analyzed using a natural language processing model to determine the proportion of error-corrected data; Determining the accuracy of the response based on the audio data and the second guidance instruction; Based on the error correction data ratio and the answer accuracy rate, a language articulation recognition result is determined.

[0012] In a second aspect, the present invention further provides a risk assessment device for stroke patients, comprising: An acquisition module, used to acquire video images and audio data of a stroke patient, wherein the video images include facial reaction images made by the stroke patient based on the first guidance instruction and limb reaction images made based on the second guidance instruction, and the audio data are responses made by the stroke patient based on the third guidance instruction; A first determination module, used to determine a facial paralysis recognition result and an eye reaction recognition result based on the facial reaction image; A second determination module is used to determine a limb movement recognition result based on the limb reaction image; A third determination module, used to determine a language articulation recognition result based on the audio data; The fourth determination module is used to determine the risk assessment level of the stroke patient based on the facial paralysis recognition results, eye reaction recognition results, limb movement recognition results and language articulation recognition results.

[0013] In a third aspect, the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0014] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0015] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: The present invention provides a risk assessment method for stroke patients, comprising: acquiring video images and audio data of stroke patients, the video images comprising facial reaction images made by the stroke patients based on a first guiding instruction and limb reaction images made based on a second guiding instruction, and the audio data being the responses made by the stroke patients based on a third guiding instruction; determining facial paralysis recognition results and eye reaction recognition results based on the facial reaction images; determining limb movement recognition results based on the limb reaction images; determining language articulation recognition results based on the audio data; determining facial paralysis recognition results and eye reaction recognition results; determining language articulation recognition results based on the audio data; determining the risk assessment level of the stroke patients based on the facial paralysis recognition results, eye reaction recognition results, limb movement recognition results, and language articulation recognition results, thereby accurately and quickly assessing the risks of the stroke patients without considering medical environment conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Also, throughout the accompanying drawings, the same reference figures are used to represent the same components. In the drawings: Figure 1 A schematic diagram of the steps of a risk assessment method for stroke patients in an embodiment of the present invention is shown; Figure 2 A schematic diagram showing the distribution of key feature points of each facial feature in an embodiment of the present invention; Figure 3 A schematic diagram showing the point position lines of key feature points of each facial feature in an embodiment of the present invention is shown; Figure 4 A schematic diagram showing an image guidance frame diagram corresponding to a stroke patient in a sitting position according to an embodiment of the present invention is shown; Figure 5 A schematic diagram showing an image guidance frame diagram corresponding to a stroke patient in a supine position according to an embodiment of the present invention is shown; Figure 6 A schematic diagram of the structure of a risk assessment device for stroke patients in an embodiment of the present invention is shown; Figure 7 A schematic diagram of the structure of a computer device for implementing a risk assessment method for stroke patients in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0018] Embodiment 1: The embodiment of the present invention provides a risk assessment method for stroke patients, such as Figure 1 As shown, including: S101, obtaining video images and audio data of a stroke patient, wherein the video images include facial reaction images of the stroke patient based on a first guidance instruction and limb reaction images based on a second guidance instruction, and the audio data are responses of the stroke patient based on a third guidance instruction; S102, determining a facial paralysis recognition result and an eye reaction recognition result based on the facial reaction image; S103, determining limb movement recognition results based on the limb reaction image; S104, determining a language articulation recognition result based on the audio data; S105, determining the risk assessment level of the stroke patient based on the facial paralysis recognition results, the eye reaction recognition results, the limb movement recognition results, and the language articulation recognition results.

[0019] Stroke, commonly known as stroke, includes two types: ischemic stroke (also known as cerebral infarction) and hemorrhagic stroke (including brain parenchymal hemorrhage, intraventricular hemorrhage and subarachnoid hemorrhage). It is a disease in which brain cells and tissues are necrotic.

[0020] Stroke is characterized by high morbidity, high disability, high recurrence rate and high mortality rate, so it needs to be given sufficient attention.

[0021] Common symptoms of stroke include sudden numbness or weakness, especially body shape abnormalities on one side, language or comprehension problems, vision problems, headaches, dizziness, etc. These are all subjective feelings of the patient. In order to be able to intuitively see the patient's performance from the appearance, professional evaluation by the medical staff is required. Therefore, video images and audio data of stroke patients can be collected.

[0022] Therefore, execute S101 to obtain video images and audio data of the stroke patient, wherein the video image includes a facial reaction image of the stroke patient based on the first guidance instruction and a limb reaction image based on the second guidance instruction, and the audio data is the response of the stroke patient based on the third guidance instruction.

[0023] In the process of collecting video images and audio data of stroke patients, voice guidance is needed to enable the stroke patients to cooperate in performing certain actions and responses.

[0024] The first guiding instruction includes: guiding the stroke patient to smile.

[0025] The second guidance instruction includes: guiding the limbs of the stroke patient to reach the appropriate position and make corresponding movements, for example, guiding the stroke patient to sit with the upper limbs raised 90 degrees, guiding the stroke patient to raise the legs 45 degrees when lying on his back, etc.

[0026] The third guiding instruction includes: guiding stroke patients to answer basic questions, etc.

[0027] Next, the collected video images and audio data need to be analyzed. Execute S102 to determine the facial paralysis recognition result and the eye reaction recognition result based on the facial reaction image.

[0028] Here we need to distinguish between facial paralysis recognition and eye reaction recognition.

[0029] For facial paralysis recognition, the following methods are used: Before performing facial paralysis recognition, the method also includes: using a Haar feature classifier to recognize the video image and locate the face area of ​​the stroke patient; and determining the facial reaction image based on the face area.

[0030] The Haar feature classifier can clearly distinguish between human faces and non-human faces, thereby determining the human face area in the video image and obtaining a facial reaction image in the human face area.

[0031] In a specific implementation, each frame of the video image is divided into multiple regions, each region is input into a Haar feature classifier, and the approximate region of the face is determined. When dividing into multiple regions, the division is performed according to the overall body of each limb of the human body.

[0032] After obtaining the approximate face area, the face area is retained by removing the non-face area.

[0033] Next, a facial reaction image of the face region is captured. Since the facial reaction image includes facial expression and eye reaction, the facial paralysis recognition result and the eye reaction recognition result can be determined respectively through the facial reaction image.

[0034] In the process of determining the two recognition results, the following recognition methods are used: Based on the facial reaction image, mark the position of each facial feature in the facial region; Based on the position of each facial feature, determine the key feature points of each facial feature; Connect the key feature points of each facial feature to form a point line; Based on the floating changes of the point line, the facial paralysis recognition results and the eye reaction recognition results are determined.

[0035] like Figure 2 As shown in Figure 1, the distribution of key feature points of each facial feature is shown. Figure 2 In addition to the key feature points of each facial feature, the key feature points of the area between the facial features are also marked, so that based on the point lines of the key feature points of each facial feature, the changes of the point lines of the key feature points in the area between the facial features can be determined to more clearly identify facial reactions. Figure 3 Shown is a schematic diagram of the point line situation of the key feature points of each facial feature.

[0036] Next, the floating changes of the point line are determined by tracking each frame of the facial reaction image.

[0037] Specifically, based on the floating changes of the point line, the changes in the target area are determined; then, based on the changes in the target area, the facial paralysis recognition result and the eye reaction recognition result are determined.

[0038] The target area here is related to the reaction to be recognized. For example, to recognize whether someone is crying, the focus is on the eye area, and to recognize whether someone is smiling, the focus is on the mouth area.

[0039] The following describes facial paralysis recognition and eye reaction recognition separately: When identifying facial paralysis, a facial paralysis identification result is determined based on a first change in the nasolabial groove and a second change in the lower face.

[0040] When the first guidance instruction instructs the stroke patient to smile, the area from both sides of the nose to both sides of the mouth corners is determined according to the changes in the point lines corresponding to the nose and mouth, which is the nasolabial fold, and the area below the mouth, that is, the lower face area, is determined. The nasolabial fold area and the lower face area are used as target areas.

[0041] For the nasolabial groove area, the first change of the nasolabial groove is first collected, and then based on the first change, it is determined whether the nasolabial groove area has become flattened and whether the nasolabial grooves on the left and right sides are asymmetrical.

[0042] For the lower face area, the second change of the lower face is first collected, and then based on the second change, it is determined whether the lower face area is unable to make a smiling action, and whether there is no action on the left or right side of the lower face area.

[0043] Finally, the facial paralysis recognition result is determined according to the judgment result. According to the judgment result of each item, the score of each item is obtained, and the score of each item is added together to obtain the facial paralysis recognition result, that is, the score of facial paralysis recognition.

[0044] Among them, the nasolabial groove is specifically located from the sides of the nose to the corners of the mouth. It is the result of the interaction between the dynamic soft tissue and the non-dynamic soft tissue of the cheek, and it will be highlighted when smiling.

[0045] When the nasolabial fold area becomes flat, it is determined that the patient has facial paralysis, and asymmetry of the nasolabial folds on the left and right sides also indicates that one of the cheeks has facial paralysis.

[0046] The face is usually divided into three areas: upper, middle and lower. The area from the eyes above is called the upper face, the area from the eyes to the corners of the mouth is called the middle face, and the area below is called the lower face.

[0047] When the patient smiles, if there is no obvious change on the left or right side of the lower facial area, it is determined that the lower facial area is unable to make a smiling movement, that is, it is determined that there is facial paralysis.

[0048] If any of the above is judged to be yes, the higher the score result.

[0049] The following describes eye reaction recognition: When identifying eye reactions, it is based on the concentration of the eyes and the reaction of the eyes to stimuli.

[0050] When the first guidance instruction guides the eye movement of the stroke patient, the eye area is used as the target area according to the floating change of the eye point line. For the eye area, the cohesion of the eye is first collected, specifically, the tracking ability corresponding to the eye point line is collected as the tracking target changes. More specifically, the tracking ability of the eye is determined according to the change rate of the point line of the key feature points on the eye.

[0051] At the same time, the eye's reaction to stimuli such as light is also collected to determine whether the eye is paralyzed or unresponsive to the stimulus. More specifically, the reaction time to the stimulus is used to determine whether the eye is paralyzed or unresponsive. The longer the reaction time, the higher the degree of paralysis, and the shorter the reaction time, the lower the degree of paralysis.

[0052] If the eye tracking ability is poor and the degree of paralysis to the stimulus is high, the eye response results are determined, thereby obtaining the eye response score.

[0053] Next, execute S103 to determine the limb movement recognition result based on the limb reaction image.

[0054] Specifically, by providing an image guidance frame, the image guidance frame is used as a control; Determine whether the target limb in the limb reaction image is lifted into the image guidance frame and maintained for a preset time, and obtain a determination result; Based on the judgment results, the limb movement recognition results are determined.

[0055] The first guidance instruction is to guide the patient to raise his upper limbs to 90° when he is in a sitting position, and to guide the patient to raise his legs to 45° when he is in a supine position. Based on the collected limb reaction images combined with the image guidance frame, it is judged whether the target limb, that is, the patient's upper limbs and lower limbs are within the image guidance frame, and the time to complete the target action is also evaluated. Figure 4 , Figure 5 shown.

[0056] The comparison condition of the image guidance frame may be determined according to the proportion of the target limb of the patient located within the image guidance frame.

[0057] According to the completion status of the stroke patient, on the one hand, it is necessary to determine the proportion of the target limb and the image guidance frame, and on the other hand, it is necessary to determine the time to complete the target action, and according to the respective weight ratios, the evaluation results of limb movement recognition are obtained. The specific evaluation results can be divided into upper limb evaluation results and lower limb evaluation results. The final evaluation result of limb movement recognition is the sum of the upper limb evaluation results and the lower limb evaluation results.

[0058] Obviously, the lower the ratio of the target limb to the image guidance frame and the shorter the time to complete the target action, the lower the evaluation result of limb movement recognition, and vice versa.

[0059] Next, the audio data needs to be recognized and evaluated. S104 is executed to determine the speech articulation recognition result based on the audio data.

[0060] Specifically, a natural language processing model is used to analyze the audio data and determine the proportion of error-corrected data; determining the accuracy of the response based on the audio data and the second guidance instruction; The results of language articulation recognition are determined based on the proportion of error-corrected data and the accuracy of the responses.

[0061] Before determining the result of speech articulation recognition, a filter is used to remove environmental or other noise contained in the audio data. Then, the audio data after the noise is filtered out is converted into text data. Next, the error correction data ratio is determined.

[0062] Among them, the natural language processing module (HanLP) can perform error correction on the text data of the audio data, record the number of error corrections, calculate the ratio of the number of error corrections, and use the ratio of the number of error corrections as the weight basis for measuring language clarity.

[0063] Then, based on the audio data and the second guidance instruction, determine the accuracy of the answer. The audio data is a reply based on the second guidance instruction. For example, the second guidance instruction is, how old are you and what is your address. Then, identify the answer, compare the answer for age with the actual situation, and compare the answer for address with the actual situation to obtain the comparison result. Of course, there are other question-and-answer processes, which are not limited here. Based on the comparison results, determine the accuracy of the answer. The higher the accuracy, the lower the score, and the lower the accuracy, the higher the score.

[0064] The evaluation results of the error correction data ratio and the response accuracy are combined to obtain the language articulation recognition results.

[0065] Specifically, the evaluation result of the error correction data ratio and the evaluation result of the answer accuracy are added together to obtain the speech articulation recognition result. The speech articulation recognition result evaluates the audio data from the aspects of accuracy and clarity, and can more accurately reflect the recognition result of the audio data.

[0066] After facial recognition, limb recognition and audio recognition, S105 is executed to determine the risk assessment level of the stroke patient based on the facial paralysis recognition results, eye reaction recognition results, limb movement recognition results and language pronunciation recognition results.

[0067] Specifically, the scores of facial paralysis recognition results, eye reaction recognition results, limb movement recognition results, and language pronunciation recognition results can be added in sequence to obtain an overall score value, and then the overall score value is matched with the risk assessment level to determine the corresponding risk assessment level. For example, the risk assessment levels are low risk, medium risk, and high risk. The higher the overall score value, the higher the risk assessment level of the stroke patient, and the lower the overall score value, the lower the risk assessment level of the stroke patient.

[0068] This assessment method can assist the vast majority of non-professionals in making assessments, and can also quickly identify the severity of the patient's condition in township clinics with relatively poor medical conditions, thereby ensuring that the patient can receive the corresponding treatment in a timely manner.

[0069] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: The present invention provides a risk assessment method for stroke patients, comprising: acquiring video images and audio data of the stroke patients, the video images comprising facial reaction images made by the stroke patients based on a first guiding instruction and limb reaction images made based on a second guiding instruction, and the audio data being the responses made by the stroke patients based on a third guiding instruction; determining facial paralysis recognition results and eye reaction recognition results based on the facial reaction images; determining limb movement recognition results based on the limb reaction images; determining language articulation recognition results based on the audio data; determining the risk assessment level of the stroke patients based on the facial paralysis recognition results, the eye reaction recognition results, the limb movement recognition results, and the language articulation recognition results, thereby accurately and quickly assessing the risks of the stroke patients without considering the medical environment conditions, thereby improving the diagnostic efficiency.

[0070] Embodiment 2: Based on the same inventive concept, the embodiment of the present invention also provides a risk assessment device for stroke patients, such as Figure 6 As shown, including: An acquisition module 601 acquires video images and audio data of a stroke patient, wherein the video images include facial reaction images of the stroke patient based on the first guidance instruction and limb reaction images based on the second guidance instruction, and the audio data are responses of the stroke patient based on the third guidance instruction; A first determination module 602 determines a facial paralysis recognition result and an eye reaction recognition result based on the facial reaction image; A second determination module 603 determines a limb movement recognition result based on the limb reaction image; A third determination module 604 determines a language articulation recognition result based on the audio data; The fourth determination module 605 determines the risk assessment level of the stroke patient based on the facial paralysis recognition result, the eye reaction recognition result, the limb movement recognition result, and the language articulation recognition result.

[0071] In an optional implementation, the method further includes: a fifth determining module, configured to: Using a Haar feature classifier to identify the video image and locate the facial area of ​​the stroke patient; Based on the face region, a facial reaction image is determined.

[0072] In an optional implementation, the first determining module 602 is configured to: Based on the facial reaction image, marking the position of each facial feature in the facial region; Based on the position of each facial feature, determine the key feature points of each facial feature; Connect the key feature points of each facial feature to form a point line; Based on the floating changes of the point line, the facial paralysis recognition results and the eye reaction recognition results are determined.

[0073] In an optional implementation, the first determining module 602 is configured to: Determine the change of the target area based on the floating change of the point line; Based on the changes in the target area, a facial paralysis recognition result and an eye expression reaction recognition result are determined.

[0074] In an optional implementation, the first determining module 602 is configured to: When identifying facial paralysis, determining a facial paralysis identification result based on a first change in the nasolabial groove and a second change in the lower face; When recognizing the eye reaction situation, the eye reaction recognition result is determined based on the concentration of the eyeballs and the reaction of the eyeballs to the stimulus.

[0075] In an optional implementation manner, the second determining module 603 is configured to: By providing an image guide frame, the image guide frame is used as a control; Determine whether the target limb in the limb reaction image is lifted into the image guidance frame and maintained for a preset time, and obtain a determination result; Based on the judgment result, the limb movement recognition result is determined.

[0076] In an optional implementation, the third determining module 604 is configured to: The audio data is analyzed using a natural language processing model to determine the proportion of error-corrected data; Determining the accuracy of the response based on the audio data and the second guidance instruction; Based on the error correction data ratio and the answer accuracy rate, a language articulation recognition result is determined.

[0077] Embodiment three: Based on the same inventive concept, an embodiment of the present invention provides a computer device, such as Figure 7 As shown, it includes a memory 704, a processor 702, and a computer program stored in the memory 704 and executable on the processor 702. When the processor 702 executes the program, the steps of the above-mentioned risk assessment method for stroke patients are implemented.

[0078] Among them, Figure 7 In the embodiment of the present invention, a bus architecture (represented by bus 700) is shown, and bus 700 may include any number of interconnected buses and bridges, and bus 700 links various circuits including one or more processors represented by processor 702 and memory represented by memory 704. Bus 700 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 706 provides an interface between bus 700 and receiver 701 and transmitter 703. Receiver 701 and transmitter 703 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices on a transmission medium. Processor 702 is responsible for managing bus 700 and general processing, while memory 704 may be used to store data used by processor 702 when performing operations.

[0079] Embodiment 4: Based on the same inventive concept, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned risk assessment method for stroke patients when executed by a processor.

[0080] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0081] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0082] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recorded in each embodiment. More specifically, as reflected in each embodiment, the inventive aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0083] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0084] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in a specific embodiment, any one of the claimed embodiments may be used in any combination.

[0085] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components of a risk assessment device for stroke patients and a computer device according to an embodiment of the present invention. The present invention may also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0086] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A risk assessment method for stroke patients, characterized in that: include: Acquire video images and audio data of a stroke patient, wherein the video images include facial reaction images of the stroke patient based on the first guidance instruction and limb reaction images based on the second guidance instruction, and the audio data are responses of the stroke patient based on the third guidance instruction; Determining a facial paralysis recognition result and an eye reaction recognition result based on the facial reaction image; Determining limb movement recognition results based on the limb reaction image; Determining a language pronunciation recognition result based on the audio data; Based on the facial paralysis recognition results, eye reaction recognition results, limb movement recognition results, and language articulation recognition results, the risk assessment level of the stroke patient is determined.

2. The method according to claim 1, characterized in that Before determining the facial paralysis recognition result and the eye reaction recognition result based on the facial reaction image, the method further includes: Using a Haar feature classifier to identify the video image and locate the facial area of ​​the stroke patient; Based on the face region, a facial reaction image is determined.

3. The method according to claim 1, characterized in that Based on the facial reaction image, determining a facial paralysis recognition result and an eye reaction recognition result includes: Based on the facial reaction image, marking the position of each facial feature in the facial region; Based on the position of each facial feature, determine the key feature points of each facial feature; Connect the key feature points of each facial feature to form a point line; Based on the floating changes of the point line, the facial paralysis recognition results and the eye reaction recognition results are determined.

4. The method according to claim 3, characterized in that Based on the floating changes of the point line, the facial paralysis recognition results and eye reaction recognition results are determined, including: Determine the change of the target area based on the floating change of the point line; Based on the changes in the target area, a facial paralysis recognition result and an eye expression reaction recognition result are determined.

5. The method according to claim 4, characterized in that Based on the change of the target area, the facial paralysis recognition result and the eye reaction recognition result are determined, including: When identifying facial paralysis, determining a facial paralysis identification result based on a first change in the nasolabial groove and a second change in the lower face; When recognizing the eye reaction situation, the eye reaction recognition result is determined based on the concentration of the eyeballs and the reaction of the eyeballs to the stimulus.

6. The method according to claim 1, characterized in that Based on the limb reaction image, determining a limb movement recognition result includes: By providing an image guide frame, the image guide frame is used as a control; Determine whether the target limb in the limb reaction image is lifted into the image guidance frame and maintained for a preset time, and obtain a determination result; Based on the judgment result, the limb movement recognition result is determined.

7. The method according to claim 1, characterized in that Determining a language pronunciation recognition result based on the audio data includes: The audio data is analyzed using a natural language processing model to determine the proportion of error-corrected data; Determining the accuracy of the response based on the audio data and the second guidance instruction; Based on the error correction data ratio and the answer accuracy rate, a language articulation recognition result is determined.

8. A risk assessment device for stroke patients, characterized in that: include: An acquisition module, used to acquire video images and audio data of a stroke patient, wherein the video images include facial reaction images made by the stroke patient based on the first guidance instruction and limb reaction images made based on the second guidance instruction, and the audio data are responses made by the stroke patient based on the third guidance instruction; A first determination module, used to determine a facial paralysis recognition result and an eye reaction recognition result based on the facial reaction image; A second determination module is used to determine a limb movement recognition result based on the limb reaction image; A third determination module, used to determine a language articulation recognition result based on the audio data; The fourth determination module is used to determine the risk assessment level of the stroke patient based on the facial paralysis recognition results, eye reaction recognition results, limb movement recognition results and language articulation recognition results.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images