Cranial nerve motor function evaluation system and method based on terminal with camera

Abstract

The invention relates to a cranial nerve motor function evaluation system and method based on a terminal with a camera. The system comprises a camera used for collecting a high-definition video sequence of facial actions; the model is used for identifying positioning points of the face according to the high-definition video sequence acquired by the camera; the analysis and evaluation module is used for quantitatively evaluating the cranial nerve motor function state according to the change of the positioning points of the face acquired by the model; the analysis and evaluation module performs analysis and evaluation according to the change quantitative indexes of the positioning points of the face; the camera, the model and the analysis and evaluation module are all deployed at the terminal; and the model and the analysis and evaluation module are deployed in an operating system of the terminal. The evaluation system and method are deployed on a common terminal, and are used for collecting a video sequence of a user completing a standard evaluation action, automatically identifying the movement of a positioning point of a face, and outputting a quantitative score according to a medical standard so as to evaluate human face muscle movement and a cranial nerve function state.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and neurological function assessment, and in particular to a cranial nerve motor function assessment system and method based on a terminal with a camera. Background Technology

[0002] Assessment of cranial nerve motor function is not only important for disease diagnosis and management, but also has certain value for healthy individuals, as detailed below:

[0003] I. Significance and Role for Disease-Related Populations

[0004] 1. Early detection of lesions

[0005] Cranial nerves control movement in key areas such as the head, face, and throat. Assessment can help identify facial muscle movement dysfunction caused by stroke, brain tumors, neuroinflammation, and neurodegenerative diseases (such as Parkinson's disease) at an early stage, allowing time for early intervention.

[0006] 2. Locate the lesion

[0007] Different cranial nerves correspond to specific brain regions or pathways. For example, oculomotor nerve abnormalities suggest midbrain lesions, while facial nerve problems may be related to brainstem or facial canal damage. Assessment can accurately locate the lesion site.

[0008] II. Effects on healthy individuals

[0009] 1. Health screening and risk warning

[0010] Some diseases may not have obvious symptoms in their early stages, but subtle abnormalities in cranial nerve motor function may be signals (such as reduced facial expressions in early Parkinson's disease, bradykinesia leading to slow eye movements, and facial paralysis caused by brain tumors). Regular assessments can help screen for potential risks.

[0011] 2. Occupational health monitoring

[0012] For professions that require frequent use of facial muscles (such as singers, teachers, and pilots), or for those who work under high pressure and for long hours, assessment can help detect overuse-induced neurological fatigue or damage early and adjust habits in a timely manner to prevent disease.

[0013] 3. Establishment and Trend Analysis of Basic Health Baselines

[0014] Recording cranial nerve motor function data in a healthy state and analyzing changes in health status based on continuously recorded data can not only provide a reference for comparing symptoms when illness occurs in the future, helping doctors to more accurately judge changes in the condition, but also allow for self-health protection based on health trends, changing work and rest methods, and maintaining a healthy state.

[0015] 4. Raise health awareness

[0016] By assessing and understanding the relationship between cranial nerve function and overall health, we can enhance the cultivation and emphasis on healthy habits in life and work, such as brain usage habits, regular work and rest, and effective release of work pressure.

[0017] In summary, cranial nerve motor function assessment provides crucial information for disease diagnosis and treatment, and also plays a role in prevention and monitoring in healthy individuals, making it an important means of maintaining nervous system health.

[0018] Currently, clinical assessment of cranial nerve motor function mainly relies on neurologists observing the muscle responses of patients after performing specific facial movements and scoring them using methods such as the Sunnybrook facial paralysis rating scale. This assessment method has problems such as high subjectivity, difficulty in quantification, inability to track long-term progress, and unsuitability for home or remote settings.

[0019] With the development of computer vision and artificial intelligence technologies, video-based automatic assessment methods have become a new research trend. However, most of them currently focus on static image analysis or rough modeling based on organ points (such as the corners of the eyes and mouth), lacking complete technical solutions for medical standard movements, muscle cluster recognition, time series analysis, and historical trend modeling. Summary of the Invention

[0020] The purpose of this invention is to provide a cranial nerve motor function assessment system and method based on a terminal with a camera, which mainly solves the problems existing in the prior art. The assessment system and method are deployed on ordinary terminals (such as portable computers, commonly known as laptops) to collect video sequences of users completing standard assessment actions, automatically identify the movement trajectory and patterns of facial positioning points, and output quantitative scores according to medical standards, thereby assessing the state of human facial muscle movement and cranial nerve function, and is particularly suitable for human health monitoring and medical auxiliary diagnosis.

[0021] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0022] A cranial nerve motor function assessment system based on a terminal with a camera, characterized in that: it includes,

[0023] A camera used to capture high-definition video sequences of facial movements;

[0024] A model for facial landmark recognition based on high-definition video sequences captured by the camera;

[0025] An analysis and evaluation module is used to quantitatively assess the state of cranial nerve motor function based on changes in facial positioning points collected by the model. The analysis and evaluation module performs analysis and evaluation based on quantitative indicators of changes in facial positioning points, i.e., a scoring system.

[0026] The camera, model, and analysis and evaluation module are all deployed on the terminal; wherein, the model and analysis and evaluation module are both deployed in the terminal's operating system.

[0027] The cranial nerve motor function assessment system based on a terminal with a camera is characterized in that: the terminal is a laptop computer.

[0028] The cranial nerve motor function assessment system based on a terminal with a camera is characterized in that the parameters of the camera are not less than 1080P and 30FPS.

[0029] The cranial nerve motor function assessment system based on a terminal with a camera is characterized in that: the system further includes a feedback module for the analysis and assessment results to be output to the user.

[0030] A method for assessing cranial nerve motor function using the system described above, characterized by comprising the following steps:

[0031] A. The system guides the user to complete several standard facial movements in sequence according to the actions, sequence of actions, and requirements specified in the applicable scoring system.

[0032] B. The system acquires high-definition video sequence data of the facial movements described in step A through the camera;

[0033] C. The system will input the high-definition video sequence data of the facial movements into the model to identify the facial positioning points.

[0034] D. The system inputs the changes in the facial positioning points identified in step C into the analysis and evaluation module, and uses the analysis and evaluation module to perform analysis and evaluation based on the quantitative indicators of the changes in the facial positioning points.

[0035] All the above methods and steps are run locally on the system.

[0036] The method is characterized in that:

[0037] It also includes step E;

[0038] E. Output the analysis and evaluation results to the user through the feedback module.

[0039] The method is characterized in that:

[0040] The model is a facial landmark recognition model obtained through AI deep learning training.

[0041] The method is characterized in that:

[0042] In step A, a complete sequence of user interaction actions is set up with reference to the Sunnybrook rating system, including multiple facial nerve control actions such as closing eyes, raising eyebrows, frowning, smiling, showing teeth, and puffing out cheeks, and voice / image prompts are provided.

[0043] The method is characterized in that:

[0044] In step B, the system uses a built-in image brightness detection module to automatically determine the ambient light conditions during the acquisition process. When the brightness is below the threshold, it guides the user to improve the background light source to ensure the quality of the acquired video.

[0045] The method is characterized in that:

[0046] In step B, the system automatically corrects head offset through a built-in facial pose estimation and correction module, thereby improving the stability of motion capture.

[0047] The method is characterized in that:

[0048] The system is equipped with a history record module, which records the user's historical evaluation data and establishes a trend analysis model; step D further includes comparing the current round of new analysis and evaluation with the trend analysis model.

[0049] The method is characterized in that:

[0050] The model is a multi-point facial localization point recognition model; the model includes localization points including multiple semantic points and contour points on the human face; these localization points are completely captured by the camera in a time sequence during facial movements;

[0051] Wherein: the semantic points are actually measurement reference points, and the selection principles are: a) how the muscles of the human face are driven by cranial nerves, these points always maintain a relatively uncertain position; and b) these points also have corresponding distributions on the left and right sides of the human face; and c) these points need to be accurately identified and located at the pixel level of the camera of the system of the present invention.

[0052] When the model is used, the facial position of the person at least two time points during facial movements is located and aligned using semantic points. After alignment, other positioning points are measured and calculated to determine whether displacement occurs during facial movements, and the displacement distance, speed, and angle. The measurement and calculation results are then input into the analysis and evaluation module for evaluation and analysis.

[0053] The method is characterized in that:

[0054] The evaluation and analysis include comparing and scoring the measurement and calculation results of one or more positioning points during facial movements with the scoring system.

[0055] The method is characterized in that:

[0056] When the model is used, the symmetry of certain positioning points is measured and calculated; then the results of the measurement and calculation are input into the analysis and evaluation module for evaluation and analysis.

[0057] In view of the above technical features, compared with the prior art, the present invention has the following advantages:

[0058] 1. The system and method of this invention realize a closed-loop architecture of guidance-collection-identification-scoring-tracking.

[0059] 2. The system and method of this invention supplement and enrich users' daily lives with a relatively inexpensive way to check the health of cranial nerve function.

[0060] 3. The system and method of this invention overcome the problem that existing neurological function assessments (such as facial paralysis assessments) rely heavily on doctors' visual observation and subjective scoring, and their accuracy and consistency are greatly affected by human experience.

[0061] 4. Through the system and method of this invention, an innovative facial positioning point recognition model (e.g., 416-point model) can be used to realize the automated and quantitative analysis of facial nerve motion, eliminate subjective bias, and improve the standardization of assessment.

[0062] 5. The system and method of this invention can adopt the Sunnybrook standard operating procedure to ensure that the evaluation results are comparable and medically applicable.

[0063] 6. The system and method of this invention, combined with the high-speed, high-resolution motion analysis, ambient light detection and automatic head movement correction technology of the camera in this invention, can be adapted to various shooting environments, improving robustness and clinical applicability.

[0064] 7. The system and method of this invention can support telemedicine and health management. It aligns with the trend of telemedicine and home-based self-health management, reducing the frequency and cost of hospital visits, and is particularly suitable for aging populations and chronic disease management scenarios. It promotes the equitable distribution of medical resources, especially in remote areas and primary healthcare institutions. By popularizing such systems, it can compensate for the shortage of professional neurological function assessment personnel and improve the early screening and rehabilitation capabilities for neurological diseases.

[0065] 8. The system and method of this invention are applicable to both civilian and medical scenarios. For example, neurological function testing can be achieved using ordinary consumer-grade devices such as laptops, providing users with a low-cost and sustainable means of neurological health management. In clinical settings, it can serve as an auxiliary diagnostic tool for doctors, supporting postoperative rehabilitation tracking, remote follow-up, and home-based rehabilitation. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the structure of the cranial nerve motor function assessment system based on a terminal with a camera, according to the present invention.

[0067] Figure 2 This is a flowchart of the cranial nerve motor function assessment method based on a terminal with a camera, according to the present invention.

[0068] Figure 3 This is a schematic diagram of the layout of each positioning point in the facial positioning point recognition model of the present invention. Detailed Implementation

[0069] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0070] Please see Figure 1 This invention discloses a cranial nerve motor function assessment system based on a terminal with a camera. As shown in the figure, it includes:

[0071] A camera used to capture high-definition video sequences of facial movements;

[0072] A model for facial landmark recognition based on high-definition video sequences captured by the camera;

[0073] An analysis and evaluation module is used to quantitatively assess the state of cranial nerve motor function based on changes in facial positioning points acquired by the model. The analysis and evaluation module performs analysis and evaluation based on quantitative indicators of changes in facial positioning points, i.e., a scoring system.

[0074] In this invention, the camera, model, and analysis and evaluation module are all deployed on the terminal; wherein, the model and analysis and evaluation module are both deployed in the terminal's operating system.

[0075] In this invention, the terminal is a laptop computer or a desktop computer with a camera, such as laptops and desktop computers manufactured by HP and Lenovo.

[0076] The camera parameters are no less than 1080P and 30FPS.

[0077] In this invention, the system also includes a feedback module for the analysis and evaluation results to the user.

[0078] like Figure 2 As shown, this is a method for assessing cranial nerve motor function using the system described above, according to the present invention. As shown in the figure, it includes the following steps:

[0079] A. The system guides the user to complete several standard facial movements in sequence according to the actions, sequence of actions, and requirements specified in the applicable scoring system.

[0080] B. The system acquires high-definition video sequence data of the facial movements described in step A through the camera;

[0081] C. The system will input the high-definition video sequence data of the facial movements into the model to identify the facial positioning points.

[0082] D. The system inputs the changes in the facial positioning points identified in step C into the analysis and evaluation module, and uses the analysis and evaluation module to perform analysis and evaluation based on the quantitative indicators of the changes in the facial positioning points.

[0083] All the above methods and steps are run locally on the system.

[0084] It also includes step E;

[0085] E. Output the analysis and evaluation results to the user through the feedback module.

[0086] The model is a facial landmark recognition model obtained through AI deep learning training.

[0087] In step A, a complete sequence of user interaction actions is set up with reference to the Sunnybrook rating system, including multiple facial nerve control actions such as closing eyes, raising eyebrows, frowning, smiling, showing teeth, and puffing out cheeks, and voice / image prompts are provided.

[0088] In step B, the system uses a built-in image brightness detection module to automatically determine the ambient light conditions during the acquisition process. When the brightness is below the threshold, it guides the user to improve the background light source to ensure the quality of the acquired video.

[0089] In step B, the system automatically corrects head offset through a built-in facial pose estimation and correction module, thereby improving the stability of motion capture.

[0090] The system is equipped with a history record module, which records the user's historical evaluation data and establishes a trend analysis model; step D further includes comparing the current round of new analysis and evaluation with the trend analysis model.

[0091] In this invention, the model is a multi-point facial positioning point recognition model; the model includes positioning points including multiple semantic points and contour points on the human face; these positioning points are completely captured by the camera in a temporal sequence during facial movements;

[0092] Wherein: the semantic points are actually measurement reference points, and the selection principles are: a) how the muscles of the human face are driven by cranial nerves, these points always maintain a relatively uncertain position; and b) these points also have corresponding distributions on the left and right sides of the human face; and c) these points need to be accurately identified and located at the pixel level of the camera of the system of the present invention.

[0093] When the model is used, the facial position of the person at least two time points during facial movements is located and aligned using semantic points. After alignment, other positioning points are measured and calculated to determine whether displacement occurs during facial movements, and the displacement distance, speed, and angle. The measurement and calculation results are then input into the analysis and evaluation module for evaluation and analysis.

[0094] When the model is used, the symmetry of certain positioning points is measured and calculated; then the results of the measurement and calculation are input into the analysis and evaluation module for evaluation and analysis.

[0095] In use, the evaluation and analysis include comparing the measurement and calculation results of one or more positioning points during facial movements with the scoring system for evaluation and scoring.

[0096] As one example, such as Figure 3 As shown, the present invention uses a recognition model with 416 facial positioning points.

[0097] The 416-point facial localization point recognition model primarily categorizes localization points into semantic points and contour points. Specifically, there are 30 semantic points and 386 contour points. The following section combines... Figure 3 The diagrams further illustrate these location points.

[0098] Table 1: Semantic points and their distribution.

[0099] total Facial contour Eyebrow Eye nose mouth Quantity (pieces) 30 2 6 10 6 6 Location Table 3 Table 3 Table 3 Table 3 Table 3 Table 3

[0100] Table 2: Outline points.

[0101]

[0102] Table 3: Index table of semantic points and partial contour line points corresponding to facial positions.

[0103]

[0104]

[0105] It should be noted that the semantic points mentioned are actually measurement reference points. The selection principles are: a) these points maintain relatively variable positions due to how facial muscles are driven by cranial nerves; b) these points are also distributed on both the left and right sides of the face; and c) these points need to be accurately identified and located at the pixel level of the camera in this invention's system. For example, points 392 (left eyelid inner canthus) and 397 (right eyelid inner canthus) in Table 3, i.e., the two inner canthi points of the face, possess this characteristic. By selecting these semantic points, the measurement reference can be determined, improving the overall measurement progress.

[0106] Table 4: Scoring system (mapping table) in the analysis and evaluation module corresponding to the recognition model of 416 facial positioning points.

[0107]

[0108]

[0109] Since the 416-point model of this invention has identified these 30 semantic points, when the system of this invention is used, there is a relatively stable benchmark for measuring the movement of other positioning points besides these semantic points, thus improving the reliability of the measurement.

[0110] The location points (especially semantic points) of the 416-point model of the present invention, as shown in Tables 1-3 above, were selected and found through training on a large number of different faces. Compared with the 68-point facial recognition model in the prior art, its recognition ability and effect have been greatly improved.

[0111] For example, the system of this invention uses a high-definition camera to capture a time-series video of a user's facial action (such as pouting or puffing out their cheeks). During the acquisition process, the positions of all positioning points are completely captured in the time sequence. It should be noted that the system guides the user through voice prompts or video demonstrations to perform the action of pouting or puffing out their cheeks (instructing the patient to close their mouth and forcefully puff out their cheeks).

[0112] When using the model, semantic points are used to locate and align the facial position at at least two time points during facial movements. After alignment, other positioning points (e.g., point 121 in the model) are measured and calculated to determine whether displacement occurs during facial movements, and the displacement distance, speed, and angle. The measurement and calculation results are then input into the analysis and evaluation module for evaluation and analysis. In use, the evaluation and analysis includes comparing the measurement and calculation results of one or more positioning points during facial movements with the scoring system for evaluation and scoring. For example, if a positioning point should have displaced during cheek puffing but did not (e.g., the displacement of point 121 is 0), an evaluation result for the cranial nerve controlling a certain facial muscle at that location is given. If a certain positioning point is displaced, but the displacement distance and / or speed differ from the expected standard, an assessment result is given (e.g., when pouting and blowing air, if the displacement of point 121 is 0-3, it is judged that the facial muscles at that point are not controlled by the corresponding cranial nerve; if the displacement is 3.1-5, it is judged that the facial muscles at that point have poor ability to be controlled by the corresponding cranial nerve; if the displacement is 5.1-7, it is judged that the facial muscles at that point are normally controlled by the corresponding cranial nerve).

[0113] In addition, when using the model, the symmetry of certain positioning points is measured and calculated; then the results of the measurement and calculation are input into the scoring system in the analysis and evaluation module for evaluation and analysis. Taking the facial action of pouting and puffing out as an example, if, during the performance of this action, certain positioning points that should move simultaneously and symmetrically according to the scoring system do not achieve symmetry, an evaluation result of the cranial nerve controlling a certain facial muscle at that location is given.

[0114] Therefore, based on the aforementioned assessment system and method, this invention can objectively and efficiently assess the motor function of cranial nerves. The assessment results can be saved in the terminal according to the assessment date for subsequent tracking. Furthermore, it can generate a trend analysis and health warning report based on multiple assessment results within a specific time period. In addition, this system and method can provide structured quantitative data for professional physicians, effectively supporting clinical auxiliary diagnosis and remote follow-up examinations.

[0115] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A cranial nerve motor function assessment system based on a terminal with a camera, characterized in that: It includes, A camera used to capture high-definition video sequences of facial movements; A model for locating facial key points and recognizing muscle clusters based on high-definition video sequences captured by the camera; An analysis and evaluation module is used to quantitatively assess the state of cranial nerve motor function based on changes in facial positioning points acquired by the model. The analysis and evaluation module performs analysis and evaluation based on quantitative indicators of changes in facial positioning points, i.e., a scoring system. The camera, model, and analysis and evaluation module are all deployed on the terminal; wherein, the model and analysis and evaluation module are both deployed in the terminal's operating system.

2. The cranial nerve motor function assessment system based on a terminal with a camera according to claim 1, characterized in that: The terminal is a laptop computer.

3. The cranial nerve motor function assessment system based on a terminal with a camera according to claim 1, characterized in that: The camera's specifications are no less than 1080P and 30FPS.

4. The cranial nerve motor function assessment system based on a terminal with a camera according to claim 1, characterized in that: The system also includes a feedback module for the analysis and evaluation results to the user.

5. A method for assessing cranial nerve motor function using the system described in claim 1, 2, 3, or 4, characterized in that: It includes the following steps: A. The system guides the user to complete several standard facial movements in sequence according to the actions, sequence of actions, and requirements specified in the applicable scoring system. B. The system acquires high-definition video sequence data of the facial movements described in step A through the camera; C. The system will input the high-definition video sequence data of the facial movements into the model to identify the facial positioning points. D. The system inputs the changes in the facial positioning points identified in step C into the analysis and evaluation module, and uses the analysis and evaluation module to perform analysis and evaluation based on the quantitative indicators of changes in key facial points and muscle clusters. All the above methods and steps are run locally on the system.

6. The method according to claim 5, characterized in that: It also includes step E; E. Output the analysis and evaluation results to the user through the feedback module.

7. The method according to claim 5, characterized in that: The model is a facial landmark recognition model obtained through AI deep learning training.

8. The method according to claim 5, characterized in that: In step A, a complete sequence of user interaction actions is set up with reference to the Sunnybrook rating system, including multiple facial nerve control actions such as closing eyes, raising eyebrows, frowning, smiling, showing teeth, and puffing out cheeks, and voice / image prompts are provided.

9. The method according to claim 5, characterized in that: In step B, the system uses a built-in image brightness detection module to automatically determine the ambient light conditions during the acquisition process. When the brightness is below the threshold, it guides the user to improve the background light source to ensure the quality of the acquired video.

10. The method according to claim 5, characterized in that: In step B, the system automatically corrects head offset through a built-in facial pose estimation and correction module, thereby improving the stability of motion capture.

11. The method according to claim 5, characterized in that: The system is equipped with a history record module, which records the user's historical evaluation data and establishes a trend analysis model; step D further includes comparing the current round of new analysis and evaluation with the trend analysis model.

12. The method according to claim 5, characterized in that: The model is a multi-point facial positioning point recognition model; the model includes positioning points including multiple semantic points and contour points on the human face; these positioning points are completely captured by the camera in a time sequence during facial movements; Wherein: the semantic points are actually measurement reference points, and the selection principles are: a) how the muscles of the human face are driven by cranial nerves, these points always maintain a relatively uncertain position; and b) these points also have corresponding distributions on the left and right sides of the human face; and c) these points need to be accurately identified and located at the pixel level of the camera of the system of the present invention. When the model is used, the facial position of the person at least two time points during facial movements is located and aligned using semantic points. After alignment, other positioning points are measured and calculated to determine whether displacement occurs during facial movements, and the displacement distance, speed, and angle. The measurement and calculation results are then input into the analysis and evaluation module for evaluation and analysis.

13. The method according to claim 12, characterized in that: When the model is used, the symmetry of certain positioning points is measured and calculated; then the results of the measurement and calculation are input into the analysis and evaluation module for evaluation and analysis.

14. The method according to claim 12, characterized in that: The evaluation and analysis include comparing and scoring the measurement and calculation results of one or more positioning points during facial movements with the scoring system.

Images