Video production method and device for Alzheimer's disease screening

Abstract

The present invention relates to a video production method and device for Alzheimer's disease screening. The method comprises: collecting eye movement data of healthy subjects when watching videos and performing data preprocessing, including dividing the data into time windows; calculating the similarity scores of eye movement data of different subjects for each time window, and screening high-similarity video clips through a preset threshold; extracting representative eye movement features from high-similarity video clips; collecting eye movement data of Alzheimer's patients when watching the same video, calculating the similarity scores with the representative eye movement features, and screening video clips that can significantly distinguish Alzheimer's patients from healthy subjects through a preset threshold; checking whether the screened video clips are adjacent, and merging adjacent video clips. The present invention improves the comprehensiveness and accuracy of video clip similarity comparison, can more accurately compare and analyze eye movement data, and provides a more reliable basis for research and diagnosis.

Description

Technical Field

[0001] The present invention belongs to the fields of medical technology, eye tracking technology and video technology, and specifically relates to a video production method and device for Alzheimer's disease screening. Background Art

[0002] Eye tracking technology has made significant progress in cognitive science, psychology, and medical research. This technology can accurately record the movement of an individual's eyeballs when viewing visual stimuli, thereby providing important information for studying the distribution of visual attention and cognitive processing. Especially when studying long-term visual stimuli (such as video content), eye tracking provides a unique method for understanding how viewers process complex and dynamic visual scenes. In the study of neurodegenerative diseases such as Alzheimer's Disease (AD), early diagnosis and understanding of the impact of the disease on cognitive function are crucial. Eye tracking technology can provide information about cognitive processes non-invasively and directly, making it a useful tool in this research field. However, there is currently no systematic video production technology. In response to this challenge, the purpose of the present invention is to develop an advanced video production technology that can accurately extract and analyze eye movement data in videos, especially when comparing the responses of AD patients and healthy individuals.

[0003] Despite advances in accuracy and scope of application, eye tracking technology still faces challenges of complexity and diversity when processing long video content. In particular, when comparing the responses of different subjects to the same stimulus, effectively identifying video clips that elicit similar eye movement responses remains an unsolved problem. Currently, there is no systematic video production technology specifically designed to address this challenge. Therefore, the goal of this invention is to develop advanced video production technology to accurately extract and analyze eye movement data in videos, especially when comparing the responses of AD patients and healthy individuals. This invention will focus on the following key issues:

[0004] 1) Challenges of complexity and diversity in data processing: Develop efficient algorithms to process eye movement data in long videos and accurately identify and analyze eye movement responses of different subjects, tasks, and stimuli.

[0005] 2) Effectively identify video clips with similar eye movement responses: Develop intelligent algorithms that can quickly and accurately identify video clips that elicit similar eye movement responses, so that researchers can more effectively compare different subjects’ responses to the same stimuli. Summary of the invention

[0006] In order to overcome the shortcomings of the prior art, the present invention provides a video production method based on eye movement data similarity, which is suitable for Alzheimer's disease screening, such as early Alzheimer's disease family screening, etc.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A video production method for Alzheimer's disease screening comprises the following steps:

[0009] Collect eye movement data from healthy subjects while watching videos;

[0010] Perform data preprocessing on the collected eye movement data, including dividing the eye movement data into time windows;

[0011] For each time window, the similarity scores of eye movement data of different subjects are calculated;

[0012] According to the similarity scores of the calculated eye movement data, high-similarity video clips are filtered through a preset threshold;

[0013] Extract representative eye movement features from the screened high-similarity video clips;

[0014] Collecting the eye movement data of patients with Alzheimer's disease when watching the same video, and calculating the similarity score between the eye movement data and the representative eye movement features;

[0015] According to the calculated similarity score between the eye movement data of the Alzheimer's patient and the representative eye movement features, the video clips that can significantly distinguish between the Alzheimer's patient and the healthy subjects are screened through a preset threshold;

[0016] Check whether the screened video clips that can significantly distinguish Alzheimer's disease patients from healthy subjects are adjacent, and merge the adjacent video clips.

[0017] Furthermore, the calculation of similarity scores of eye movement data of different subjects includes calculating distance similarity and direction similarity, and finally obtaining an average similarity score for each time window.

[0018] Furthermore, the method of screening high-similarity video clips through a preset threshold includes: setting a similarity score critical threshold T1, when the similarity score of the eye movement data is greater than the critical threshold T1, the video clip corresponding to the eye movement data is a high-similarity video clip; when the similarity score of the eye movement data is less than the critical threshold T1, the video clip corresponding to the eye movement data is an ordinary video clip.

[0019] Furthermore, the representative eye movement feature is an average value of eye movement coordinates of all healthy subjects in high-similarity video clips.

[0020] Furthermore, the video clips that can significantly distinguish between Alzheimer's disease patients and healthy subjects by screening with a preset threshold value include:

[0021] A similarity threshold T2 is set, which is used to divide the differences in visual attention between Alzheimer's patients and healthy subjects, and the proportion r below the similarity threshold T2 is counted, where r is the confidence level of the video clips in distinguishing Alzheimer's patients from healthy subjects;

[0022] A confidence threshold T3 is set, and segments with high confidence are screened out according to the confidence threshold T3, which are the required video segments.

[0023] Furthermore, the merging of adjacent video clips includes: merging the video clips respectively according to divided video windows of different lengths, and then performing a union process.

[0024] Furthermore, the screened video clips that can significantly distinguish Alzheimer's patients from normal people are graded according to the similarity scores within the video clips, and are divided into four levels: asymptomatic, mild, moderate and severe, to judge the degree of the disease in the subjects.

[0025] A video production device for Alzheimer's disease screening, comprising:

[0026] A healthy subject eye movement data collection module, used to collect eye movement data of healthy subjects when watching videos;

[0027] A data preprocessing module, used for preprocessing the collected eye movement data, including dividing the eye movement data into time windows;

[0028] A first similarity analysis module is used to calculate similarity scores of eye movement data of different subjects for each time window;

[0029] A high-similarity video segment identification module is used to filter high-similarity video segments through a preset threshold according to the calculated similarity scores of the eye movement data;

[0030] A representative eye movement feature extraction module is used to extract representative eye movement features from the screened high-similarity video clips;

[0031] The Alzheimer's disease patient eye movement data collection module is used to collect the eye movement data of Alzheimer's disease patients when they watch the same video;

[0032] A second similarity analysis module is used to calculate the similarity score between the eye movement data collected from Alzheimer's patients when watching the same video and the representative eye movement features;

[0033] A video clip screening module, for screening video clips that can significantly distinguish Alzheimer's patients from healthy subjects through a preset threshold value according to the calculated similarity score between the eye movement data of the Alzheimer's patient and the representative eye movement features;

[0034] The video segment merging module is used to check whether the screened video segments that can significantly distinguish Alzheimer's disease patients from healthy subjects are adjacent, and merge the adjacent video segments.

[0035] Compared with the prior art, the video production method for detecting Alzheimer's disease patients based on eye movement data of the present invention has the following advantages and benefits:

[0036] 1) Comprehensiveness and accuracy: By using multiple metrics such as cosine similarity and Euclidean distance, the similarity of the direction and distance of video clips is comprehensively considered, which improves the comprehensiveness and accuracy of the similarity comparison of video clips.

[0037] 2) Refined video search: By dividing the video into small windows and gradually merging adjacent segments, a more refined video search is achieved. This helps users locate and retrieve the required video content more accurately, improving the accuracy and efficiency of the search.

[0038] 3) Application of representative features: Based on the uniform eye movement features in healthy subjects as standard features, a benchmark is established for evaluating the eye movement patterns of other subjects or patients. This helps to more accurately compare and analyze eye movement data, providing a more reliable basis for research and diagnosis.

[0039] 4) Comprehensive video analysis and search experience: By comprehensively applying the above methods, comprehensive video analysis and search are achieved. Not only can the characteristics and similarities of video content be better understood, but the required video clips can also be searched more effectively, providing users with a more comprehensive and accurate video analysis and search experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 4 is a flowchart of the steps of the video production method for detecting AD patients in the embodiment.

[0041] Figure 2 4 is a flowchart of the steps of the method for calculating the similarity of eye movement data of healthy subjects in an embodiment.

[0042] Figure 3 It is a flowchart of the steps of the method for calculating the similarity between the eye movement features of AD patients and representative eye movement features.

[0043] Figure 4 It is a schematic diagram of video merging.

[0044] Figure 5 It is an overall schematic diagram of similarity calculation and confidence calculation in each step. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below through specific embodiments and drawings.

[0046] Figure 1 The present invention demonstrates a method for producing videos for detecting AD patients based on eye movement data. The specific steps involved are as follows:

[0047] Step 1: Collection of eye movement data from healthy subjects. A group of subjects of similar age to the patients, without visual impairment, cognitive impairment or other neurological diseases, were selected as the control group. Eye tracking equipment was used to record the eye movement data of the subjects while they watched the complete long video. During the collection of eye movement data, a quiet and evenly lit indoor environment should be selected. Control the indoor lighting to avoid screen reflections or strong direct optical fibers; ensure that the subjects are about 60 to 70 cm away from the screen to reduce visual fatigue, while keeping the line of sight level with the center of the screen; use high-precision eye tracking equipment, such as the Tobii Pro series or similar devices, to ensure accurate capture of eye movement information; accurately calibrate the eye tracking equipment for each subject to ensure that the equipment can accurately track the subject's eye movements; then play the video, select a long video with rich content, such as a documentary or TV series, and ensure that the video is played in high-definition format with a resolution of at least 1080p for a clear visual experience; finally, collect the eye movement trajectory data of the subjects while watching the video. The same playback equipment and software were used in the experiment to ensure that the video content and playback quality watched by all subjects and the equipment for collecting data were consistent.

[0048] Step 2: Data preprocessing. The collected data is divided into time windows, and the divided eye movement data is cleaned and formatted for subsequent analysis. First, two fine-grained windows of 5 seconds and 10 seconds are selected to balance the feasibility of eye movement details and data management. Matlab or Python tools are used to automatically divide the continuous eye movement trajectory data into these two windows. Make sure that each window contains complete x and y coordinate information of the fixation point. In this process, it is necessary to pay attention to maintaining the continuity of the data. Next, the data is cleaned to remove abnormal data that may be caused by equipment errors or non-fixation behavior of the subjects. At the same time, ensure that the subject data is aligned in time, that is, each time window corresponds to the data of all subjects in the same time period. If there are differences in the start time of the subject watching the video, the data needs to be synchronized by adjusting the timestamp. Finally, the processed eye movement data is converted into a table format, with each row representing the data in a time window. Save the data as a CSV or Excel file for easy import and further processing in various data analysis software. This series of steps provides a clear basis for subsequent data similarity analysis.

[0049] Step 3: Data similarity analysis. For each time window, calculate the similarity scores of the eye movement data of different subjects to identify the time windows that cause the eye movement patterns to be more similar. The similarity calculation analysis includes distance similarity and direction similarity, using Euclidean distance and cosine similarity respectively, and finally obtain the average similarity score for each time window. The specific detailed calculation steps are as follows:

[0050] First, the Euclidean distance is calculated. For the eye movement data of two subjects A and B in the same time window A i and B i , and its corresponding eye movement coordinates are (x ai ,y ai ),(x bi ,y bi ), the Euclidean distance calculation formula is: Where n is the number of data points in the time window, a i and b i are the eye movement coordinates of subjects A and B at the i-th data point, respectively.

[0051] Secondly, the cosine similarity is calculated. Considering that the Euclidean distance may have a problem of too large values, the normalization technique is used to map the original data to [0,1] as its similarity score, that is, Calculate the cosine similarity for the eye movement data A of two subjects A and B in the same time window i and B i , and its corresponding eye movement coordinates are (x ai ,y ai ),(x bi ,y bi ). Dot product calculation Cosine similarity Calculate a weighted similarity score.

[0052] Then, for each group of subjects, a weighted similarity score was calculated, taking into account the Euclidean distance and cosine similarity S(A,B) = ω1(1-D norm (A,B))+ω2C, ω1 and ω2 are the preset weights of Euclidean distance and cosine similarity respectively.

[0053] Finally, the similarity scores are summarized. For each time window, similarity scores are calculated for all possible pairs of subjects, and the average similarity score of all subjects in each time window is calculated as the similarity score of the window, denoted as S a , According to the time window division in step 2, different similarity calculations are performed.

[0054] Figure 2 The method used in step 3 to calculate the similarity of eye movement data of healthy subjects is given.

[0055] Step 4: Filter high similarity clips. According to the preset threshold, filter out video clips with higher similarity scores. That is, set the similarity score critical threshold T1. When the similarity score of the eye movement data is greater than the critical threshold, the video clip corresponding to the eye movement data is a high similarity video clip; when the similarity score of the eye movement data is less than the critical threshold, the video clip corresponding to the eye movement data is an ordinary video clip.

[0056] Step 5: Extract representative features. For the selected high-similarity video clips, extract representative eye movement features, that is, the average of the eye movement coordinates of all healthy subjects in the high-similarity video clips. Summarize the eye movement data of all healthy subjects in the high-similarity video clips, including the eye movement coordinates (x, y) of each subject. Then use point-to-point averaging to calculate the average of the eye movement coordinates of all subjects for each time point: t is a certain moment, (x it ,y it ) is the eye movement coordinate of subject i, and N is the total number of subjects. The calculated coordinates are connected to form a continuous track, which represents the average gaze pattern of all viewers in this time window. In step 2, each time window of different lengths has its "representative" eye movement features.

[0057] Step 6: Collect and analyze eye movement data of AD patients. That is, collect eye movement data of AD patients when watching the same video, and focus on comparing the similarity between eye movement data in high-similarity video clips and representative features of healthy subjects. According to the established diagnostic criteria, AD patients who meet the requirements are screened from the existing patient population as experimental subjects. In the experiment, in order to ensure the consistency of the experiment and the comparability of the data, AD patients will use the same eye tracking equipment and methods as healthy subjects to watch the same video content, and analyze their eye movement data within a time window consistent with that of healthy subjects. Within this time window, the patient's eye movement data is compared with the "representative" features of the time window, and the similarity score is calculated using the same method as in step 3, that is, for each patient AD j Calculate the Euclidean distance and cosine similarity and get the final similarity score by weighting This score comprehensively reflects the similarity between the patient's eye movement data and representative features.

[0058] Figure 3 A schematic diagram of the method for calculating the similarity between the eye movement features of AD patients and the representative eye movement features in step 6 is given.

[0059] Step 7: Screen the video clips that can significantly distinguish between AD patients and healthy subjects. According to the set similarity threshold, identify the video clips that can significantly distinguish between AD patients and healthy subjects. Further deepen the analysis, the purpose is to distinguish those video clips that can significantly distinguish between AD patients and healthy subjects in visual attention by setting a confidence threshold. Set a similarity threshold T2, which is used to divide the differences between AD patients and healthy subjects in visual attention, and count the proportion r below the threshold T2. r is the confidence of the video clip in distinguishing AD patients from healthy subjects. The confidence threshold T3 is set. The high-confidence clips screened out according to the confidence threshold T3 are the required video clips. When the confidence of a video clip is less than the confidence threshold T3, it indicates that the clip has little difference in causing eye movements between AD patients and healthy subjects. In this case, the video clip is marked as a "high-similarity video clip". This may mean that the clip has little effect on the visual attention pattern of the two groups of subjects, or that the two groups show similar behavioral characteristics when watching this part of the content. On the contrary, when the confidence of a video clip is greater than the confidence threshold T3, it means that the video clip has significant differences in causing eye movement behavior between the two groups of subjects. Such high-confidence video clips are marked as "high-difference video clips". High-difference video clips may reveal the specificity of AD patients in processing certain visual information, which is crucial to understanding the impact of the disease on visual and cognitive functions.

[0060] Figure 5 A summary diagram of the above seven steps is given.

[0061] Step 8: Classification of disease levels. The disease is classified into four levels: asymptomatic, mild, moderate and severe, based on the video clips obtained in step 7 and the similarity scores within the clips, to judge the degree of the subject's disease.

[0062] Step 9: Merge adjacent video clips. Since video windows of different lengths (such as 5 seconds and 10 seconds) are divided in step 2, the video needs to be merged separately according to the length of the video window when merging the video, and finally two groups of merged video clips will be obtained. In order to ensure that there is no cutting or omission of high-confidence video clips, the two groups of video clips are finally merged. The specific merging method is: according to the analysis in step 7, all time windows with confidence higher than the preset threshold T3 are identified. At this stage, the specific position and duration of each time window need to be recorded in detail. Then, these high-confidence time windows are checked to determine whether they are adjacent. For those continuous and adjacent high-confidence windows, they are merged into a longer continuous video clip. This process requires precise time alignment to ensure the temporal continuity and integrity of the merged clips. All merged clips will be recorded in detail, including their start and end times in the original video and the total length of the clip. At the same time, each merged clip is accurately marked in the original video to facilitate the separation of these clips from the original video.

[0063] Figure 4 A schematic diagram of video merging in step 9 is given.

[0064] The key points of the present invention include:

[0065] 1) Similarity comparison: Combining cosine similarity and Euclidean distance, considering the similarity of eye movement data direction and distance, the final similarity score is obtained by weighting to more accurately capture the similarity between the two sets of data.

[0066] 2) Representative features: The uniform eye movement features of healthy subjects are selected as the video clip standard to establish a benchmark for evaluating other individual eye movement patterns and achieve more accurate comparison and analysis.

[0067] 3) Classification: Based on the similarity scores of representative features, the patient's condition is divided into different levels, and classification is achieved by comparing the similarity between the patient's characteristics and the representative features.

[0068] 4) Video search and stitching: Split the video into small windows, gradually compare and merge adjacent segments to achieve more precise video search, improve search efficiency, and preserve the integrity of video content.

[0069] Similarity calculation alternatives in the present invention: In addition to using cosine similarity and Euclidean distance to calculate similarity, other methods may also be used to calculate similarity, such as Pearson correlation coefficient or Manhattan distance.

[0070] Another embodiment of the present invention provides a video production device for Alzheimer's disease screening, comprising:

[0071] A healthy subject eye movement data collection module, used to collect eye movement data of healthy subjects when watching videos;

[0072] A data preprocessing module, used for preprocessing the collected eye movement data, including dividing the eye movement data into time windows;

[0073] A first similarity analysis module is used to calculate similarity scores of eye movement data of different subjects for each time window;

[0074] A high-similarity video segment identification module is used to filter high-similarity video segments through a preset threshold according to the calculated similarity scores of the eye movement data;

[0075] A representative eye movement feature extraction module is used to extract representative eye movement features from the screened high-similarity video clips;

[0076] The Alzheimer's disease patient eye movement data collection module is used to collect the eye movement data of Alzheimer's disease patients when they watch the same video;

[0077] A second similarity analysis module is used to calculate the similarity score between the eye movement data collected from Alzheimer's patients when watching the same video and the representative eye movement features;

[0078] A video clip screening module, for screening video clips that can significantly distinguish Alzheimer's patients from healthy subjects through a preset threshold value according to the calculated similarity score between the eye movement data of the Alzheimer's patient and the representative eye movement features;

[0079] The video segment merging module is used to check whether the screened video segments that can significantly distinguish Alzheimer's disease patients from healthy subjects are adjacent, and merge the adjacent video segments.

[0080] The specific implementation process of each module refers to the above description of the method of the present invention.

[0081] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0082] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0083] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. A video production method for Alzheimer's disease screening, characterized in that: The following steps are involved: Collect eye movement data from healthy subjects while watching videos; Perform data preprocessing on the collected eye movement data, including dividing the eye movement data into time windows; For each time window, the similarity scores of eye movement data of different subjects are calculated; According to the calculated similarity scores of the eye movement data, high-similarity video clips are selected from the video clips corresponding to the eye movement data of the healthy subjects through a preset threshold; Extract representative eye movement features from the screened high-similarity video clips; Collecting the eye movement data of patients with Alzheimer's disease when watching the same video, and calculating the similarity score between the eye movement data and the representative eye movement features; According to the calculated similarity score between the eye movement data of the Alzheimer's patient and the representative eye movement feature, a video clip that can significantly distinguish the Alzheimer's patient from the healthy subjects is selected from the video clips corresponding to the eye movement data of the Alzheimer's patient by using a preset threshold; Check whether the screened video clips that can significantly distinguish Alzheimer's disease patients from healthy subjects are adjacent, and merge the adjacent video clips.

2. The method according to claim 1, characterized in that The calculating of similarity scores of eye movement data of different subjects includes calculating distance similarity and direction similarity, and finally obtaining an average similarity score of each time window.

3. The method according to claim 1, characterized in that The method of screening high-similarity video clips from video clips corresponding to eye movement data of healthy subjects through a preset threshold includes: setting a similarity score critical threshold T1, when the similarity score of the eye movement data is greater than the critical threshold T1, the video clip corresponding to the eye movement data is a high-similarity video clip; when the similarity score of the eye movement data is less than the critical threshold T1, the video clip corresponding to the eye movement data is an ordinary video clip.

4. The method according to claim 1, characterized in that: The representative eye movement feature is the average of the eye movement coordinates of all healthy subjects in the high similarity video clips.

5. The method according to claim 1, characterized in that The method of selecting video clips that can significantly distinguish between Alzheimer's patients and healthy subjects from video clips corresponding to the eye movement data of Alzheimer's patients by using a preset threshold comprises: A similarity threshold T2 is set, which is used to divide the differences in visual attention between Alzheimer's patients and healthy subjects, and the proportion r below the similarity threshold T2 is counted, where r is the confidence level of the video clips in distinguishing Alzheimer's patients from healthy subjects; A confidence threshold T3 is set, and segments with high confidence are screened out according to the confidence threshold T3, which are the required video segments.

6. The method according to claim 1, characterized in that The merging of adjacent video clips includes: merging the video clips respectively according to divided video windows of different lengths, and then performing a merging process.

7. The method according to claim 1, characterized in that For the screened video clips that can significantly distinguish Alzheimer's patients from normal people, they are graded according to the similarity scores within the video clips, and divided into four levels: asymptomatic, mild, moderate and severe, to judge the degree of the subject's illness.

8. A video production device for Alzheimer's disease screening, characterized in that: include: A healthy subject eye movement data collection module, used to collect eye movement data of healthy subjects when watching videos; A data preprocessing module, used for preprocessing the collected eye movement data, including dividing the eye movement data into time windows; A first similarity analysis module is used to calculate similarity scores of eye movement data of different subjects for each time window; A high-similarity video segment identification module is used to screen high-similarity video segments from video segments corresponding to the eye movement data of healthy subjects through a preset threshold according to the calculated similarity scores of the eye movement data; A representative eye movement feature extraction module is used to extract representative eye movement features from the screened high-similarity video clips; The Alzheimer's disease patient eye movement data collection module is used to collect the eye movement data of Alzheimer's disease patients when they watch the same video; A second similarity analysis module is used to calculate the similarity score between the eye movement data collected from Alzheimer's patients when watching the same video and the representative eye movement features; a video clip screening module, for screening video clips that can significantly distinguish Alzheimer's patients from healthy subjects from video clips corresponding to the eye movement data of Alzheimer's patients according to a preset threshold value based on the calculated similarity score between the eye movement data of the Alzheimer's patients and the representative eye movement features; The video segment merging module is used to check whether the screened video segments that can significantly distinguish Alzheimer's disease patients from healthy subjects are adjacent, and merge the adjacent video segments.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images