Coaxial playing and comparing method based on video file

Abstract

The invention belongs to the field of video playing, and provides a coaxial playing and comparing method based on a video file, which comprises the following steps: S1, video preprocessing; s2, carrying out coaxial alignment treatment; s3, synchronously playing the video; s4, area selection and dynamic comparison; s5, performing real-time comparison display; s6, dynamic analysis and intelligent assistance; through frame rate synchronous processing and a coaxial alignment algorithm, different videos can play the same frame number at the same time point, and key frames are accurately aligned, so that the video comparison efficiency is greatly improved; a user does not need to manually adjust the playing progress and the alignment mode of the video, so that time and energy are saved; according to the method, multiple feature extraction and matching algorithms are adopted, the feature matching accuracy of the video is improved, an affine transformation matrix is calculated through multiple sets of feature point pairs, an average value is taken, the accuracy of the transformation matrix is further improved, and pictures of different videos can be overlapped and compared more accurately.

Description

Technical Field

[0001] The invention belongs to the field of video playback, in particular to a coaxial playback and comparison method based on video files. Background Art

[0002] With the continuous development of camera technology, the resolution and frame rate of image sensors are constantly improving, which enables the acquisition of clearer and smoother video images; this makes the video data obtained in different scenarios more and more abundant, and the demand for video comparison and analysis is also increasing. For example, in the field of security monitoring, the widespread use of high-definition cameras has generated a large amount of video data, and it is necessary to compare and analyze surveillance videos in different time periods and locations to ensure safety.

[0003] Directors, editors, etc. need to compare different versions of shooting materials, videos processed with special effects, etc., in order to select the best editing scheme and special effects. In the post-production of movies, it is necessary to compare video versions with different editing orders and video effects after adding different special effects in order to produce high-quality film and television works; the practical process is often recorded by multiple cameras, and researchers need to compare and analyze experimental videos under different perspectives and different parameters to verify the accuracy of the experimental results.

[0004] Traditional video comparison methods may simply play two videos side by side, lacking effective visualization methods, making it difficult for users to intuitively find the differences between the videos.

[0005] Therefore, those skilled in the art have proposed a coaxial playback and comparison method based on video files to solve the problems raised by the background technology. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a coaxial playback and comparison method based on video files to solve the problem that the video comparison method in the prior art may simply play two videos side by side, lacks effective visualization means, and makes it difficult for users to intuitively find the differences between the videos.

[0007] A coaxial playback and comparison method based on video files comprises the following steps:

[0008] S1, video preprocessing, extracting the frame rate of the video file to be compared, and performing frame rate synchronization processing so that each video file has the same playback frame rate to ensure synchronous playback;

[0009] S2, coaxial alignment processing, by selecting key frames based on video content changes and time intervals, using SI FT to extract feature points, using the nearest neighbor distance ratio for feature matching and using RANSAC to remove false matches, using the least squares method to calculate the affine transformation matrix and averaging multiple groups to improve accuracy, and finally overlapping the adjusted different video images coaxially and using alpha blending and auxiliary lines to display through the terminal;

[0010] S3, video synchronous playback, by assigning timestamps and frame numbers to videos, using timers to achieve frame synchronization, and synchronously adjusting the playback status of all videos when the terminal adjusts the playback speed and drags the progress bar;

[0011] S4, area selection and dynamic comparison, the terminal selects the comparison area, analyzes the brightness, color, and texture characteristics of the area, detects changes frame by frame and marks them, and realizes dynamic comparison;

[0012] S5, real-time comparison display, using pseudo-color and other means to highlight the difference area, providing multiple comparison modes and generating comparison reports, intuitively viewing and in-depth analysis through the terminal;

[0013] S6, dynamic analysis and intelligent assistance, uses convolutional neural network algorithm to automatically detect changed areas, pops up a prompt window when major changes occur, and statistically analyzes the changes to help users understand the patterns of video changes.

[0014] Preferably, the S1 comprises the following steps:

[0015] S101, video input and parameter extraction: select two or more video files to be compared through the file selection interface on the terminal, read the metadata of the video files, extract parameters such as frame rate, resolution, duration, etc., and display them on the interface;

[0016] S102, frame rate unification: determine a target frame rate, and change the frame rate to the target frame rate through a frame interpolation algorithm and a frame sampling algorithm; for example, select the highest frame rate in the input video as the target frame rate;

[0017] For videos with a frame rate lower than the target frame rate, a frame interpolation algorithm is used to analyze the pixel motion between adjacent frames, predict the content of the intermediate frames, and insert new frames to increase the frame rate to the target frame rate.

[0018] For videos with a frame rate higher than the target frame rate, a frame sampling algorithm is used to reduce the frame rate to the target frame rate.

[0019] Preferably, S2 comprises the following steps:

[0020] S201, key frame selection, using a key frame selection method based on video content changes to calculate feature changes of video frames, including color histogram differences and edge changes; when the feature changes exceed a set threshold, the frame is determined as a key frame;

[0021] S202, feature point extraction, using SI FT (ScaleInvariant Feature Transform) to detect key frames, detect feature points of key frames, and calculate the descriptor of each feature point for subsequent feature matching;

[0022] S203, feature matching, using a feature matching method based on the nearest neighbor distance ratio, for each feature point in a key frame of one video, searching for the nearest neighbor and the next nearest neighbor feature point in the key frame of another video; if the distance ratio between the nearest neighbor feature point and the next nearest neighbor feature point is less than a set threshold, the two feature points are considered to be matched; the RANSAC algorithm is used to remove false matches and improve the accuracy of matching;

[0023] S204, transform matrix calculation, based on the matched feature point pairs, using the least square method to calculate the affine transform matrix; transforming the key frame of one video to a position corresponding to another video; using multiple sets of matched feature point pairs for calculation and taking the average value to improve the accuracy of the transform matrix;

[0024] S205, coaxial overlap, overlapping the images of different videos adjusted by the transformation matrix on the same window; using the alpha blending algorithm, blending the pixels of the two videos according to the transparency parameter; in order to observe through the terminal, auxiliary lines, marks, etc. can be displayed on the overlapping video images to highlight the aligned position.

[0025] Preferably, S3 comprises the following steps:

[0026] S301, frame synchronization, the terminal assigns a timestamp and frame number to the video, and ensures that the same number of frames are played at the same time point according to the correction progress during playback. A timer is used, and when a video is played to a specific frame, a callback is triggered to check and adjust the progress of other videos;

[0027] S302, synchronous control, the terminal adjusts the playback speed, adjusts the playback speed of all videos at the same time according to the speed ratio, modifies the frame rate and playback time interval; the terminal drags the progress bar, calculates the frame number according to the total number of frames and the progress bar percentage, and adjusts the playback position of all videos.

[0028] Preferably, S4 comprises the following steps:

[0029] S401, dynamic area selection, providing tools such as mouse dragging box selection and polygon drawing, after the terminal selects the comparison area, the system records the coordinate information, and only analyzes the comparison area later;

[0030] S402, regional feature analysis, extracting brightness, color, and texture features from the selected region, using mean, variance, and histogram to describe brightness and color features, and using gray-level co-occurrence matrix and local binary pattern to extract texture features, and combining them into a regional feature vector for frame-by-frame analysis;

[0031] S403, dynamic change detection, during playback, the feature vector of the selected region is calculated frame by frame and compared with the previous frame, the difference is measured using the Euclidean distance, and the changed region is marked if it exceeds a threshold.

[0032] Preferably, S5 comprises the following steps:

[0033] S501, difference visualization, highlighting the selected area difference with pseudo color and brightness change, such as red for large difference area and yellow or green for small difference area, integrating the difference information with the original picture to enhance the contrast effect;

[0034] S502, multi-mode display, providing single mode (overlap display of difference areas), dual window mode (side by side comparison), and layered mode (adjustment of transparency comparison);

[0035] S503, output comparison data, generate a comparison report to record the frame number, position and change of the difference area, which can be exported as text, table or image file, and provide a data interface for other tools to analyze.

[0036] Preferably, S6 comprises the following steps:

[0037] S601, automatic change detection, based on machine learning algorithms such as convolutional neural networks, first collect annotated video data to train the model to identify key change areas, and then input the video to be compared in actual applications, and the model outputs the location and probability of the change area;

[0038] S602, intelligent prompt, when a major change is detected, a prompt window will pop up automatically to display the location, size and extent of the changed area, and the user can click to quickly locate and view it;

[0039] S603, change statistical analysis, statistics of total change frame number, change frequency and other indicators, drawing change curves to show change trends and distribution, so that users can understand video stability and change patterns.

[0040] Preferably, the affine transformation matrix in S204 is calculated by the least squares method, and the formula is:

[0041] x′=a·x+b·y+e;

[0042] y′=c·x+d·y+f;

[0043] Where a, b, c, d represent scaling and rotation coefficients, and e, f represent translation coefficients.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention uses frame rate synchronization processing and coaxial alignment algorithms to enable different videos to play the same number of frames at the same time point and accurately align key frames, thereby greatly improving the efficiency of video comparison; users do not need to manually adjust the playback progress and alignment of the video, saving time and energy; a variety of feature extraction and matching algorithms are used to improve the feature matching accuracy of the video, and the affine transformation matrix is ​​calculated by multiple groups of feature point pairs and the average is taken to further improve the accuracy of the transformation matrix, so that the pictures of different videos can be overlapped and compared more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the steps of this method;

[0047] Figure 2 Schematic diagram of the steps for coaxial alignment processing. DETAILED DESCRIPTION

[0048] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0049] Embodiment 1: As shown in the attached Figure 1 To Attachment Figure 2 As shown: The present invention provides a coaxial playback and comparison method based on video files, comprising the following steps:

[0050] S1, video preprocessing, extracting the frame rate of the video file to be compared, and performing frame rate synchronization processing so that each video file has the same playback frame rate to ensure synchronous playback;

[0051] S101, video input and parameter extraction: select two or more video files to be compared through the file selection interface on the terminal, read the metadata of the video files, extract parameters such as frame rate, resolution, duration, etc., and display them on the interface;

[0052] S102, frame rate unification: determine a target frame rate, and change the frame rate to the target frame rate through a frame interpolation algorithm and a frame sampling algorithm; for example, select the highest frame rate in the input video as the target frame rate;

[0053] For videos with a frame rate lower than the target frame rate, a frame interpolation algorithm is used to analyze the pixel motion between adjacent frames, predict the content of the intermediate frames, and insert new frames to increase the frame rate to the target frame rate.

[0054] For videos with a frame rate higher than the target frame rate, a frame sampling algorithm is used to reduce the frame rate to the target frame rate.

[0055] S2, coaxial alignment processing, by selecting key frames based on video content changes and time intervals, using SI FT to extract feature points, using the nearest neighbor distance ratio for feature matching and using RANSAC to remove false matches, using the least squares method to calculate the affine transformation matrix and averaging multiple groups to improve accuracy, and finally overlapping the adjusted different video images coaxially and using alpha blending and auxiliary lines to display through the terminal;

[0056] S201, key frame selection, using a key frame selection method based on video content changes to calculate feature changes of video frames, including color histogram differences and edge changes; when the feature changes exceed a set threshold, the frame is determined as a key frame;

[0057] S202, feature point extraction, using SIFT (Scale-Invariant Feature Transform) to detect key frames, detect feature points of key frames, and calculate the descriptor of each feature point for subsequent feature matching;

[0058] S203, feature matching, using a feature matching method based on the nearest neighbor distance ratio, for each feature point in a key frame of one video, searching for the nearest neighbor and the next nearest neighbor feature point in the key frame of another video; if the distance ratio between the nearest neighbor feature point and the next nearest neighbor feature point is less than a set threshold, the two feature points are considered to be matched; the RANSAC algorithm is used to remove false matches and improve the accuracy of matching;

[0059] S204, transform matrix calculation, based on the matched feature point pairs, using the least square method to calculate the affine transform matrix; transforming the key frame of one video to a position corresponding to another video; using multiple sets of matched feature point pairs for calculation and taking the average value to improve the accuracy of the transform matrix;

[0060] The affine transformation matrix is ​​calculated by the least squares method, and the formula is:

[0061] x′=a·x+b·y+e;

[0062] y′=c·x+d·y+f;

[0063] Where a, b, c, d represent scaling and rotation coefficients, and e and f represent translation coefficients;

[0064] S205, coaxial overlap, overlapping the images of different videos adjusted by the transformation matrix on the same window; using the alpha blending algorithm, mixing the pixels of the two videos according to the transparency parameter; in order to observe through the terminal, auxiliary lines, marks, etc. can be displayed on the overlapping video images to highlight the aligned position.

[0065] S3, video synchronous playback, by assigning timestamps and frame numbers to videos, using timers to achieve frame synchronization, and synchronously adjusting the playback status of all videos when the terminal adjusts the playback speed and drags the progress bar;

[0066] S301, frame synchronization, the terminal assigns a timestamp and frame number to the video, and ensures that the same number of frames are played at the same time point according to the correction progress during playback. A timer is used, and when a video is played to a specific frame, a callback is triggered to check and adjust the progress of other videos;

[0067] S302, synchronous control, the terminal adjusts the playback speed, adjusts the playback speed of all videos at the same time according to the speed ratio, modifies the frame rate and playback time interval; the terminal drags the progress bar, calculates the frame number according to the total number of frames and the progress bar percentage, and adjusts the playback position of all videos;

[0068] S4, area selection and dynamic comparison, the terminal selects the comparison area, analyzes the brightness, color, and texture characteristics of the area, detects changes frame by frame and marks them, and realizes dynamic comparison;

[0069] S401, dynamic area selection, providing tools such as mouse dragging box selection and polygon drawing, after the terminal selects the comparison area, the system records the coordinate information, and only analyzes the comparison area later;

[0070] S402, regional feature analysis, extracting brightness, color, and texture features from the selected region, using mean, variance, and histogram to describe brightness and color features, and using gray-level co-occurrence matrix and local binary pattern to extract texture features, and combining them into a regional feature vector for frame-by-frame analysis;

[0071] S403, dynamic change detection, during playback, the feature vector of the selected region is calculated frame by frame and compared with the previous frame, the difference is measured using the Euclidean distance, and the changed region is marked if it exceeds a threshold.

[0072] S5, real-time comparison display, using pseudo-color and other means to highlight the difference area, providing multiple comparison modes and generating comparison reports, intuitively viewing and in-depth analysis through the terminal;

[0073] S501, difference visualization, highlighting the selected area difference with pseudo color and brightness change, such as red for large difference area and yellow or green for small difference area, integrating the difference information with the original picture to enhance the contrast effect;

[0074] S502, multi-mode display, providing single mode (overlap display of difference areas), dual window mode (side by side comparison), and layered mode (adjustment of transparency comparison);

[0075] S503, output comparison data, generate a comparison report to record the frame number, position and change of the difference area, which can be exported as text, table or image file, and provide a data interface for other tools to analyze.

[0076] S6, dynamic analysis and intelligent assistance, using convolutional neural network algorithm to automatically detect the changed area, pop up a prompt window when there is a major change, and statistically analyze the changes to help users understand the law of video changes;

[0077] S601, automatic change detection, based on machine learning algorithms such as convolutional neural networks, first collect annotated video data to train the model to identify key change areas, and then input the video to be compared in actual applications, and the model outputs the location and probability of the change area;

[0078] S602, intelligent prompt, when a major change is detected, a prompt window will pop up automatically to display the location, size and extent of the changed area, and the user can click to quickly locate and view it;

[0079] S603, change statistical analysis, statistics of total change frame number, change frequency and other indicators, drawing change curves to show change trends and distribution, so that users can understand video stability and change patterns.

[0080] Embodiment 2: S1. Video preprocessing:

[0081] Video input and parameter extraction: The user selects two surveillance video files A and B with different resolutions but related content through the file selection dialog box on the software interface. The system reads the metadata of the two video files, extracts the frame rate of video A as 25fps, resolution as 1920×1080, and duration as 30 minutes; the frame rate of video B as 30fps, resolution as 1280×720, and duration as 25 minutes, and displays these parameters on the interface.

[0082] Frame rate unification: The target frame rate is set to 30fps. For video A, since its frame rate is lower than the target frame rate, a frame interpolation algorithm is used. The pixel motion between adjacent frames is analyzed. Assuming that the pixel change trend of the intermediate frame is calculated by the optical flow method, a new frame is inserted to increase the frame rate of video A to 30fps.

[0083] S2. Coaxial alignment processing:

[0084] Key frame selection: Calculate the color histogram difference and edge change of each frame of video A and B. For the color histogram difference, the Bhattacharyya distance is used for calculation. When the Bhattacharyya distance is greater than the set threshold of 0.5, the color change of the frame is considered to be large. For edge change, the Canny edge detection algorithm is used to calculate the change in the number of edge pixels of adjacent frames. When the change exceeds a certain number, the edge change of the frame is considered to be large. When the color histogram difference and edge change meet the conditions at the same time, the frame is determined as a key frame.

[0085] Feature point extraction: Use the SIFT algorithm to detect feature points on the key frames of videos A and B. For each key frame, multiple feature points are detected and a 128-dimensional descriptor is calculated for each feature point.

[0086] Feature matching: For each feature point in the key frame of video A, find the nearest neighbor and the next nearest neighbor feature point in the key frame of video B. If the distance ratio between the nearest neighbor feature point and the next nearest neighbor feature point is less than 0.8, the two feature points are considered to be matched. Then use the RANSAC algorithm to remove false matches. Assume that after the RANSAC algorithm is processed, the false match rate is reduced from the original 20% to 5%.

[0087] Transformation matrix calculation: Based on the matched feature point pairs, the affine transformation matrix is ​​calculated using the least squares method.

[0088] According to the formula x′=a·x+b·y+e;

[0089] y′=c·x+d·y+f;

[0090] Where a, b, c, d represent scaling and rotation coefficients, and e and f represent translation coefficients; the initial affine transformation matrix is ​​calculated through a large number of matching feature point pairs, and then multiple sets of matching feature point pairs are used for calculation and the average is taken to improve the accuracy of the transformation matrix. Assume that after multiple calculations and averaging, the accuracy of the transformation matrix is ​​improved by 10%.

[0091] Coaxial Overlap: Overlap the images of video A and video B after transformation matrix adjustment on the same window. Use alpha blending algorithm, set the transparency parameter to 0.5, and blend the pixels of the two videos. Display red auxiliary lines on the overlapping video images to highlight the alignment position.

[0092] S3, video synchronous playback:

[0093] Frame synchronization: Assign timestamps to videos A and B, starting from 0, and increase the timestamp by 1 / 30 for each frame played (because the target frame rate is 30fps). During playback, correct the progress based on the timestamp to ensure that the same number of frames are played at the same time point. Use a timer to check the playback progress of the video every 100 milliseconds. If one video plays to a specific frame, trigger a callback function to check the playback progress of the other video and make adjustments.

[0094] Synchronous control: When the user adjusts the playback speed to 1.5 times the original speed, the system adjusts the playback speed of videos A and B at the same time according to the speed ratio. By modifying the frame rate, the frame rate is adjusted from 30fps to 45fps to achieve faster playback speed. When the user drags the progress bar, assuming that the total length of the progress bar is 100, the current position is 50, and the total number of video frames is 9000, the current frame number is calculated as 4500 according to the formula, and videos A and B are adjusted to this frame number for playback.

[0095] S4. Region selection and dynamic comparison

[0096] Dynamic area selection: Provides a mouse dragging box selection tool, allowing users to select a specific area in the overlapping coaxial videos, such as the face area of ​​a person. The system records the coordinate information of the area and only analyzes the comparison area later.

[0097] Regional feature analysis: Extract brightness, color, and texture features from the selected facial region. Use mean and variance to describe brightness features, and calculate the mean brightness of the pixels in the region as 120 and the variance as 20. Use histogram to describe color features, and get the color distribution histogram of the region. Use gray-level co-occurrence matrix to extract texture features, and calculate parameters such as texture contrast and correlation. Combine these features into a regional feature vector for frame-by-frame analysis.

[0098] Dynamic change detection: During playback, the feature vector of the selected region is calculated frame by frame and compared with the feature vector of the previous frame. The difference is measured using the Euclidean distance. When the Euclidean distance is greater than 10, it is considered that the region has undergone a significant change and is marked in red.

[0099] S5, real-time comparison display:

[0100] Difference visualization: Use pseudo-color to highlight the differences in the selected area. When the difference is large, it is displayed in red; when the difference is small, it is displayed in yellow. The difference information is merged with the original picture to enhance the contrast effect.

[0101] Multi-mode display: Provides a single mode, that is, overlapping display of difference areas. Users can directly see the difference between video A and B at the same position.

[0102] Comparison data output: Generate a comparison report to record the frame number, position and changes of the difference area. For example, in the 2000th frame, the facial area of ​​the character has a significant change, and the position is from coordinates (100,200) to (200,300). The comparison report can be exported as a text file for further analysis and processing through the terminal.

[0103] S6, Dynamic Analysis and Intelligent Assistance

[0104] Automatic change detection: Based on the convolutional neural network algorithm, a large amount of annotated surveillance video data is collected, including areas of change such as the appearance of people and the movement of objects, as well as areas of no change. This data is used to train a machine learning model so that it can identify key areas of change in the video. In actual applications, the videos A and B to be compared are input into the trained machine learning model, and the model outputs the location and probability of possible change areas in each frame.

[0105] Smart Prompt: When the system detects a major change, such as a person disappearing suddenly or a new object appearing, a prompt window will automatically pop up to display the location, size, and extent of the changed area. Users can quickly locate the changed area for viewing by clicking on the prompt window.

[0106] Change statistics analysis: Statistics on the total number of changed frames, change frequency and other indicators of video A and B. Draw a change curve to show the trend and distribution of changes in the video. For example, in a 30-minute video, the total number of changed frames is 500 frames, and the change frequency is 16.7 frames per minute. Users can understand the stability and change rules of the video through the change curve.

[0107] As can be seen from the above, this method uses frame rate synchronization processing and coaxial alignment algorithm to enable different videos to play the same number of frames at the same time point and accurately align key frames, greatly improving the efficiency of video comparison; users do not need to manually adjust the playback progress and alignment of the video, saving time and energy.

[0108] By adopting a variety of feature extraction and matching algorithms, such as SIFT, nearest neighbor distance ratio, and RANSAC, the accuracy of video feature matching is improved; at the same time, the affine transformation matrix is ​​calculated by multiple sets of feature point pairs and the average is taken to further improve the accuracy of the transformation matrix, so that the images of different videos can be overlapped and compared more accurately;

[0109] It provides a variety of comparison modes, such as single mode, dual-window mode, layered mode, etc., to meet the comparison needs of users in different scenarios. At the same time, it uses pseudo-color to highlight the difference area, generate comparison reports and other functions to enhance the visualization effect of the comparison, and intuitively understand the differences between videos through the terminal.

[0110] The use of convolutional neural network algorithms for automatic change detection can quickly and accurately identify key change areas in the video, and through intelligent prompts and change statistical analysis functions, it provides users with a more convenient video analysis tool. Users can understand major changes in the video in a timely manner, improving the efficiency and accuracy of video analysis.

[0111] Embodiment three:

[0112] S1. Video preprocessing:

[0113] Video input and parameter extraction: The user selects two medical imaging video files C and D shot at different angles. The system extracts video C with a frame rate of 15fps, a resolution of 800×600, and a duration of 10 minutes; video D with a frame rate of 20fps, a resolution of 1024×768, and a duration of 12 minutes, and displays the parameters on the interface.

[0114] Frame rate unification: The target frame rate is set to 20fps. For video C, a frame interpolation algorithm is used to insert new frames between adjacent frames through linear interpolation to increase the frame rate to 20fps. For video D, a frame sampling algorithm is used to extract one frame every other frame to reduce the frame rate to 20fps.

[0115] S2. Coaxial alignment processing:

[0116] Key frame selection: Calculate the color histogram difference and edge change of each frame of video C and D. For the color histogram difference, the chi-square distance calculation is used. When the chi-square distance is greater than the set threshold of 0.4, the color change of the frame is considered to be large. For edge change, the Sobel edge detection algorithm is used to calculate the change in the edge pixel intensity of adjacent frames. When the change exceeds a certain threshold, the edge change of the frame is considered to be large. When the color histogram difference and edge change meet the conditions at the same time, the frame is determined as a key frame.

[0117] Feature point extraction: Use the SIFT algorithm to detect feature points on the key frames of videos C and D. Detect multiple feature points in the key frames and calculate a descriptor for each feature point.

[0118] Feature matching: For each feature point in the key frame of video C, find the nearest and second nearest neighbor feature points in the key frame of video D. If the distance ratio between the nearest neighbor feature point and the second nearest neighbor feature point is less than 0.7, the two feature points are considered to be matched. The RANSAC algorithm is used to remove false matches, and after processing, the false match rate is reduced to 8%.

[0119] Transformation matrix calculation: Based on the matched feature point pairs, the affine transformation matrix is ​​calculated using the least squares method.

[0120] According to the formula: x′=a·x+b·y+e;

[0121] y′=c·x+d·y+f;

[0122] The initial affine transformation matrix is ​​calculated through a large number of matching feature point pairs; then multiple sets of matching feature point pairs are used for calculation and the average is taken to improve the accuracy of the transformation matrix. Assume that after multiple calculations and averaging, the accuracy of the transformation matrix is ​​improved by 8%.

[0123] Coaxial Overlap: Overlap the images of videos C and D after transformation matrix adjustment on the same window. Use alpha blending algorithm, set the transparency parameter to 0.6, and blend the pixels of the two videos. Display blue auxiliary lines on the overlapping video images to highlight the alignment position.

[0124] S3, video synchronous playback:

[0125] Frame synchronization: Assign frame numbers to videos C and D, starting from 1 and increasing in sequence. During playback, correct the progress according to the frame numbers to ensure that the same number of frames are played at the same time point. Use a timer to check the playback progress of the video every 80 milliseconds. When one video plays to a specific frame, trigger the callback function to check the playback progress of the other video and make adjustments.

[0126] Synchronous control: When the user adjusts the playback speed to 0.8 times the original speed, the system adjusts the playback speed of videos C and D at the same time according to the speed ratio. By modifying the playback time interval, the original playback time interval per frame is adjusted from 50 milliseconds to 62.5 milliseconds to slow down the playback speed. When the user drags the progress bar, assuming that the total length of the progress bar is 120, the current position is 70, and the total number of video frames is 12,000, the current frame number is calculated according to the formula to be 7000, and videos C and D are adjusted to this frame number for playback.

[0127] S4, area selection and dynamic comparison:

[0128] Dynamic area selection: Provides polygon drawing tools, allowing users to draw a specific area in the overlapping coaxial videos, such as the area of ​​a lesion. The system records the coordinate information of the area and only analyzes the comparison area later.

[0129] Regional feature analysis: Extract brightness, color, and texture features from the selected lesion area. Use mean, variance, and histogram to describe brightness and color features. Calculate the mean brightness of the pixels in the area as 80 and the variance as 15 to obtain the color distribution histogram. Use local binary patterns to extract texture features and calculate parameters such as texture uniformity. Combine these features into a regional feature vector for frame-by-frame analysis.

[0130] Dynamic change detection: During playback, the feature vector of the selected region is calculated frame by frame and compared with the feature vector of the previous frame. The difference is measured using the Euclidean distance. When the Euclidean distance is greater than 8, the region is considered to have undergone a significant change and is marked in green.

[0131] S5, real-time comparison display:

[0132] Difference visualization: For the differences in the selected area, use brightness changes to highlight them. When the difference is large, increase the brightness of the area; when the difference is small, reduce the brightness of the area. Merge the difference information with the original image to enhance the contrast effect.

[0133] Multi-mode display: Provides dual-window mode to display videos C and D side by side for comparison, allowing you to observe the overall picture of the two videos at the same time through the terminal.

[0134] Comparison data output: Generate a comparison report to record the frame number, position and changes of the difference area. For example, in the 5000th frame, the lesion area has a significant change, and the position is from coordinates (300,400) to (400,500); the comparison report can be exported as a table file for data analysis through the terminal.

[0135] S6. Dynamic analysis and intelligent assistance:

[0136] Automatic change detection: Based on the recurrent neural network algorithm, a large amount of annotated medical imaging video data is collected, including the changed areas and unchanged areas of the lesion. These data are used to train the machine learning model so that it can identify the key changed areas in the video. In practical applications, the videos C and D to be compared are input into the trained machine learning model, and the model outputs the location and probability of the possible changed areas in each frame. The recurrent neural network algorithm uses the long short-term memory network (LSTM), which is suitable for processing sequence data. In medical imaging videos, continuous frames can be regarded as time series. LSTM can effectively learn long-term dependencies, which is very helpful for capturing the changing trend of the lesion over time. It can remember past information and update the state based on the current input and previous memory, so as to better predict the situation of the next frame.

[0137] The machine learning model uses an LSTM-based sequence classification model to build a neural network model with an LSTM layer as the core, taking continuous video frames as the input sequence; the output of the model can be the category of changes in the lesion area (such as changed area, no changed area), and through training on a large amount of labeled data, the model can learn the feature change patterns between different frames, so that it can accurately identify key change areas;

[0138] Encoder-decoder structure: Use the encoder to encode the input video sequence into a vector representation of a fixed length, and then use the decoder to gradually generate the output sequence to predict the changes in the lesion area; LSTM can be combined in the encoder and decoder to better handle the encoding and decoding process of sequence data; introduce the attention mechanism based on the recurrent neural network to allow the model to pay more attention to the key frames or key areas in the video; for medical imaging videos, the attention mechanism can help the model focus on the lesion site and improve the recognition accuracy of the changed area.

[0139] Smart Prompt: When the system detects a significant change, such as a significant change in the size or shape of the lesion, a prompt window will automatically pop up to display the location, size, and extent of the changed area. Users can quickly locate the changed area for viewing by clicking on the prompt window.

[0140] Change statistics analysis: Statistics on the total number of changed frames, change frequency and other indicators of videos C and D. Draw a change curve to show the trend and distribution of changes in the video. For example, in a 12-minute video, the total number of changed frames is 400 frames, and the change frequency is 33.3 frames per minute. Users can understand the stability and change rules of the video through the change curve.

[0141] From the above, it can be seen that this method can accurately compare medical images taken at different angles through precise coaxial alignment processing and regional feature analysis, helping doctors better observe the changes in the lesion site and improve the accuracy of diagnosis.

[0142] The video synchronization function can adjust the playback speed and progress bar according to the user's needs, making it easier for doctors to carefully observe the details of medical images in different situations. At the same time, multiple comparison modes and visualization effects allow doctors to choose the most suitable comparison method according to their own habits.

[0143] The use of machine learning algorithms for automatic change detection and intelligent prompts can promptly detect major changes in medical images and provide doctors with important diagnostic clues. The statistical analysis function of changes can help doctors understand the changing trends and patterns of lesions and provide a reference for formulating treatment plans.

[0144] The embodiments of the present invention are provided for the purpose of illustration and description. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A coaxial playback and comparison method based on video files, characterized in that: The following steps are involved: S1, extracting the frame rates of the video files to be compared, and performing frame rate synchronization processing so that each video file has the same playback frame rate; S2, by selecting key frames based on video content changes and time intervals, using SIFT to extract feature points, using the nearest neighbor distance ratio for feature matching and using RANSAC to remove false matches, using the least squares method to calculate the affine transformation matrix and averaging multiple groups to improve accuracy, and finally overlapping the adjusted different video images coaxially and using alpha blending and auxiliary lines to display them through the terminal; S3, by assigning timestamps and frame numbers to the videos, using a timer to achieve frame synchronization, and synchronously adjusting the playback status of all videos when the terminal adjusts the playback speed and drags the progress bar; S4. Select a comparison area through the terminal, analyze the brightness, color, and texture characteristics of the area, detect changes frame by frame, and mark them; S5. Use pseudo-color and other means to highlight the difference area, provide multiple comparison modes and generate comparison reports, and intuitively view and deeply analyze through the terminal; S6. Use the convolutional neural network algorithm to automatically detect the changed area, pop up a prompt window when there is a major change, and statistically analyze the changes to help understand the law of video changes.

2. A coaxial playback and comparison method based on video files as claimed in claim 1, characterized in that: The S1 comprises the following steps: S101, video input and parameter extraction: selecting two or more video files to be compared through a file selection interface on a terminal, reading metadata parameters of the video files, wherein the metadata parameters include extraction frame rate, resolution, and duration, and displaying them on a terminal interface; S102, frame rate unification: determine a target frame rate, and change the frame rate to the target frame rate through a frame interpolation algorithm and a frame sampling algorithm.

3. A coaxial playback and comparison method based on video files as claimed in claim 2, characterized in that: The S2 comprises the following steps: S201, key frame selection, using a key frame selection method based on video content changes to calculate feature changes of video frames, including color histogram differences and edge changes; when the feature changes exceed a set threshold, the frame is determined as a key frame; S202, feature point extraction, using SIFT to detect key frames, detect feature points of key frames, and calculate the descriptor of each feature point for subsequent feature matching; S203, feature matching, using a feature matching method based on nearest neighbor distance ratio to perform feature matching; S204, transform matrix calculation, based on the matched feature point pairs, using the least square method to calculate the affine transform matrix; transforming a key frame of one video to a position corresponding to another video; using multiple sets of matched feature point pairs for calculation and taking an average value; S205, coaxial overlap, overlapping the images of different videos adjusted by the transformation matrix on the same window; mixing the pixels of the two videos according to the transparency parameter; and displaying auxiliary lines and marks on the overlapped video images through terminal observation.

4. A coaxial playback and comparison method based on video files as claimed in claim 3, characterized in that: The S3 comprises the following steps: S301, frame synchronization, the terminal assigns a timestamp and frame number to the video, and ensures that the same number of frames are played at the same time point according to the correction progress during playback. A timer is used, and when a video is played to a specific frame, a callback is triggered to check and adjust the progress of other videos; S302, synchronous control, the terminal adjusts the playback speed, adjusts the playback speed of all videos at the same time according to the speed ratio, modifies the frame rate and playback time interval; the terminal drags the progress bar, calculates the frame number according to the total number of frames and the progress bar percentage, and adjusts the playback position of all videos.

5. A coaxial playback and comparison method based on video files as claimed in claim 4, characterized in that: The S4 comprises the following steps: S401, dynamic area selection, providing mouse dragging box selection and polygon drawing tools, after the terminal selects the comparison area, the system records the coordinate information, and only analyzes the comparison area later; S402, regional feature analysis, extracting brightness, color, and texture features from the selected region, using mean, variance, and histogram to describe brightness and color features, and using gray-level co-occurrence matrix and local binary pattern to extract texture features, and combining them into a regional feature vector for frame-by-frame analysis; S403, dynamic change detection, calculate the feature vector of the selected area frame by frame during playback and compare it with the previous frame, use Euclidean distance to measure the difference, and mark the changed area if it exceeds the threshold.

6. A coaxial playback and comparison method based on video files as claimed in claim 5, characterized in that: The S5 comprises the following steps: S501, difference visualization, highlighting the selected area difference with pseudo color and brightness change, and integrating the difference information with the original picture to enhance the contrast effect; S502, multi-mode display, providing single mode, dual window mode, and layered mode; S503: Output the comparison data and generate a comparison report to record the frame number, position and change of the difference area.

7. A coaxial playback and comparison method based on video files as claimed in claim 6, characterized in that: The S6 comprises the following steps: S601, automatic change detection, based on machine learning algorithms such as convolutional neural networks, first collect annotated video data to train the model to identify key change areas, and then input the video to be compared in actual applications, and the model outputs the location and probability of the change area; S602, intelligent prompt, when a major change is detected, a prompt window will pop up automatically to display the location, size and extent of the changed area; S603, statistical analysis of changes, statistics of total number of changed frames, change frequency and other indicators, drawing a change curve to show the change trend and distribution.

8. A coaxial playback and comparison method based on video files as claimed in claim 3, characterized in that: The affine transformation matrix in S204 is calculated by the least square method, and the formula is: x′=a·x+b·y+e; y′=c·x+d·y+f; Where a, b, c, d represent scaling and rotation coefficients, and e, f represent translation coefficients.

Images