An internet-based clinical teaching management method and system

Abstract

The present application relates to the field of online teaching video processing, and more particularly to a clinical teaching management method and system based on the Internet, which determines a feature region based on the dense parameters of the moving track of the target contour center point in each video region of the sample teaching video, considers screening out effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame, combines the effective video frame segments, calculates a dynamic difference parameter according to the difference of the target contour center point coordinates in each video frame of each sample teaching video segment, screens out effective video frame segments, and subsequently compares and calculates the fitting parameters of the sample video segment and each effective video segment by preferentially comparing the target contour in the sample video with the target contour in the effective video segment after splitting and extracting the target contour of the sample video, and determines the abnormal video segment. Through the above process, only the part with strong data representation in the sample video is analyzed, and the identification accuracy of the abnormal video segment in the sample video is ensured on the premise of saving computing power.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of online teaching video processing, in particular to an internet-based clinical teaching management method and system. BACKGROUND

[0002] With the development of Internet technology and computer technology, it provides a technical basis for online teaching management, especially in the field of clinical teaching, which can realize the sharing of teaching videos, so that the user end can watch the teaching video online, especially with the progress of graphics processing technology, the features in the video can be analyzed, and the standard of related actions in the video can be analyzed, which widens the application range.

[0003] For example, Chinese patent publication No. 201910807918.X discloses a human motion contrast analysis method based on image retrieval, which obtains video stream data of human motion through a monocular camera or a mobile phone, records and saves the basic information of the target object, including name, gender, age, height, and action name, pre-processes the video data, and estimates the posture of each frame of human body in the video to obtain key point coordinates. When action contrast analysis is needed, use local sensitive hash algorithm LSH to quickly search for similar video frame key point coordinates in the image feature library, use similar key point coordinates to obtain corresponding video frames, save the video frames and combine them into a new video to compare with the target video, and evaluate the standard of related actions.

[0004] However, the prior art has the following problems,

[0005] When analyzing the features in the image in the prior art, the global contrast form is often used, which will interfere with the judgment and cause the precision to decrease due to a large number of useless features, and the algorithm occupies a large amount of power. SUMMARY

[0006] Therefore, the present application provides an internet-based clinical teaching management method and system to overcome the problem that when analyzing the features in the image in the prior art, the global contrast form is often used, which will interfere with the judgment and cause the precision to decrease due to a large number of useless features, and the algorithm occupies a large amount of power.

[0007] To achieve the above purpose, the internet-based clinical teaching management method provided by the present application comprises:

[0008] Step S1, determining a feature region based on the dense parameters of the moving track of the target contour center point in each video region of the sample teaching video;

[0009] Step S2, analyzing the sample teaching videos to screen effective video segments, including screening effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame, combining each effective frame to obtain an effective video segment, and calculating a dynamic difference parameter according to the difference between the target contour center point coordinates in each video frame of each sample teaching video segment to screen an effective video segment;

[0010] Step S3, comparing the target contour after the sample video uploaded by the user terminal is split and extracted with the target contour in the effective video segment to calculate the fitting parameter of the sample video segment and each effective video segment, and screening the maximum fitting parameter;

[0011] Step S4, determining whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment;

[0012] Step S5, sending the abnormal video segment in the sample video after being marked to the user terminal.

[0013] Further, in step S1, the dense parameter of the target contour center point moving track in each video region is calculated according to formula (1),

[0014]

[0015] In formula (1), E represents the dense parameter, L represents the length of the moving track in the video region, L0 represents the preset moving track length threshold, N represents the number of moving track intersection points in the video region, N0 represents the preset track intersection point number threshold, a represents the length weight coefficient, and β represents the number weight coefficient.

[0016] Further, in step S1, the dense parameter corresponding to each video region is compared with the preset dense degree threshold, wherein,

[0017] If the dense degree of the video region is greater than the predetermined dense degree threshold, it is determined that the video region is a feature region.

[0018] Further, in step S2, the process of screening effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame includes,

[0019] A rectangular coordinate system is established with the center of the video frame as the reference, the coordinates of the target contour center point in the video frame are obtained, and the coordinate range of the feature region is determined,

[0020] If the coordinates of the target contour center point are in the coordinate range, the video frame is screened as an effective frame.

[0021] Further, in step S2, the dynamic difference parameter is calculated according to formula (2),

[0022]

[0023] In formula (2), K represents a dynamic difference parameter, n represents the number of video frames in the sample teaching video segment, y i represents the y-axis coordinate of the center point of the target contour in the i-th video frame, y i-1 represents the y-axis coordinate of the center point of the target contour in the i-1-th video frame, x i represents the x-axis coordinate of the center point of the target contour in the i-th video frame, x i-1 represents the x-axis coordinate of the center point of the target contour in the i-1-th video frame, and i is an integer greater than 1.

[0024] Further, in the step S2, the process of screening the effective video segment according to the dynamic difference parameter of each sample teaching video segment comprises,

[0025] comparing the dynamic difference parameter of the sample teaching video segment with a preset difference parameter threshold,

[0026] if the dynamic difference parameter is greater than the difference parameter threshold, determining that the teaching video segment is an effective video segment.

[0027] Further, in the step S2, a predetermined number of effective frames are combined in sequence to obtain an effective video segment.

[0028] Further, in the step S3, after the sample video uploaded by the user terminal is split and the target contour is extracted, the target contour is compared with the target contour in the effective video segment, and the process of calculating the fitting parameter of the sample video segment and each effective video segment comprises,

[0029] comparing the movement trajectory of each feature point in the sample video segment with the movement trajectory of each feature point in the effective video segment to calculate the trajectory fitting degree, and determining the average value of the trajectory fitting degree as the fitting parameter of the sample video segment and the effective video segment, the feature point being selected in the target contour in advance.

[0030] Further, in the step S4, the process of determining whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment comprises,

[0031] comparing the maximum fitting parameter corresponding to the sample video segment with a preset fitting parameter comparison threshold,

[0032] if the maximum fitting parameter is less than the preset fitting parameter comparison threshold, determining that the sample video segment is an abnormal video segment.

[0033] The application also provides an internet-based clinical teaching management system, which comprises:

[0034] one or more processors;

[0035] a memory;

[0036] and one or more programs,

[0037] wherein the one or more programs are configured to be executed by one or more processors, and the memory includes a storage medium storing a computer program, which is executed by the processor to execute the internet-based clinical teaching management method.

[0038] Compared with the prior art, the application determines the feature region based on the dense parameters of the moving track of the target contour center point in each video region of the sample teaching video, considers screening the effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame, combines each effective frame to obtain the effective video segment, calculates the dynamic difference parameter according to the difference of the target contour center point coordinates in each video frame of each sample teaching video segment, screens the effective video segment, and then compares and calculates the fitting parameter of the sample video segment and each effective video segment by preferentially comparing the target contour in the sample video uploaded by the user with the target contour in the effective video segment after the target contour in the sample video is split and extracted, and determines the abnormal video segment. Through the above process, only the part with strong data representation in the sample video is analyzed, and the identification accuracy of the abnormal video segment in the sample video is ensured under the premise of saving the computing power.

[0039] Especially, the application identifies the feature region in the sample teaching video. In actual situations, the key actions of the sample teaching video related to clinical teaching may be concentrated in a certain region in the sample teaching video, for example, in the related teaching video of the clinical puncture technology, the emphasis is on the operation method, that is, the region with dense moving contour of the hand in the sample teaching video. Further, the application determines the feature region according to the dense parameters of the moving track of the target contour center point in each video region of the sample teaching video, screens the feature region with strong data representation, provides data support for the subsequent screening of the effective video segment and the determination of the abnormal video segment, and further ensures the identification accuracy of the abnormal video segment in the sample video under the premise of saving the computing power.

[0040] Especially, the application considers multiple dimensions, screens effective video segments, in actual conditions, the image feature data in the feature region of the sample teaching video is strong in representation, represents the dense operation region, and the target contour is a pre-labeled key contour, for example, in the related teaching video of clinical puncture technology, the hand contour is labeled as the target contour, by considering the relative position relationship between the target contour center point and the feature region, effective frames are screened to form effective video segments, the effective video segment data is strong in representation, in the non-effective video frame, the target contour is outside the feature region, may perform invalid action or be in a static state, when performing image analysis, interference is easily caused, and the application considers the dynamic characteristics of the target contour outside the feature region in the sample teaching video segment, screens part of the teaching video segment, avoids omission of part of the features, and thus guarantees the recognition accuracy of the abnormal video segment in the sample video under the premise of saving the computing power.

[0041] Especially, the application compares the target contour after splitting and extracting the sample video with the target contour in the effective video segment, calculates the fitting parameters, determines whether the abnormal video segment exists according to the maximum fitting parameters corresponding to the sample video segment, determines the abnormal video segment only by using the effective video frame, the abnormal video segment data is strong in representation, and the recognition accuracy of the abnormal video segment in the sample video is guaranteed under the premise of saving the computing power. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The figure is a step diagram of the internet-based clinical teaching management method of the embodiment of the application.

[0043] Figure 2 The figure is a feature region determination logic diagram of the embodiment of the application.

[0044] Figure 3 The figure is an effective video segment determination logic diagram of the embodiment of the application.

[0045] Figure 4 The figure is an abnormal video segment determination logic diagram of the embodiment of the application. DETAILED DESCRIPTION

[0046] In order to make the objects and advantages of the application clearer, the application is further described below in combination with embodiments; it should be understood that the specific embodiments described herein are only used to explain the application, and do not limit the protection scope of the application.

[0047] The preferred embodiments of the application are described below with reference to the drawings. Those skilled in the art should understand that the embodiments are only used to explain the technical principles of the application, and do not limit the protection scope of the application.

[0048] Please refer to Figures 1 to 4 as shown, Figure 1 The figure is a step diagram of the internet-based clinical teaching management method of the embodiment of the application.Figure 2 A feature region determination logic diagram for an embodiment of the invention, Figure 3 An effective video segment determination logic diagram for an embodiment of the invention, Figure 4 An abnormal video segment determination logic diagram for an embodiment of the invention, the internet-based clinical teaching management method of the invention comprises:

[0049] Step S1, determining a feature region based on the dense parameters of the moving track of the target contour center point in each video region of the sample teaching video;

[0050] Step S2, analyzing the sample teaching video to screen effective video segments, comprising screening effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame, combining the effective frames to obtain effective video segments, and calculating a dynamic difference parameter according to the difference of the target contour center point coordinates in each video frame of each sample teaching video segment to screen effective video segments;

[0051] Step S3, comparing the target contour extracted from the sample video uploaded by the user end with the target contour in the effective video segment to calculate the fitting parameters of the sample video segment and each effective video segment, and screening the maximum fitting parameter;

[0052] Step S4, determining whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment;

[0053] Step S5, calibrating the abnormal video segment in the sample video and sending it to the user end.

[0054] Specifically, the present invention does not limit the acquisition method of the sample teaching video, in this embodiment, the clinical teaching video of the demonstrator performing the predetermined action can be collected as the sample teaching video.

[0055] Specifically, the present invention does not limit the method of determining the target contour in the image, preferably, the image algorithm or model capable of recognizing the corresponding target contour can be pre-trained and imported into the computer to realize the corresponding function through the computer, which is prior art and will not be repeated.

[0056] Specifically, it can be understood that the target contour is a key feature, which can be different in different application environments, for example, the clinical puncture technology related teaching video, the clinical puncture technology focuses on the operation method, therefore, the hand contour can be determined as the target contour by the person skilled in the art, which will not be repeated.

[0057] Specifically, the present invention does not limit the method of determining the contour center, for example, for the determination of the contour center in the image, the OpenCV image processing library can be used, the center coordinates of the contour can be obtained by calculating the moment of the contour, which is prior art and will not be repeated.

[0058] Specifically, the present application does not limit the acquisition method of the sample video. It can be understood that the sample video can be obtained by collecting the video of the student performing the predetermined action. Of course, in the present embodiment, for the convenience of processing, the shooting angle of the sample video is as same as the shooting angle of the sample teaching video as far as possible, and the focal length of the sample video is as same as the focal length of the sample teaching video as far as possible.

[0059] Specifically, in the step S1, the dense parameters of the moving track of the target contour center point in each video region are calculated according to the formula (1),

[0060]

[0061] In the formula (1), E represents the dense parameter, L represents the length of the moving track in the video region, L0 represents the preset moving track length threshold, N represents the number of the intersection points of the moving track in the video region, N0 represents the preset track intersection point number threshold, a represents the length weight coefficient, and β represents the number weight coefficient. In the present embodiment, a = 0.6, and β = 0.4.

[0062] The moving track length threshold and the track intersection point number threshold are obtained by pre-computation, wherein,

[0063] The average value ΔL of the moving track length and the average value ΔN of the track intersection point number of each video region of the sample teaching video are obtained, L0 = g1 x ΔL is set, N0 = ΔN x g2 is set, g1 represents the first precision coefficient, g2 represents the second precision coefficient, 1.25 < g1 < 1.5, and 1.5 < g2 < 2.

[0064] Specifically, in the step S1, the dense parameters corresponding to each of the video regions are compared with the preset dense degree threshold.

[0065] If the dense degree of the video region is greater than the predetermined dense degree threshold, the video region is determined as the feature region.

[0066] In the present embodiment, the dense degree threshold is selected in the interval [1.3, 1.6].

[0067] The application identifies a feature region in a sample teaching video. In actual situations, key actions of a sample teaching video related to clinical teaching may be concentrated in a certain region of the sample teaching video. For example, in a teaching video related to clinical puncture technology, the focus is on the operation method, i.e. the region in which the hand contour moves intensively in the sample teaching video. Then, the application determines the feature region according to the intensive parameter of the moving track of the target contour center point in each video region of the sample teaching video, screens out the feature region with strong data representation, and provides data support for subsequent screening of effective video segments and determination of abnormal video segments, thereby ensuring the identification accuracy of the abnormal video segment in the sample video under the premise of saving computing power.

[0068] Specifically, in step S2, the process of screening out effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame includes,

[0069] A rectangular coordinate system is established with the center of the video frame as the reference, the coordinates of the target contour center point in the video frame are obtained, and the coordinate range of the feature region is determined,

[0070] If the coordinates of the target contour center point are in the coordinate range, the video frame is screened out as an effective frame.

[0071] Specifically, in step S2, the dynamic difference parameter is calculated according to formula (2),

[0072]

[0073] In formula (2), K represents the dynamic difference parameter, n represents the number of video frames in the sample teaching video segment, y i represents the y-axis coordinate of the target contour center point in the i-th video frame, y i-1 represents the y-axis coordinate of the target contour center point in the i-1-th video frame, x i represents the x-axis coordinate of the target contour center point in the i-th video frame, x i-1 represents the x-axis coordinate of the target contour center point in the i-1-th video frame, and i is an integer greater than 1.

[0074] Specifically, in step S2, the process of screening out effective video segments according to the dynamic difference parameters of each sample teaching video segment includes,

[0075] The dynamic difference parameter of the sample teaching video segment is compared with a preset difference parameter threshold,

[0076] If the dynamic difference parameter is greater than the difference parameter threshold, the teaching video segment is determined to be an effective video segment.

[0077] Specifically, in this embodiment, the difference parameter threshold K0 is obtained by pre-computation, wherein,

[0078] A video segment when the object moves at a speed of 5 cm / s is collected, and a dynamic difference parameter of the video segment is calculated, denoted as Ke, K0 is set as Ke x γ, and γ represents an expansion coefficient, 1.15 < γ < 1.2.

[0079] Specifically, in the step S2, a predetermined number of effective frames are combined in sequence to obtain an effective video segment. It can be understood that, for continuous video frames, a video segment is obtained after combination, which will not be repeated here.

[0080] In the embodiment, taking 60 frames of video as an example, the number of video frames contained in 2 seconds of video is determined as the predetermined number.

[0081] The application considers multiple dimensions to screen out effective video segments. In actual situations, image feature data in a feature region of a sample teaching video is highly representative, representing a dense operation region, and a target contour is a key contour that is pre-labeled. For example, in a related teaching video of clinical puncture technology, a hand contour is labeled as a target contour. By considering the relative position relationship between a center point of the target contour and the feature region, effective frames are screened out to form an effective video segment. The effective video segment data is highly representative. In non-effective video frames, the target contour is outside the feature region, and may perform invalid actions or be in a static state, which can easily cause interference during image analysis. Moreover, the application considers the dynamic characteristics of the target contour outside the feature region in a sample teaching video segment, screens out part of the teaching video segments, avoids omission of part of the features, and further ensures the recognition accuracy of abnormal video segments in the sample video under the premise of saving computing power.

[0082] Specifically, in the step S3, after the target contour is extracted from the sample video uploaded by the user end and compared with the target contour in the effective video segment, the process of calculating the fitting parameter of the sample video segment and each effective video segment includes,

[0083] The moving track of each feature point in the sample video segment is compared with the moving track of each feature point in the effective video segment to calculate a track fitting degree, and an average value of the track fitting degree is determined as the fitting parameter of the sample video segment and the effective video segment. The feature point is selected in the target contour in advance.

[0084] It can be understood that the feature point can be selected as multiple or a single one, as long as it has certain representativeness for the action. In the embodiment, taking a related teaching video of clinical puncture technology as an example, the hand contour is determined as the target contour, and the finger joint can be determined as the feature point.

[0085] In some possible embodiments, taking a heart-lung compression related teaching video as an example, the key of heart-lung compression lies in the pressing position and fluctuation rule of the hand, and the hand can be determined as the target contour, and the center of the hand can be determined as the feature point, which will not be repeated here.

[0086] Specifically, the present application does not limit the specific way of solving the trajectory fitting degree, for example, OpenCV provides many functions for image processing and analysis, the functions in OpenCV can be used to detect and track feature points, and provides some curve fitting methods such as least squares method and RANSAC algorithm.

[0087] The functions and toolboxes provided by MATLAB can also be used for image processing and analysis, for example, the CurveFitting Toolbox in MATLAB can be used to fit the moving trajectory of multiple feature points in the image and calculate the fitting degree.

[0088] Specifically, in the step S4, the process of determining whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment includes,

[0089] Comparing the maximum fitting parameter corresponding to the sample video segment with a preset fitting parameter comparison threshold,

[0090] If the maximum fitting parameter is less than the preset fitting parameter comparison threshold, the sample video segment is determined as an abnormal video segment.

[0091] Specifically, the present application does not limit the setting method of the fitting parameter comparison threshold, in the embodiment, a sample teaching video can be constructed in advance, a number of sample videos conforming to the action specification are manually screened, the fitting parameters of the sample teaching video and each sample video are solved, and the average value of the fitting parameters is set as the fitting parameter comparison threshold.

[0092] The present application compares the target contour extracted after splitting the sample video with the target contour in the effective video segment, calculates the fitting parameter, determines whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment, and only determines the abnormal video segment with the effective video frame, so that the abnormal video segment data is characteristic and the recognition accuracy of the abnormal video segment in the sample video is ensured on the premise of saving computing power.

[0093] The present application also provides an internet-based clinical teaching management system, which comprises:

[0094] One or more processors;

[0095] Memory;

[0096] And one or more programs,

[0097] Wherein, the one or more programs are configured to be executed by one or more processors, the memory includes a storage medium storing a computer program, and the computer program is executed by the processor to execute the internet-based clinical teaching management method.

[0098] So far, the technical solutions of the present application have been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical solutions after these changes or replacements will all fall within the protection scope of the present application.

Claims

1. An Internet-based clinical teaching management method, characterized by, The method comprises the following steps: Step S1, determining a feature region based on the dense parameters of the moving track of the target contour center point in each video region of a sample teaching video, wherein the target contour needs to be pre-labeled; Step S2, analyzing the sample teaching video to screen effective video segments, including screening effective frames in the feature region according to the relative position relationship between the target contour center point and the feature region in each video frame, combining the effective frames to obtain effective video segments, and calculating a dynamic difference parameter according to the difference of the target contour center point coordinates in each video frame of each sample teaching video segment to screen effective video segments outside the feature region; Step S3, comparing the target contour of the sample video uploaded by the user terminal after being split and extracted with the target contour in the effective video segments to calculate the fitting parameters of the sample video segment and each effective video segment, and screening the maximum fitting parameter; Step S4, determining whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment; Step S5, sending the labeled abnormal video segment in the sample video to the user terminal; In step S1, the dense parameters of the moving track of the target contour center point in each video region are calculated according to formula (1), In formula (1), E represents the dense parameter, L represents the length of the moving track in the video region, L0 represents a preset moving track length threshold, N represents the number of moving track intersection points in the video region, N0 represents a preset track intersection point number threshold, a represents a length weight coefficient, and β represents a number weight coefficient; In step S1, the dense parameters corresponding to each video region are compared with a preset density threshold, wherein, If the density of the video region is greater than the predetermined density threshold, it is determined that the video region is a feature region.

2. The Internet-based clinical teaching management method of claim 1, wherein, In step S2, the process of screening effective frames according to the relative position relationship between the target contour center point and the feature region in each video frame comprises: establishing a rectangular coordinate system with the center of the video frame as the reference, obtaining the coordinates of the target contour center point in the video frame, and determining the coordinate range of the feature region, if the coordinates of the target contour center point are in the coordinate range, the video frame is screened as an effective frame.

3. The Internet-based clinical teaching management method of claim 1, wherein, In step S2, the dynamic difference parameter is calculated according to formula (2), In formula (2), K represents a dynamic difference parameter, n represents a number of video frames in a sample teaching video segment, y i represents a y-axis coordinate of a center point of a target contour in an i-th video frame, y i-1 represents a y-axis coordinate of a center point of a target contour in an i-1-th video frame, x i represents an x-axis coordinate of a center point of a target contour in an i-th video frame, x i-1 represents an x-axis coordinate of a center point of a target contour in an i-1-th video frame, and i is an integer greater than 1.

4. The Internet-based clinical teaching management method of claim 3, wherein, In step S2, the process of screening effective video segments according to the dynamic difference parameters of each sample teaching video segment comprises: comparing the dynamic difference parameters of the sample teaching video segment with a preset difference parameter threshold, if the dynamic difference parameter is greater than the difference parameter threshold, it is determined that the teaching video segment is an effective video segment.

5. The Internet-based clinical teaching management method of claim 1, wherein, In step S2, a predetermined number of effective frames are combined in sequence to obtain an effective video segment.

6. The Internet-based clinical teaching management method of claim 1, wherein, In step S3, the process of comparing the target contour of the sample video uploaded by the user terminal after being split and extracted with the target contour in the effective video segments to calculate the fitting parameters of the sample video segment and each effective video segment comprises: The moving track of each feature point in the sample video segment is compared with the moving track of each feature point in the effective video segment to calculate a track fitting degree, and an average value of the track fitting degrees is determined as a fitting parameter of the sample video segment and the effective video segment, and the feature points are selected in advance in a target contour.

7. The Internet-based clinical teaching management method of claim 1, wherein, In the step S4, the process of determining whether there is an abnormal video segment according to the maximum fitting parameter corresponding to the sample video segment comprises: The maximum fitting parameter corresponding to the sample video segment is compared with a preset fitting parameter comparison threshold value, If the maximum fitting parameter is less than the preset fitting parameter comparison threshold value, it is determined that the sample video segment is an abnormal video segment.

8. A system for applying the Internet-based clinical teaching management method according to any one of claims 1 to 7, characterized in that, Comprise: One or more processors; Memory; And one or more programs, Wherein, the one or more programs are configured to be executed by one or more processors, the memory includes a storage medium storing a computer program, and the computer program is executed by the processor to execute an internet-based clinical teaching management method.

Images