A virtual simulation method for capturing anesthesiologist motion

Abstract

The present invention relates to the field of virtual simulation technology, and in particular to a virtual simulation anesthesiologist motion capture method, comprising: obtaining a frame image of each action video of the anesthesiologist, obtaining a local feature descriptor of the key point according to the grayscale value of the pixel point within the preset window range of the key point in the frame image, judging whether different key points in the frame image are in the same topological structure, and then obtaining a number of topological structures of the frame image, obtaining a topological structure set according to the grayscale value of the key point in the topological structure of different frame images and the local feature descriptor of the key point, obtaining the missing important key points in the topological structure according to the centroid coordinates and the number of the key points of the topological structure in the topological structure set, and then completing the establishment of the virtual simulation model of the anesthesiologist. The present invention effectively modifies the SIFT algorithm according to the importance of different key points, so that the missing key points can be effectively detected, and the capture of the anesthesiologist's actions is completed.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual simulation, and in particular to a method for capturing anesthesiologist's motion in virtual simulation. Background Art

[0002] Virtual simulation is a technology that simulates real-life scenes or processes, making users feel like they are in a virtual world through a computer-generated environment. Virtual simulation technology is increasingly being used in areas such as medical training and anesthesia surgery planning. These systems need to accurately capture the real-time movements of anesthesiologists to ensure the realism of the simulation environment and the effectiveness of training. However, existing methods for capturing anesthesiologist motion often have problems such as low accuracy, high cost, and complex equipment.

[0003] The motion capture of anesthesiologists mainly uses the anesthesia video of the anesthesiologists during their work to detect the corresponding key points. In the prior art, in order to reduce the cost of motion capture, the SIFT algorithm is generally used to detect the key points of the anesthesia video of the anesthesiologists, and then the detected key points are used to establish the virtual character model of the anesthesiologist for virtual simulation technology. However, when using the SIFT algorithm for key point detection, its basic logic is to use local extreme points at different scales as key point detection. However, because the anesthesia movements of the anesthesiologists are too complicated, the key points detected under large movements are often not accurate enough, and the motion capture is not precise enough, resulting in large motion deviations when establishing virtual reality character models. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a virtual simulation anesthesiologist motion capture method.

[0005] A virtual simulation anesthesiologist motion capture method of the present invention adopts the following technical solution:

[0006] An embodiment of the present invention provides a method for capturing anesthesiologist's motion in a virtual simulation, the method comprising the following steps:

[0007] Collect multiple action videos of the anesthesiologist and obtain several frame images of each action video;

[0008] According to the frame images, a number of key points of each frame image are obtained, and according to the gray values ​​of the pixels within the preset window range of the key points in the frame images, a local feature descriptor of each key point in each frame image is obtained, and according to the local feature descriptors of different key points in the frame images and the distances between different key points, a number of key points in the same topological structure in each frame image are obtained;

[0009] According to the coordinates of the key points in the same topological structure in the frame image, several topological structures of each frame image are obtained, according to the gray values ​​of the key points in the topological structures of different frame images and the local feature descriptors of the key points, the best matching topological structure of each topological structure in each frame image is obtained, and according to the best matching topological structure of each topological structure in the frame image, several topological structure sets are obtained;

[0010] According to the centroid coordinates of the key points of the topological structures in the topological structure set, several important key points of the topological structures in each topological structure set are obtained. According to the number of key points of adjacent topological structures in the topological structure set, the missing key points of each topological structure in each topological structure set are obtained. According to the important key points of the topological structures in the topological structure set and the missing key points of each topological structure in the topological structure set, the important missing key points in each topological structure are obtained. According to several key points of each frame image of each action video and the important missing key points in each topological structure, a virtual simulation model of the anesthesiologist is established.

[0011] Furthermore, the step of obtaining a plurality of key points of each frame image according to the frame image includes the following specific steps:

[0012] Any action video is recorded as the target action video, and any frame image of the target action video is recorded as the target frame image. The target frame image is subjected to image pyramid downsampling and Gaussian blur processing to obtain several Gaussian blurred images of the target frame image. The SIFT algorithm is used to perform key point detection on each Gaussian blurred image to obtain several key points of the target frame image.

[0013] Furthermore, the method of obtaining the local feature descriptor of each key point in each frame image according to the grayscale value of the pixel point within the preset window range of the key point in the frame image includes the following specific steps:

[0014]

[0015] In the formula, G n,m,j is the gray value of the jth pixel in the window range of ω×ω with the mth key point as the center in the nth frame image, ω is a preset first value, and J is the number of pixels in the window range of ω×ω with the mth key point as the center in the nth frame image; max(G n,m ) is the maximum gray value of the pixel in the window range of ω×ω with the mth key point as the center in the nth frame image; min(G n,m ) is the minimum gray value of the pixel in the window range of ω×ω with the mth key point as the center in the nth frame image; || is the absolute value, OG n,m It is the local feature descriptor of the mth key point in the nth frame image.

[0016] Furthermore, the method of obtaining a plurality of key points in the same topological structure in each frame image according to the local feature descriptors of different key points in the frame image and the distances between the different key points includes the following specific steps:

[0017]

[0018] In the formula, OG n,m is the local feature descriptor of the mth key point in the nth frame image, OG n,c is the local feature descriptor of the cth key point in the nth frame image, || is the absolute value, d m,c is the Euclidean distance between the mth key point and the cth key point, Norm[] is the linear normalization function, YZ n,m,c The probability that the mth key point and the cth key point in the nth frame image are in the same topological structure;

[0019] A first threshold is preset, denoted as th1, if YZ n,m,c >th1, the mth key point and the cth key point in the nth frame image are in the same topological structure, if YZ n,m,c ≤th1, the mth key point and the cth key point in the nth frame image are not in the same topological structure.

[0020] Furthermore, the method of obtaining the best matching topological structure of each topological structure in each frame image according to the grayscale values ​​of the key points in the topological structures of different frame images and the local feature descriptors of the key points includes the following specific steps:

[0021]

[0022] In the formula, AG n,q The local feature descriptor sequence obtained by arranging the local feature descriptors of all key points in the qth topological structure in the nth frame image from small to large is recorded as the first sequence, AG n+1,f The local feature descriptor sequence obtained by arranging the local feature descriptors of all key points in the f-th topological structure in the n+1-th frame image from small to large is recorded as the second sequence, cos(AG n,q AG n+1,f ) is the cosine similarity between the first sequence and the second sequence; δ n,q is the coefficient of variation of the grayscale values ​​of all key points in the qth topological structure in the nth frame image, δ n+1,f is the coefficient of variation of the grayscale values ​​of all key points in the f-th topological structure in the n+1-th frame image; || is the absolute value, P n,q,n+1,f is the matching degree between the qth topological structure in the nth frame image and the fth topological structure in the n+1th frame image;

[0023] The matching degree between the qth topological structure in the nth frame image and each topological structure in the n+1th frame image is obtained, and the topological structure corresponding to the maximum matching degree in the n+1th frame image is taken as the best matching topological structure of the qth topological structure in the nth frame image.

[0024] Furthermore, the step of obtaining a plurality of topological structure sets according to the best matching topological structure of each topological structure in the frame image includes the following specific steps:

[0025] Construct an empty topological structure set, record the qth topological structure in the first frame image as the first topological structure, and use the first topological structure as the first element in the topological structure set. Among all the topological structures in the second frame image, obtain the best matching topological structure of the qth topological structure in the first frame image, record it as the second topological structure, and use the second topological structure as the second element in the topological structure set. Among all the topological structures in the third frame image, obtain the best matching topological structure of the second topological structure, record it as the third topological structure, and use the third topological structure as the third element in the topological structure set, and so on, until the best matching topological structure is obtained among all the topological structures in the last frame image and added to the topological structure set, and finally obtain the topological structure set corresponding to the qth topological structure in the first frame image.

[0026] Furthermore, the method of obtaining several important key points of the topological structures in each topological structure set according to the centroid coordinates of the key points of the topological structures in the topological structure set includes the following specific steps:

[0027]

[0028] In the formula, X q,g G is the centroid coordinates of all key points in the g-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image. q is the number of topological structures in the topological structure set corresponding to the qth topological structure in the first frame image, is the centroid coordinates of all key points in all topological structures in the topological structure set corresponding to the qth topological structure in the first frame image, X i,h is the centroid coordinates of all key points in the hth topological structure in the remaining topological structure set except the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image; X i is the centroid coordinates of all key points in the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image, ||||2 is the two-norm, IP q,iis the initial importance degree of the key points of the $i$-th topological structure in the set of topological structures corresponding to the $q$-th topological structure in the first frame image;

[0029] Perform linear normalization on the initial importance degrees of the key points of all topological structures in the set of topological structures corresponding to all topological structures in the first frame image, and use the obtained result as the importance degree of the key points of each topological structure in each set of topological structures;

[0030] Based on the importance degrees of the key points of each topological structure in each set of topological structures, obtain several important key points of the topological structures in each set of topological structures.

[0031] Further, the step of obtaining several important key points of the topological structures in each set of topological structures according to the importance degrees of the key points of each topological structure in each set of topological structures includes the following specific steps:

[0032] Preset a second threshold, denoted as th2, and denote the importance degree of the key points of the $i$-th topological structure in the set of topological structures corresponding to the $q$-th topological structure in the first frame image as IP q ′ ,i , if IP q ′ ,i ≥th2, the key points of the $i$-th topological structure in the set of topological structures corresponding to the $q$-th topological structure in the first frame image are important key points, if IP q ′ ,i <th2, the key points of the $i$-th topological structure in the set of topological structures corresponding to the $q$-th topological structure in the first frame image are not important key points.

[0033] Further, the step of obtaining the missing key points of each topological structure in each set of topological structures according to the number of key points of adjacent topological structures in the set of topological structures includes the following specific steps:

[0034] For the set of topological structures corresponding to the $q$-th topological structure in the first frame image, denote the number of key points of the $i$-th topological structure in the set of topological structures as T1, and denote the number of key points of the $(i - 1)$-th topological structure in the set of topological structures as T2. If T1 < T2, the key points of the $i$-th topological structure in the set of topological structures are missing key points. If T1 ≥ T2, the key points of the $i$-th topological structure in the set of topological structures are not missing key points.

[0035] Further, the step of obtaining the missing important key points of each topological structure according to the important key points of the topological structures in the set of topological structures and the missing key points of each topological structure in the set of topological structures includes the following specific steps:

[0036] A topological structure in which any key point is a missing key point and an important key point is recorded as a target topological structure. In the topological structure set to which the target topological structure belongs, the previous topological structure of the target topological structure is obtained, which is recorded as an initial topological structure. The local feature descriptor of each key point in the initial topological structure is compared with the local feature descriptor of each important key point in the target topological structure. The two key points corresponding to the minimum absolute value of the difference between the two local feature descriptors are taken as the key points at the same position. Several key points that are not at the same position in the initial topological structure are obtained and recorded as initial key points.

[0037] For the frame image belonging to the target topological structure, the extreme value selection threshold in the SIFT algorithm is traversed from small to large, and several key points corresponding to the frame image under each selection threshold are obtained. The local feature descriptor corresponding to each key point under each selection threshold is obtained and compared with the local feature descriptor of the initial key point. For any initial key point, the key point with the smallest absolute difference with the local feature descriptor of the initial key point is obtained, which is used as the important key point corresponding to the initial key point in the target topological structure, and is also recorded as the important key point missing in the target topological structure; for each initial key point, the corresponding important key point in the target topological structure is obtained to obtain several important key points missing in the target topological structure.

[0038] The beneficial effect of the technical solution of the present invention is as follows: after obtaining the action video of the anesthesiologist, the present invention obtains a frame image of the action video, obtains key points using the SIFT algorithm, and judges whether different key points in each frame image are in the same topological structure based on the local feature descriptors of different key points in the frame image and the distances between different key points. Considering that the anesthesiologist may lose important key points during the action process, several topological structures of each frame image are obtained based on the coordinates of the key points in the same topological structure in the frame image, and then several topological structure sets are obtained based on the best matching topological structure of each topological structure in the frame image. By analyzing the centroid coordinates and the number of key points of the topological structure in the topological structure set, the missing important key points are judged, and the extreme value selection threshold in the SIFT algorithm is traversed to obtain the missing key points in the topological structure. Therefore, according to the several key points of each frame image of each action video and the important key points missing in each topological structure, a virtual simulation model of the anesthesiologist is established to complete the capture of the detailed actions of the anesthesiologist. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of the steps of a virtual simulation anesthesiologist motion capture method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the virtual simulation anesthesiologist motion capture method proposed by the present invention, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0042] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0043] The specific scheme of the virtual simulation anesthesiologist motion capture method provided by the present invention is described in detail below with reference to the accompanying drawings.

[0044] See also Figure 1 , which shows a flowchart of a method for capturing anesthesiologist's motion in virtual simulation provided by an embodiment of the present invention, the method comprising the following steps:

[0045] Step S001: collect multiple action videos of an anesthesiologist and obtain several frame images of each action video.

[0046] It should be noted that the purpose of this embodiment is to establish a virtual simulation model of the anesthesiologist by performing motion capture based on key points acquired from the video data collected by the anesthesiologist during the anesthesia process. Therefore, before performing key point detection on the anesthesia video of the anesthesiologist, it is first necessary to obtain the anesthesiologist's action video for subsequent motion capture.

[0047] Specifically, high-definition cameras are installed at multiple angles to capture multiple action videos of the anesthesiologist.

[0048] It should be noted that the SIFT algorithm is an algorithm for image processing. Before detecting key points, the action video needs to be preprocessed so that it meets the detection basis of the SIFT algorithm.

[0049] Specifically, all frame images of each action video are obtained; it should be noted that obtaining frame images of video data is an existing method and will not be described in detail in this embodiment.

[0050] At this point, all frame images of each action video are obtained.

[0051] Step S002, obtaining a number of key points of each frame image according to the frame image, obtaining a local feature descriptor of each key point in each frame image according to the grayscale value of the pixel points within a preset window range of the key point in the frame image, and judging whether different key points in each frame image are in the same topological structure according to the local feature descriptors of different key points in the frame image and the distances between different key points.

[0052] It should be noted that when using the SIFT algorithm to detect key points, the detection threshold of each frame of video data is the same, but some specific key points in different frames are affected by angles and lighting, resulting in the lack of conditions for key point detection and cannot be effectively detected. Therefore, key point detection in different frames of video data may not necessarily achieve better results. In order to make the detection of key points in frame images more accurate, this embodiment uses the SIFT algorithm to detect key points in each frame of the action video, and then obtains the key points that cannot be matched and the importance of the key points through matching the key points and analyzing the corresponding topological structure, and effectively corrects the SIFT algorithm according to the importance of different key points, and adjusts its specific threshold for key point detection, so that the missing key points can be effectively detected.

[0053] Specifically, several key points of each frame image are obtained according to the frame image, as follows:

[0054] Any action video is recorded as a target action video, and any frame image of the target action video is recorded as a target frame image. The target frame image is subjected to image pyramid downsampling and Gaussian blur processing to obtain several Gaussian blurred images of the target frame image. The SIFT algorithm is used to perform key point detection on each Gaussian blurred image to obtain several key points of the target frame image. It should be noted that performing image pyramid downsampling and Gaussian blur processing to obtain several Gaussian blurred images and key point detection is an existing method of the SIFT algorithm, which will not be repeated in this embodiment.

[0055] It should be noted that the key points in each frame image may be related to the local anesthesia action of the anesthesiologist, so when capturing the anesthesiologist's action, the topological structure should be established for different key points first, and the topological structure corresponding to each key point should be obtained, so as to facilitate the subsequent timing analysis of different key points; the establishment of the topological results of different key points in this embodiment is to use the local feature descriptors of different key points and the spatial positions of different key points to establish the topological structure.

[0056] Specifically, according to the grayscale values ​​of the pixels within the preset window range of the key points in the frame image, the local feature descriptor of each key point in each frame image is obtained, as follows:

[0057]

[0058] In the formula, G n,m,j is the grayscale value of the jth pixel in the window range of ω×ω with the mth key point as the center in the nth frame image, ω is a preset first value, and this embodiment is described as ω=5, J is the number of pixels in the window range of ω×ω with the mth key point as the center in the nth frame image, where the number of pixels includes the center point, i.e., the mth key point, max(G n,m ) is the maximum gray value of the pixel in the window range of ω×ω with the mth key point as the center in the nth frame image, min(G n,m ) is the minimum grayscale value of the pixel in the window range of ω×ω with the mth key point as the center in the nth frame image, || is the absolute value, OG n,m It is the local feature descriptor of the mth key point in the nth frame image.

[0059] It should be noted that if the key point in the frame image is close to the boundary of the frame image, the window range of ω×ω centered on the key point will exceed the boundary of the frame image. At this time, the present embodiment uses the quadratic linear interpolation method to fill the part exceeding the boundary; the local feature descriptor of the key point is used to describe a local feature of the mth key point in the nth frame image. The present embodiment describes the trend of the gray value change of the mth key point compared with the global gray value by quantifying the value. The actual meaning of the descriptor is: to judge the gray value distribution range of the mth key point compared with the global pixel points. The reason for judging the global distribution range is to facilitate the calculation of the judgment conditions of different key points in the future.

[0060] It should be noted that the local feature descriptors of the key points in the nth frame image are calculated. Since the key points used to represent the same local action of the anesthesiologist should have local feature descriptors that are close in spatial position and have similar values ​​due to the amplitude of the action, based on this logic, this embodiment analyzes different key points in the frame image to obtain the judgment conditions for whether different key points are in the same topological structure.

[0061] Specifically, according to the local feature descriptors of different key points in the frame image and the distances between different key points, the possibility of different key points in each frame image being in the same topological structure is obtained, as follows:

[0062]

[0063] In the formula, OG n,m is the local feature descriptor of the mth key point in the nth frame image, OG n,c is the local feature descriptor of the cth key point in the nth frame image, where the cth key point and the mth key point are not the same key point, || is the absolute value, d m,c is the Euclidean distance between the mth key point and the cth key point, Norm[] is a linear normalization function, and the normalized object is the distance between any two key points in the nth frame image. YZ n,m,c It is the probability that the m-th key point and the c-th key point in the n-th frame image are in the same topological structure.

[0064] It should be noted that when the mth key point and the cth key point in the nth frame image are in the same topological structure, that is, the key points of the same local action of the anesthesiologist, their corresponding distances should be relatively close, and the local feature descriptors should be relatively similar. Therefore, this embodiment calculates the difference value and the Euclidean distance of the local feature descriptors of the two key points to obtain the judgment condition of whether the two key points are in the same topological structure, that is, the possibility of being in the same topological structure, YZ n,m,c The larger it is, the more similar the local feature descriptors of the two key points are and the closer the distance is, the greater the possibility that they are the same topological structure, and vice versa.

[0065] Furthermore, according to the possibility that different key points in the frame image are in the same topological structure, several key points in each frame image that are in the same topological structure are obtained, as follows:

[0066] A first threshold is preset, denoted as th1. In this embodiment, th1=0.9 is used for description. If YZ n,m,c>th1, the mth key point and the cth key point in the nth frame image are in the same topological structure, if YZ n,m,c ≤th1, the mth key point and the cth key point in the nth frame image are not in the same topological structure.

[0067] At this point, it is determined whether different key points in each frame image are in the same topological structure.

[0068] Step S003, according to the coordinates of the key points in the same topological structure in the frame image, several topological structures of each frame image are obtained, according to the grayscale values ​​of the key points in the topological structures of different frame images and the local feature descriptors of the key points, several topological structure sets are obtained, and according to the centroid coordinates of the key points of the topological structures in the topological structure sets, the importance of each topological point in each topological structure set is obtained.

[0069] It should be noted that the above obtains whether different key points in each frame image are in the same topological structure. By obtaining all key points in the same topological structure in each frame image, several topological structures corresponding to each frame image can be obtained.

[0070] Specifically, according to the coordinates of the key points in the same topological structure in the frame image, several topological structures of each frame image are obtained, as follows:

[0071] Obtain the coordinates of all key points in the same topological structure in the nth frame image, and obtain several topological structures of the nth frame image according to the coordinates of all key points in the same topological structure in the nth frame image; it should be noted that obtaining several topological structures of the nth frame image according to the coordinates of all key points in the same topological structure in the nth frame image is an existing method and will not be repeated in this embodiment.

[0072] It should be noted that not all corresponding key points in the topological structures have the same importance. Therefore, after some key points in different topological structures are missing, the necessity of different key points when searching for the missing key points is different. Therefore, this embodiment first matches different topologies on continuous frame images to obtain a temporal topological structure set, and then analyzes the importance corresponding to each temporal topological structure set, and searches for the missing key points in the temporal topological structure according to their importance. Under normal circumstances, each topological structure can always find the corresponding topological structure in the next frame image, and the number of key points in the topological structure is the same and one-to-one. However, as the angle of the anesthesiologist's action changes, the lighting is affected, and some key points in the topological structure are missing.

[0073] Specifically, according to the grayscale values ​​of key points in the topological structures of different frame images and the local feature descriptors of the key points, the best matching topological structure of each topological structure in each frame image is obtained, as follows:

[0074]

[0075] In the formula, AG n,q The local feature descriptor sequence obtained by arranging the local feature descriptors of all key points in the qth topological structure in the nth frame image from small to large is recorded as the first sequence, AG n+1,f The local feature descriptor sequence obtained by arranging the local feature descriptors of all key points in the f-th topological structure in the n+1-th frame image from small to large is recorded as the second sequence, cos(AG n,q AG n+1,f ) is the cosine similarity of the first sequence and the second sequence, where cos(AG n,q AG n+1,f ) is as follows: if the number of local feature descriptors in the first sequence and the second sequence is different, 0 is added to make the number of local feature descriptors in the first sequence and the second sequence the same, the high-dimensional vector determined by all local feature descriptors in the first sequence is recorded as the first vector, the high-dimensional vector determined by all local feature descriptors in the second sequence is recorded as the second vector, and the cosine similarity of the first vector and the second vector is recorded as cos(AG n,q AG n+1,f );δ n,q is the coefficient of variation of the grayscale values ​​of all key points in the qth topological structure in the nth frame image, δ n+1,f is the coefficient of variation of the grayscale values ​​of all key points in the f-th topological structure in the n+1-th frame image; it should be noted that the specific method for obtaining the coefficient of variation is a well-known method, which will not be repeated in this embodiment; || is the absolute value, P n,q,n+1,f is the matching degree between the qth topological structure in the nth frame image and the fth topological structure in the n+1th frame image.

[0076] The matching degree between the qth topological structure in the nth frame image and each topological structure in the n+1th frame image is obtained, and the topological structure corresponding to the maximum matching degree in the n+1th frame image is taken as the best matching topological structure of the qth topological structure in the nth frame image.

[0077] It should be noted that the qth topological structure belongs to the nth frame image, and the fth topological structure belongs to the n+1th frame image. It can be known from prior knowledge that the difference between two consecutive frame images is generally not very large. Therefore, assuming that the two topological structures represent the same local action of the anesthesiologist, the local feature descriptors corresponding to the different key points in the two topological structures should have a high similarity. Therefore, this embodiment first quantifies the similarity of the feature descriptors in the two topological structures, that is, the cosine similarity. The greater the similarity, the greater the possibility that the two are topological structures describing the same action; and when the two topological structures are topological structures of different frames describing the same action behavior, the key points in the two topological structures should be consistent. Therefore, this embodiment takes the key points in the two topological structures as the basis, quantifies the gray value distribution of the key points of the two topological structures, and uses the coefficient of variation to calculate the different key points in the two topological structures, and then obtains the difference value. The smaller the value, the greater the possibility that the two topological structures are topological structures describing the same action; finally, the best matching topological structure of the qth topological structure in the nth frame image is obtained.

[0078] Furthermore, according to the best matching topological structure of each topological structure in the frame image, several topological structure sets are obtained, as follows:

[0079] Construct an empty topological structure set, record the qth topological structure in the first frame image as the first topological structure, and use the first topological structure as the first element in the topological structure set. Among all the topological structures in the second frame image, obtain the best matching topological structure of the qth topological structure in the first frame image, record it as the second topological structure, and use the second topological structure as the second element in the topological structure set. Among all the topological structures in the third frame image, obtain the best matching topological structure of the second topological structure, record it as the third topological structure, and use the third topological structure as the third element in the topological structure set, and so on, until the best matching topological structure is obtained among all the topological structures in the last frame image and added to the topological structure set, and finally obtain the topological structure set corresponding to the qth topological structure in the first frame image.

[0080] Furthermore, according to the centroid coordinates of the key points of the topological structures in the topological structure set, the importance of each key point of the topological structure in each topological structure set is obtained, as follows:

[0081]

[0082] In the formula, X g G is the centroid coordinates of all key points in the g-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image. qis the number of topological structures in the topological structure set corresponding to the qth topological structure in the first frame image, is the centroid coordinates of all key points in all topological structures in the topological structure set corresponding to the qth topological structure in the first frame image, X h is the centroid coordinates of all key points in the hth topological structure in the topological structure set corresponding to the qth topological structure in the first frame image, except for the ith topological structure, where i∈[1,G q ], X i is the centroid coordinates of all key points in the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image, ||||2 is the two-norm, IP q,i is the initial importance of the key point of the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image. It should be noted that obtaining the centroid coordinates of the key points is an existing method, which will not be described in detail in this embodiment.

[0083] The initial importance of the key points of all topological structures in the topological structure set corresponding to all topological structures in the first frame image is linearly normalized, and the obtained result is used as the importance of the key point of each topological structure in each topological structure set.

[0084] It should be noted that, during the anesthesia performance of the anesthesia behavior, some key points correspond to the anesthesiologist's body parts and these key points are relatively stable. The local actions of the anesthesiologist corresponding to these key points are not important compared to the entire anesthesia behavior, while some key points correspond to the anesthesiologist's body parts that often move, that is, in the local actions, these key points participate in a large number of complex local anesthesia behaviors of the anesthesiologist, and often contain specific details of the overall anesthesia behavior. Therefore, for the above key points, this embodiment compares the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image with the rest of the G q -1 topological structure is used to calculate the centroid displacement offset. The larger the value is, the more important the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image is. It represents the weight of the degree of deviation of the anesthesiologist's action corresponding to the topological structure set. The larger the value is, the more important the key points of all the topological structures in the topological structure set are, and vice versa. Finally, the importance of the key point of the i-th topological structure is calculated using the above logic. The larger the value is, the more important the key point of the i-th topological structure is, and vice versa.

[0085] At this point, the importance of the key points of each topological structure in each topological structure set is obtained.

[0086] Step S004: Based on whether the key points of the topological structures in the topological structure set are important key points and whether there are missing key points in each topological structure in the topological structure set, obtain the missing important key points in each topological structure. Then, based on the several key points of each frame image of each action video and the missing important key points in each topological structure, establish a virtual simulation model of anesthesiologists.

[0087] It should be noted that the importance levels of the key points of each topological structure in each topological structure set are obtained above. Then, it is necessary to check for missing key points for the key points corresponding to each topological structure in each frame image, and based on the judgment of the importance level of the key points corresponding to the topological structure, decide whether to modify the SIFT algorithm to find the missing key points.

[0088] Specifically, based on the importance levels of the key points of each topological structure in each topological structure set, obtain several important key points of the topological structures in each topological structure set, as follows:

[0089] Preset a second threshold, denoted as th2. In this embodiment, th2 = 0.85 is used for description. Denote the importance level of the key points of the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image as IP q ′ ,i , if IP q ′ ,i ≥ th2, the key points of the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image are important key points. If IP q ′ ,i < th2, the key points of the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image are not important key points.

[0090] Furthermore, based on the number of key points of adjacent topological structures in the topological structure set, obtain the missing key points of each topological structure in each topological structure set, as follows:

[0091] For the topological structure set corresponding to the q-th topological structure in the first frame image, denote the number of key points of the i-th topological structure in the topological structure set as T1, and denote the number of key points of the (i - 1)-th topological structure in the topological structure set as T2. If T1 < T2, the key points of the i-th topological structure in the topological structure set are missing key points. If T1 ≥ T2, the key points of the i-th topological structure in the topological structure set are not missing key points. It should be noted that the first topological structure in the topological structure set is not judged for missing key points.

[0092] Furthermore, according to the important key points of the topological structures in the topological structure set and the key points missing from each topological structure in the topological structure set, the important key points missing from each topological structure are obtained, as follows:

[0093] A topological structure in which any key point is a missing key point and an important key point is recorded as a target topological structure. In the topological structure set to which the target topological structure belongs, the previous topological structure of the target topological structure is obtained, which is recorded as an initial topological structure. The local feature descriptor of each key point in the initial topological structure is compared with the local feature descriptor of each important key point in the target topological structure. The two key points corresponding to the minimum absolute value of the difference between the two local feature descriptors are taken as key points at the same position. Several key points that are not at the same position in the initial topological structure are obtained, that is, the corresponding missing important key points in the target topological structure, which are recorded as initial key points.

[0094] For the frame image belonging to the target topological structure, the extreme value selection threshold in the SIFT algorithm is traversed from small to large, and several key points corresponding to the frame image under each selection threshold are obtained. The local feature descriptor corresponding to each key point under each selection threshold is obtained and compared with the local feature descriptor of the initial key point. For any initial key point, the key point with the smallest absolute difference with the local feature descriptor of the initial key point is obtained, which is used as the important key point corresponding to the initial key point in the target topological structure, and is also recorded as the important key point missing in the target topological structure; for each initial key point, the corresponding important key point in the target topological structure is obtained to obtain several important key points missing in the target topological structure.

[0095] It should be noted that, by adjusting the steps of the SIFT algorithm, the important key points lost in the process of establishing the virtual simulation model by the anesthesiologist are obtained, and the model can be established after the missing important key points are obtained.

[0096] Specifically, according to several key points of each frame image of each action video and the important key points missing in each topological structure, a virtual simulation model of the anesthesiologist is established, that is, the motion capture of the anesthesiologist is completed. It should be noted that the virtual simulation model of the anesthesiologist can be established by skeleton model and virtual character rendering, which is an existing method and will not be repeated in this embodiment.

[0097] Through the above steps, a virtual simulation anesthesiologist motion capture method is completed.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A virtual simulation anesthesiologist motion capture method, characterized in that: The method includes the following steps: Collect multiple action videos of anesthesiologists and obtain several frame images of each action video; Obtain several key points of each frame image according to the frame images, obtain the local feature descriptors of each key point in each frame image according to the gray values of the pixel points within the preset window range of the key points in the frame images, and obtain several key points in the same topological structure in each frame image according to the local feature descriptors of different key points in the frame images and the distances between different key points; Obtain several topological structures of each frame image according to the coordinates of the key points in the same topological structure in the frame images, obtain the best-matched topological structure of each topological structure in each frame image according to the gray values of the key points and the local feature descriptors of the key points in the topological structures of different frame images, and obtain several topological structure sets according to the best-matched topological structures of each topological structure in the frame images; The specific steps included in obtaining the best-matched topological structure of each topological structure in each frame image according to the gray values of the key points and the local feature descriptors of the key points in the topological structures of different frame images are as follows: In the formula, AG n,q The local feature descriptor sequence obtained by arranging the local feature descriptors of all key points in the qth topological structure in the nth frame image from small to large is recorded as the first sequence, AG n+1,f The local feature descriptor sequence obtained by arranging the local feature descriptors of all key points in the f-th topological structure in the n+1-th frame image from small to large is recorded as the second sequence, cos(AG n,q AG n+1,f ) is the cosine similarity between the first sequence and the second sequence; δ n,q is the coefficient of variation of the grayscale values ​​of all key points in the qth topological structure in the nth frame image, δ n+1,f is the coefficient of variation of the grayscale values ​​of all key points in the f-th topological structure in the n+1-th frame image; || is the absolute value, P n,q,n+1,f is the matching degree between the qth topological structure in the nth frame image and the fth topological structure in the n+1th frame image; Obtain the matching degree between the q-th topological structure in the n-th frame image and each topological structure in the (n + 1)-th frame image, and take the topological structure corresponding to the maximum matching degree in the (n + 1)-th frame image as the best-matched topological structure of the q-th topological structure in the n-th frame image; Obtain several important key points of the topological structures in each topological structure set according to the centroid coordinates of the key points of the topological structures in the topological structure set, obtain the missing key points of each topological structure in each topological structure set according to the number of key points of adjacent topological structures in the topological structure set, obtain the missing important key points in each topological structure according to the important key points of the topological structures in the topological structure set and the missing key points of each topological structure in the topological structure set, and establish a virtual simulation model of an anesthesiologist according to the several key points of each frame image of each action video and the missing important key points in each topological structure; The specific steps included in obtaining the missing key points of each topological structure in each topological structure set according to the number of key points of adjacent topological structures in the topological structure set are as follows: For the topological structure set corresponding to the q-th topological structure in the first frame image, denote the number of key points of the i-th topological structure in the topological structure set as T1, and denote the number of key points of the (i - 1)-th topological structure in the topological structure set as T2. If T1 < T2, the key points of the i-th topological structure in the topological structure set are the missing key points; if T1 ≥ T2, the key points of the i-th topological structure in the topological structure set are not the missing key points; The specific steps included in obtaining the missing important key points in each topological structure according to the important key points of the topological structures in the topological structure set and the missing key points of each topological structure in the topological structure set are as follows: A topological structure in which any key point is a missing key point and an important key point is recorded as a target topological structure. In the topological structure set to which the target topological structure belongs, the previous topological structure of the target topological structure is obtained, which is recorded as an initial topological structure. The local feature descriptor of each key point in the initial topological structure is compared with the local feature descriptor of each important key point in the target topological structure. The two key points corresponding to the minimum absolute value of the difference between the two local feature descriptors are taken as the key points at the same position. Several key points that are not at the same position in the initial topological structure are obtained and recorded as initial key points. For the frame image belonging to the target topological structure, the extreme value selection threshold in the SIFT algorithm is traversed from small to large, and several key points corresponding to the frame image under each selection threshold are obtained. The local feature descriptor corresponding to each key point under each selection threshold is obtained and compared with the local feature descriptor of the initial key point. For any initial key point, the key point with the smallest absolute difference with the local feature descriptor of the initial key point is obtained, which is used as the important key point corresponding to the initial key point in the target topological structure, and is also recorded as the important key point missing in the target topological structure; for each initial key point, the corresponding important key point in the target topological structure is obtained to obtain several important key points missing in the target topological structure.

2. A virtual simulation anesthesiologist motion capture method according to claim 1, characterized in that: The specific steps of obtaining a plurality of key points of each frame image according to the frame image are as follows: Any action video is recorded as the target action video, and any frame image of the target action video is recorded as the target frame image. The target frame image is subjected to image pyramid downsampling and Gaussian blur processing to obtain several Gaussian blurred images of the target frame image. The SIFT algorithm is used to perform key point detection on each Gaussian blurred image to obtain several key points of the target frame image.

3. The method for capturing the motion of an anesthesiologist in virtual simulation according to claim 1, characterized in that: The specific steps of obtaining the local feature descriptor of each key point in each frame image according to the grayscale value of the pixel point within the preset window range of the key point in the frame image are as follows: In the formula, G n,m,j is the gray value of the jth pixel in the window range of ω×ω with the mth key point as the center in the nth frame image, ω is a preset first value, and J is the number of pixels in the window range of ω×ω with the mth key point as the center in the nth frame image; max(G n,m ) is the maximum gray value of the pixel in the window range of ω×ω with the mth key point as the center in the nth frame image; min(G n,m ) is the minimum gray value of the pixel in the window range of ω×ω with the mth key point as the center in the nth frame image; || is the absolute value, OG n,m It is the local feature descriptor of the mth key point in the nth frame image.

4. The method for capturing the motion of an anesthesiologist in virtual simulation according to claim 1, characterized in that: The method of obtaining a plurality of key points in the same topological structure in each frame image according to the local feature descriptors of different key points in the frame image and the distances between the different key points includes the following specific steps: In the formula, OG n,m is the local feature descriptor of the mth key point in the nth frame image, OG n,c is the local feature descriptor of the cth key point in the nth frame image, || is the absolute value, d m,c is the Euclidean distance between the mth key point and the cth key point, Norm[] is the linear normalization function, YZ n,m,c The probability that the mth key point and the cth key point in the nth frame image are in the same topological structure; A first threshold is preset, denoted as th1, if YZ n,m,c >th1, the mth key point and the cth key point in the nth frame image are in the same topological structure, if YZ n,m,c ≤th1, the mth key point and the cth key point in the nth frame image are not in the same topological structure.

5. The method for capturing the motion of an anesthesiologist in virtual simulation according to claim 1, characterized in that: The specific steps of obtaining a plurality of topological structure sets according to the best matching topological structure of each topological structure in the frame image are as follows: Construct an empty topological structure set, record the qth topological structure in the first frame image as the first topological structure, and use the first topological structure as the first element in the topological structure set. Among all the topological structures in the second frame image, obtain the best matching topological structure of the qth topological structure in the first frame image, record it as the second topological structure, and use the second topological structure as the second element in the topological structure set. Among all the topological structures in the third frame image, obtain the best matching topological structure of the second topological structure, record it as the third topological structure, and use the third topological structure as the third element in the topological structure set, and so on, until the best matching topological structure is obtained among all the topological structures in the last frame image and added to the topological structure set, and finally obtain the topological structure set corresponding to the qth topological structure in the first frame image.

6. The method for capturing the motion of an anesthesiologist in virtual simulation according to claim 1, characterized in that: The method of obtaining several important key points of the topological structures in each topological structure set according to the centroid coordinates of the key points of the topological structures in the topological structure set includes the following specific steps: Where, X q,g G is the centroid coordinates of all key points in the g-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image. q is the number of topological structures in the topological structure set corresponding to the qth topological structure in the first frame image, is the centroid coordinates of all key points in all topological structures in the topological structure set corresponding to the qth topological structure in the first frame image, X i,h is the centroid coordinates of all key points in the hth topological structure in the remaining topological structure set except the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image; X i is the centroid coordinates of all key points in the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image, ||||2 is the two-norm, IP q,i is the initial importance of the key point of the ith topological structure in the topological structure set corresponding to the qth topological structure in the first frame image; The initial importance of the key points of all topological structures in the topological structure set corresponding to all topological structures in the first frame image is linearly normalized, and the obtained result is used as the importance of the key point of each topological structure in each topological structure set; According to the importance of the key point of each topological structure in each topological structure set, several important key points of the topological structures in each topological structure set are obtained.

7. A virtual simulation anesthesiologist motion capture method according to claim 6, characterized in that: The steps of obtaining several important key points of the topological structures in each topological structure set according to the importance of the key points of each topological structure in each topological structure set are as follows: Preset a second threshold, denoted as th2, and denote the importance degree of the key points of the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image as IP q ′ ,i , if IP q ′ ,i ≥ th2, the key points of the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image are important key points. If IP q ′ ,i < th2, the key points of the i-th topological structure in the topological structure set corresponding to the q-th topological structure in the first frame image are not important key points.

Images