Intraoperative spinal motion monitoring methods, devices, equipment, media and products

Abstract

The present invention discloses a method, device, equipment, medium and product for monitoring spinal motion during surgery, and belongs to the field of data processing technology. The method comprises: obtaining spinal posture data of a target object during surgery collected in real time by an optical guidance system, wherein the spinal posture data is status data of a target spinal part that has a motion association relationship with a predetermined spinal part; when the predicted moment is detected to have arrived, a set of spinal posture data collected within a set time period from the predicted moment is input into a pre-trained motion recognition model to obtain a spinal motion type identifier of the predetermined spinal part, wherein the pre-trained motion recognition model is configured to recognize at least two spinal motion types, and the at least two spinal motion types include a respiratory motion type. The technical solution provided by the embodiment of the present invention can accurately identify the motion type of the predetermined spinal part when the predetermined spinal part moves.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment, medium and product for monitoring spinal motion during surgery. Background Art

[0002] During spinal surgery, the spinal position is not static due to the patient's breathing and heartbeat. Since the patient's breathing and heartbeat are periodic, the prior art generally believes that the spinal movement during surgery is periodic, and spinal surgery is performed on the patient based on this.

[0003] In the process of implementing the technical solution of the present application, the inventors found that only associating the intraoperative spinal movement of the patient with the patient's breathing and heartbeat can easily lead to low accuracy of intraoperative spinal movement monitoring results, thereby resulting in poor surgical effect in avoiding spinal movement. Summary of the invention

[0004] The present invention provides a method, device, equipment, medium and product for monitoring spinal motion during surgery, so as to solve the problem of low accuracy of existing spinal motion monitoring results during surgery.

[0005] According to one aspect of the present invention, a method for monitoring spinal motion during surgery is provided. The method is executed by a processor and comprises:

[0006] Acquire spinal posture data of the target object during surgery collected in real time by the optical guidance system, wherein the spinal posture data is status data of the target spinal part that has a motion association relationship with the predetermined spinal part;

[0007] When the predicted moment is detected, the set of spinal posture data collected within the set time period from the predicted moment is input into a pre-trained motion recognition model to obtain the spinal motion type identification of the predetermined spinal part. The pre-trained motion recognition model is configured to recognize at least two spinal motion types, and the at least two spinal motion types include a respiratory motion type.

[0008] According to another aspect of the present invention, a device for monitoring spinal motion during surgery is provided. The method is executed by a processor and includes:

[0009] A data acquisition module, used to acquire spinal posture data of a target object during surgery collected in real time by an optical guidance system, wherein the spinal posture data is status data of a target spinal part that has a motion association relationship with a predetermined spinal part;

[0010] A movement type recognition module is used to input a set of spinal posture data collected within a set time period from the predicted moment into a pre-trained movement recognition model to obtain a spinal movement type identification of the predetermined spinal part when the predicted moment is detected. The pre-trained movement recognition model is configured to recognize at least two spinal movement types, and the at least two spinal movement types include a respiratory movement type.

[0011] According to another aspect of the present invention, a motion recognition model training method is provided, comprising:

[0012] Acquire a training sample set, the training sample set including a predetermined number of training samples, the training samples including spinal posture data and labels within a predetermined time period, the labels being determined based on a video corresponding to each of the training samples, the videos including a predetermined spinal part state within a corresponding time period of the corresponding training sample and an operation content applied to the predetermined spinal part;

[0013] The model parameter optimization of the motion recognition model is completed based on each of the training samples in the training sample set to obtain a pre-trained motion recognition model.

[0014] According to another aspect of the present invention, there is provided a motion recognition model training device, comprising:

[0015] A sample set module, used to obtain a training sample set, wherein the training sample set includes a predetermined number of training samples, wherein the training samples include spinal posture data and labels within a predetermined time period, wherein the labels are determined based on videos corresponding to each of the training samples, and wherein the videos include a predetermined spinal part state within a corresponding time period of the corresponding training sample and an operation content applied to the predetermined spinal part;

[0016] The model training module is used to optimize the model parameters of the motion recognition model based on each of the training samples in the training sample set to obtain a pre-trained motion recognition model.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the intraoperative spinal motion monitoring method described in any embodiment of the present invention or the motion recognition model training method described in any embodiment.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the intraoperative spinal motion monitoring method described in any embodiment of the present invention or the motion recognition model training method described in any embodiment when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements the intraoperative spinal motion monitoring method described in any embodiment or the motion recognition model training method described in any embodiment.

[0023] The technical solution of the intraoperative spinal motion monitoring method provided by the embodiment of the present invention is that since the spinal posture data is the status data of the target spinal part that has a motion association relationship with the predetermined spinal part, the spinal posture data set within the set time period from the prediction moment includes the posture transformation process of the spine within the set time period; in addition, the pre-trained motion recognition model is configured to recognize at least two spinal motion types including the respiratory motion type. Therefore, based on the pre-trained motion recognition model, the spinal posture data set within the predetermined time period is analyzed, and the specific spinal motion type identification of the predetermined spinal part can be determined according to the corresponding posture transformation process. In this way, the doctor can adopt corresponding countermeasures according to the spinal motion type corresponding to the specific spinal motion type identification, which helps to improve the accuracy of spinal surgery and the post-healing health status of the target object.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a method for monitoring spinal motion during surgery provided according to an embodiment of the present invention;

[0027] Figure 2 is another flow chart of a method for monitoring spinal motion during surgery provided according to an embodiment of the present invention;

[0028] Figure 3 is a schematic structural diagram of an intraoperative spinal motion monitoring device provided according to an embodiment of the present invention;

[0029] Figure 4 is another structural schematic diagram of an intraoperative spinal motion monitoring device provided according to an embodiment of the present invention;

[0030] Figure 5 is another structural schematic diagram of an intraoperative spinal motion monitoring device provided according to an embodiment of the present invention;

[0031] Figure 6 2 is a flow chart of a motion recognition model training method provided by an embodiment of the present invention;

[0032] Figure 7 2 is a schematic diagram of the structure of a motion recognition model training device provided by an embodiment of the present invention;

[0033] Figure 8 It is a schematic diagram of the structure of an electronic device for implementing the intraoperative spinal motion monitoring method of an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "predetermined", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Figure 1A flowchart of a method for monitoring spinal motion during surgery is provided for an embodiment of the present invention. This embodiment can be applied to analyzing a set of spinal posture data of a target object collected within a set period of time from the prediction moment using a pre-trained motion recognition model to determine the spinal motion type of the target object. The method can be performed by an intraoperative spinal motion monitoring device, which can be implemented in the form of hardware and / or software, and can be configured in a processor of an electronic device. Figure 1 As shown, the method includes:

[0037] S110, acquiring spinal posture data of the target object during surgery collected in real time by the optical guidance system, wherein the spinal posture data is status data of a target spinal part that has a motion correlation relationship with a predetermined spinal part.

[0038] This embodiment uses the existing optical guidance system to collect the spinal posture data of the target object during the operation in real time, which will not be described in detail here.

[0039] The sampling frequency of the spinal posture data can be selected as 60Hz, 70Hz, and other values. It is understandable that the higher the sampling frequency, the better the real-time performance of intraoperative spinal motion recognition, but the higher the data processing speed requirement of the processor, so the sampling frequency can be set according to the processor performance and the motion recognition speed requirement of the predetermined spinal part.

[0040] Intraoperative target subjects specifically refer to patients who are under anesthesia and are waiting to undergo spinal surgery or are undergoing spinal surgery.

[0041] The spinal posture data can be understood as data used to describe the movement state of the spine, including first state data used to describe the rotation state of the spine and second state data used to describe the spatial position of the spine. The first data is a quaternion and the second data is three-dimensional coordinate data.

[0042] The predetermined spinal site refers to the spinal surgery site of the target subject, i.e., the surgical area.

[0043] The target spinal part is the location where the tracer is set. Movement of the predetermined spinal part will cause movement of the target spinal part, and the tracer set at the target spinal part will not affect the surgery of the predetermined spinal part. Exemplarily, the predetermined spinal part (operative area) is the vertebra marked as L3, and the target spinal part is the L1 vertebra or the L5 vertebra.

[0044] S120. When the predicted moment is detected, a set of spinal posture data collected within a set time period from the predicted moment is input into a pre-trained motion recognition model to obtain a spinal motion type identification for a predetermined spinal part. The pre-trained motion recognition model is configured to recognize at least two spinal motion types, and the at least two spinal motion types include a respiratory motion type.

[0045] The scheduled time period can be set according to actual conditions, such as 1 minute, 1.5 minutes, 2 minutes, etc.

[0046] Based on the pre-trained motion recognition model, the spinal posture data set collected within the set time period from the prediction moment is analyzed to obtain the spinal motion type identification of the predetermined spinal part.

[0047] In one embodiment, the at least two spinal movement types include a respiratory movement type, a traction movement type, an anesthesia failure movement type and an interactive movement type; the respiratory movement type corresponds to the spinal movement caused by respiratory movement; the traction movement type corresponds to the spinal movement caused by surgical traction; the anesthesia failure movement type corresponds to the spinal movement caused by anesthesia failure; the interactive movement type corresponds to the spinal movement caused by the interaction between surgical instruments and bone tissue.

[0048] Specifically, if the pre-trained motion recognition model outputs the first motion type identifier, it means that the target object's spine has undergone spinal movement caused by respiratory movement; if the pre-trained motion recognition model outputs the second motion type identifier, it means that the target object's spine has undergone spinal movement caused by surgical traction; if the pre-trained motion recognition model outputs the third motion type identifier, it means that the target object's spine has undergone spinal movement caused by anesthesia failure; if the pre-trained motion recognition model outputs the fourth motion type identifier, it means that the target object's spine has undergone spinal movement caused by the interaction between surgical instruments and bone tissue.

[0049] In one embodiment, after the spinal motion type identifier is determined, the spinal motion type corresponding to the spinal motion type identifier is displayed in a visualization interface.

[0050] Specifically, the surgical robot system includes a display screen and a processor. After determining the spinal motion type, the processor controls the display screen to display a corresponding visualization interface, and displays the spinal motion type corresponding to the spinal motion type identifier in the visualization interface. The visualization interface may be a target interface for displaying the real-time dynamics of the surgical area. This embodiment allows the user to promptly discover the spinal motion type of the target object, and take corresponding measures for the spinal motion corresponding to the spinal motion type.

[0051] On the basis of the aforementioned embodiment, while displaying the spinal motion type corresponding to the motion type identifier, if the spinal motion type is a non-respiratory motion type, an alarm prompt information is output. The alarm prompt information can be a sound alarm information or a color alarm information. Exemplarily, when displaying the spinal motion type corresponding to the spinal motion type identifier, the spinal motion type is identified as a striking color, such as all motion types of spinal motion caused by non-respiratory motion are identified as yellow, orange or red, etc., or the description information of each spinal motion caused by non-respiratory motion is identified as a corresponding color, such as the description information of the spinal motion caused by surgical traction is identified as yellow, the description information of the spinal motion caused by anesthesia failure is identified as orange, the description information of the spinal motion caused by the interaction of surgical instruments and bone tissue is identified as red, etc.

[0052] The technical solution of the intraoperative spinal motion monitoring method provided by the embodiment of the present invention is that since the spinal posture data is the status data of the target spinal part that has a motion association relationship with the predetermined spinal part, the spinal posture data set within the set time period from the prediction moment includes the posture transformation process of the spine within the set time period; in addition, the pre-trained motion recognition model is configured to recognize at least two spinal motion types including the respiratory motion type. Therefore, based on the pre-trained motion recognition model, the spinal posture data set within the predetermined time period is analyzed, and the specific spinal motion type identification of the predetermined spinal part can be determined according to the corresponding posture transformation process. In this way, the doctor can adopt corresponding countermeasures according to the spinal motion type corresponding to the specific spinal motion type identification, which helps to improve the accuracy of spinal surgery and the post-healing health status of the target patient.

[0053] Figure 2 This is another flow chart of the intraoperative spinal motion monitoring method provided by an embodiment of the present invention. This embodiment adds a step of processing a spinal posture data set based on the previous embodiment. Figure 2 As shown, the method includes:

[0054] S2101. Obtain spinal posture data of the target object during surgery collected in real time by the optical guidance system. The spinal posture data is status data of a target spinal part that has a motion association relationship with a predetermined spinal part.

[0055] S2102, decentralize the spinal posture data set to obtain an intermediate spinal posture data set.

[0056] Each spinal posture data in the spinal posture data set includes first state data and second state data, the first state data is a four-element for representing rotation, and the second state data is a three-dimensional space coordinate.

[0057] For example, the spinal posture data is a 7-dimensional data, denoted as ,in is the quaternion used to represent the direction of rotation, Indicates spatial location.

[0058] A quaternion is an extended complex number consisting of a real part and an imaginary part. A quaternion can be expressed as

[0059] , where w is the real part, (x, y, z) is the component on the rotation axis corresponding to the imaginary part, the real part w represents the cosine value of the rotation angle, and the imaginary part (x, y, z) represents the product of the sine value of the rotation angle and the rotation axis.

[0060] After the spinal posture data set is determined, the mean of all elements under each feature in the spinal posture data set is determined; the corresponding mean is subtracted from the element under each feature in the spinal posture data set to obtain an intermediate spinal posture data set.

[0061] Exemplarily, the spine posture data set is The matrix is ​​as follows:

[0062]

[0063] For each feature , calculate its mean , where n is the feature identifier, and then the corresponding mean is subtracted from each sample to obtain the decentralized data matrix , as follows:

[0064] .

[0065] in, is the mean column vector of the feature identified as n, that is, , is the identity matrix.

[0066] S2103, determining the principal component direction of the intermediate spine posture data set, and projecting the intermediate spine posture data set to the principal component direction to update the spine posture data set.

[0067] In one embodiment, the covariance matrix of the intermediate spinal posture data set is determined; based on the covariance matrix, the maximum eigenvalue of the intermediate spinal posture data set and the maximum eigenvector corresponding to the maximum eigenvalue are determined; the product of the intermediate spinal posture data set and the maximum eigenvector is determined, and the product is used as the updated spinal posture data set.

[0068] Specifically, based on the above-mentioned embodiment, the covariance matrix of the intermediate spine posture data set can be expressed as:

[0069] .

[0070] This is a 7×7 matrix. Find the covariance matrix The eigenvalue and eigenvector of , the eigenvalue represents the variance of the data in a certain direction, and the eigenvector represents this direction. The eigenvector can be expressed as:

[0071] .

[0072] where λ is the eigenvalue, is the corresponding eigenvector. The core idea of ​​dimensionality reduction is to select the eigenvector corresponding to the largest eigenvalue , this eigenvector is the principal component direction of the 1-dimensional matrix after dimensionality reduction. After the principal component direction is determined, the intermediate spine posture data set is projected onto the principal component direction. Specifically, the projection operation is performed by Multiply by the eigenvector

[0073] To complete, as follows:

[0074] .

[0075] in, The reduced one-dimensional matrix data can be input into the pre-trained motion recognition model.

[0076] It should be noted that the embodiment of the present invention may also use other data dimensionality reduction methods to complete the dimensionality reduction operation of the intermediate spine posture data set, which will not be described in detail in this embodiment.

[0077] S220. When the predicted moment is detected, a set of spinal posture data collected within a set period of time from the predicted moment is input into a pre-trained motion recognition model to obtain a spinal motion type identification of a predetermined spinal part. The pre-trained motion recognition model is configured to recognize at least two spinal motion types, and the at least two spinal motion types include a respiratory motion type.

[0078] The technical solution of the intraoperative spinal motion monitoring method provided in an embodiment of the present invention performs decentralized processing on the spinal posture data set to obtain an intermediate spinal posture data set, and completes the dimensionality reduction of the intermediate spinal posture data set by projecting the intermediate spinal posture data set to the direction of its principal component; through the dimensionality reduction of the intermediate spinal posture data set, the data calculation amount of the pre-trained motion recognition model can be reduced, and its speed in determining the spinal motion type can be improved.

[0079] Figure 3 Schematic diagram of the structure of the intraoperative spinal motion monitoring device provided by an embodiment of the present invention. Figure 3As shown, the device comprises:

[0080] A data acquisition module 31 is used to acquire the spinal posture data of the target object during the operation collected in real time by the optical guidance system, wherein the spinal posture data is the state data of the target spinal part that has a motion association relationship with the predetermined spinal part;

[0081] The movement type recognition module 32 is used to input the spinal posture data set collected within a set time period from the predicted moment into a pre-trained movement recognition model to obtain the spinal movement type identification of the predetermined spinal part when the predicted moment is detected. The pre-trained movement recognition model is configured to recognize at least two spinal movement types, and the at least two spinal movement types include a respiratory movement type.

[0082] In one embodiment, the at least two spinal motion types include a breathing motion type, a traction motion type, an anesthesia failure motion type, and an interactive motion type;

[0083] The respiratory movement type corresponds to the spinal movement caused by the respiratory movement;

[0084] The type of traction movement corresponds to the spinal movement caused by surgical traction;

[0085] The anesthesia failure movement type corresponds to spinal movement caused by anesthesia failure;

[0086] The interactive motion type corresponds to spinal motion caused by the interaction of the surgical instrument with the bone tissue.

[0087] In one embodiment, the spinal posture data includes first state data and second state data, the first state data is a four-element for representing rotation, and the second state data is a three-dimensional space coordinate, such as Figure 4 As shown, the device further includes a data processing module 33, and the data processing module 33 includes:

[0088] A decentralization unit, used for decentralizing the spinal posture data set to obtain an intermediate spinal posture data set;

[0089] A projection unit is used to determine the principal component direction of the intermediate spine posture data set and project the intermediate spine posture data set to the principal component direction to update the spine posture data set.

[0090] In one example, the decentralized unit is specifically used to:

[0091] Determining the mean of all elements under each feature in the spinal posture data set;

[0092] Subtract the corresponding mean value from each characteristic element in the spinal posture data set to obtain an intermediate spinal posture data set;

[0093] The projection unit is specifically used for:

[0094] Determining a covariance matrix of the intermediate spine posture data set;

[0095] Determine the maximum eigenvalue of the intermediate spinal posture data set and the maximum eigenvector corresponding to the maximum eigenvalue based on the covariance matrix;

[0096] The product of the intermediate spinal posture data set and the maximum eigenvector is determined, and the product is used as the updated spinal posture data set.

[0097] In one embodiment, Figure 5 As shown, the device further includes a display module 34, which is used to:

[0098] Displaying the spinal motion type corresponding to the spinal motion type identifier in a visual interface;

[0099] When the spinal motion type is a non-respiratory motion type, an alarm prompt message is output.

[0100] The technical solution of the intraoperative spinal motion monitoring device provided by the embodiment of the present invention is that since the spinal posture data is the status data of the target spinal part that has a motion association relationship with the predetermined spinal part, the spinal posture data set within the set time period from the prediction moment includes the posture transformation process of the spine within the set time period; in addition, the pre-trained motion recognition model is configured to recognize at least two spinal motion types including the respiratory motion type. Therefore, based on the pre-trained motion recognition model, the spinal posture data set within the predetermined time period is analyzed, and the specific spinal motion type identification of the predetermined spinal part can be determined according to the corresponding posture transformation process. In this way, the doctor can adopt corresponding countermeasures according to the spinal motion type corresponding to the specific spinal motion type identification, which helps to improve the accuracy of spinal surgery and the post-healing health status of the target object.

[0101] The intraoperative spinal motion monitoring device provided in the embodiment of the present invention can execute the intraoperative spinal motion monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Figure 6 Flow chart of the motion recognition model training method provided by the embodiment of the present invention. Figure 6 As shown, the method includes:

[0103] S310. Obtain a training sample set, the training sample set includes a predetermined number of training samples, the training samples include spinal posture data and labels within a predetermined time period, the labels are determined based on the video corresponding to each training sample, and the video includes the state of the predetermined spinal part within the time period corresponding to the corresponding training sample and the operation content applied to the predetermined spinal part.

[0104] While collecting the spinal posture data of the predetermined spinal part in real time, a video including the state of the predetermined spinal part and the operation content applied to the predetermined spinal part is also collected. Specifically, while collecting the spinal posture data of the surgical area, a video of the surgical area is also collected in real time, so that the video will include the state of the surgical area and all the operation content applied to the surgical area, such as traction, surgical instruments and interactions, etc.

[0105] Based on the predetermined window interval and the sliding window corresponding to the predetermined duration, the summary results of the spinal posture data of the predetermined spinal part collected at each time are divided into multiple spinal posture data sets. In the multiple spinal posture data sets, the starting time interval corresponding to the adjacent spinal posture data sets is the predetermined window interval. Exemplarily, the predetermined window interval is the sum of the durations corresponding to 10 spinal posture data; the predetermined duration corresponds to the sum of the durations corresponding to 128 spinal posture data. After each spinal posture data set is determined, the sliding window moves backward by 10 data, and then the next spinal posture data set is determined. In this way, the first spinal posture data set includes spinal posture data marked as 0 to spinal posture data marked as 127; the second spinal posture data set includes spinal posture data marked as 10 to spinal posture data marked as 137, and so on. The Mth spinal posture data set includes spinal posture data marked as 10 (M-1) to spinal posture data marked as 10 (M-1) + 127.

[0106] Based on a predetermined window interval and a sliding window corresponding to the predetermined duration, the video is divided into video segments corresponding to each spinal posture data set; then, each spinal posture data set is labeled according to the video segments corresponding to each spinal posture data set to obtain training samples corresponding to each spinal posture data set.

[0107] It can be understood that since the video clips corresponding to each spinal posture data set include the motion state information and the cause of the motion of the predetermined spinal part corresponding to the corresponding spinal posture data set, such as whether there is surgical traction in the corresponding time period, or whether there is interaction between surgical instruments and bone tissue, etc., the label of each spinal posture data set can be accurately determined based on the video clips corresponding to each spinal posture data set.

[0108] Regarding the determination of the label corresponding to the spinal posture data set. In one embodiment, when the target object enters an anesthetized state and is in a surgical position but the operation has not yet begun, a set of spinal posture data sets is obtained, and the spinal posture data sets are combined into a basic spinal posture data set; each spinal posture data set collected during the operation is subtracted from the basic spinal posture data set to obtain a difference; if the difference meets the target data condition (for example, the spinal posture data difference at each moment is close to 0), then the corresponding label content of the corresponding spinal posture data set is "breathing", that is, the spinal movement corresponding to the spinal posture data set is caused by normal breathing; if the difference does not meet the target data condition, then the spinal movement corresponding to the spinal posture data set is not caused by normal breathing, and at this time, the corresponding label of the spinal posture data set is determined according to the video corresponding to the spinal posture data set. Since ideally, the spinal movement that occurs during spinal surgery is basically caused by the normal breathing movement of the target object, the difference method can be used to quickly determine the label content of some spinal posture data sets, thereby improving the label content determination speed of all spinal posture data sets.

[0109] S320 , optimizing the model parameters of the motion recognition model based on each training sample in the training sample set to obtain a pre-trained motion recognition model.

[0110] The motion recognition model may be a neural network model or other learning models. In one embodiment, the motion recognition model may be a three-layer neural network model, specifically, the input layer includes neurons for receiving 128 spinal posture data, the middle layer includes N neurons, and the output layer includes one neuron, where N is a positive integer. Optionally, in order to improve the accuracy of monitoring, an attention mechanism is set, such as a cross attention mechanism.

[0111] After the training sample set is determined, the motion recognition model is trained based on each training sample in the training sample set until the model parameters of the motion recognition model meet the preset conditions, thereby obtaining a pre-trained motion recognition model.

[0112] In one embodiment, while obtaining the training sample set, the verification sample set is obtained based on the same method, and the ratio between the number of training samples included in the training sample set and the number of verification samples included in the verification sample set can be 7:3 or 4:1, etc. After the pre-trained motion recognition model is determined, the verification sample set is used to verify the pre-trained motion recognition model; the prediction results corresponding to each verification sample in the verification sample set by the pre-trained motion recognition model are determined; if the number of verification samples in the verification sample set that meet the predetermined verification conditions exceeds the target threshold, it is determined that the pre-trained motion recognition model can be used for intraoperative spinal motion monitoring; if the number of verification samples in the verification sample set that meet the predetermined verification conditions does not exceed the target threshold, it is determined that the pre-trained motion recognition model cannot be used for intraoperative spinal motion monitoring, and it needs to be retrained according to S110-S120.

[0113] The technical solution of the motion recognition model training method provided in the embodiment of the present invention labels each spinal posture data set according to the video clips corresponding to each spinal posture data set, and obtains training samples corresponding to each spinal posture data set. Therefore, the training samples include accurate labels. Therefore, the motion recognition model is trained based on the training sample set, and the obtained pre-trained motion recognition model has higher prediction accuracy.

[0114] Figure 7 Schematic diagram of the structure of the motion recognition model training device provided by the embodiment of the present invention. Figure 7 As shown, the device comprises:

[0115] The sample set module 41 is used to obtain a training sample set, wherein the training sample set includes a predetermined number of training samples, wherein the training samples include spinal posture data and labels within a predetermined time period, wherein the labels are determined based on videos corresponding to each of the training samples, wherein the videos include a predetermined spinal part state within a corresponding time period of the corresponding training sample and an operation content applied to the predetermined spinal part;

[0116] The model training module 42 is used to optimize the model parameters of the motion recognition model based on each of the training samples in the training sample set to obtain a pre-trained motion recognition model.

[0117] The technical solution of the motion recognition model training device provided in the embodiment of the present invention labels each spinal posture data set according to the video clips corresponding to each spinal posture data set, and obtains training samples corresponding to each spinal posture data set. Therefore, the training samples include accurate labels. Therefore, the motion recognition model is trained based on the training sample set, and the obtained pre-trained motion recognition model has higher prediction accuracy.

[0118] The motion recognition model training device provided in the embodiment of the present invention can execute the motion recognition model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0119] Figure 8 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0120] like Figure 8 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0121] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a spinal motion monitoring method or a motion recognition model training method.

[0123] In some embodiments, the spinal motion monitoring method or the motion recognition model training method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the spinal motion monitoring method or the motion recognition model training method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the spinal motion monitoring method or the motion recognition model training method by any other appropriate means (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0130] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements a spinal motion monitoring method or a motion recognition model training method as provided in any embodiment of the present application.

[0131] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0133] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring spinal motion during surgery, characterized in that: The method is executed by a processor and includes: Acquire spinal posture data of the target object during surgery collected in real time by the optical guidance system, wherein the spinal posture data is state data of the target spinal part that has a motion association relationship with the predetermined spinal part, and the spinal posture data includes first state data and second state data, wherein the first state data is a four-element for representing rotation, and the second state data is a three-dimensional space coordinate; Decentralizing the spinal posture data set to obtain an intermediate spinal posture data set; Determine the principal component direction of the intermediate spine posture data set, and project the intermediate spine posture data set to the principal component direction to update the spine posture data set; When the predicted moment is detected, the updated spinal posture data set within the set time period from the predicted moment is input into a pre-trained motion recognition model to obtain the spinal motion type identification of the predetermined spinal part. The pre-trained motion recognition model is configured to recognize at least two spinal motion types, and the at least two spinal motion types include a respiratory motion type.

2. The method according to claim 1, characterized in that The at least two spinal motion types include a respiratory motion type, a traction motion type, an anesthesia failure motion type and an interactive motion type; The respiratory movement type corresponds to the spinal movement caused by the respiratory movement; The type of traction movement corresponds to the spinal movement caused by surgical traction; The anesthesia failure movement type corresponds to spinal movement caused by anesthesia failure; The interactive motion type corresponds to spinal motion caused by the interaction of the surgical instrument with the bone tissue.

3. The method according to claim 1, characterized in that The decentralizing the spinal posture data set to obtain an intermediate spinal posture data set includes: Determining the mean of all elements under each feature in the spinal posture data set; Subtract the corresponding mean value from each characteristic element in the spinal posture data set to obtain an intermediate spinal posture data set; The determining of the principal component direction of the intermediate spine posture data set and projecting the intermediate spine posture data set onto the principal component direction to update the spine posture data set comprises: Determining a covariance matrix of the intermediate spine posture data set; Determine the maximum eigenvalue of the intermediate spinal posture data set and the maximum eigenvector corresponding to the maximum eigenvalue based on the covariance matrix; The product of the intermediate spinal posture data set and the maximum eigenvector is determined, and the product is used as the updated spinal posture data set.

4. The method according to claim 1, characterized in that: After obtaining the spinal motion type identification, it also includes: Displaying the spinal motion type corresponding to the spinal motion type identifier in a visual interface; When the spinal motion type is a non-respiratory motion type, an alarm prompt message is output.

5. A device for monitoring spinal motion during surgery, characterized in that: include: A data acquisition module, used to acquire spinal posture data of a target object during surgery collected in real time by an optical guidance system, wherein the spinal posture data is state data of a target spinal part that has a motion association relationship with a predetermined spinal part, and the spinal posture data includes first state data and second state data, wherein the first state data is a four-element for representing rotation, and the second state data is a three-dimensional space coordinate; Decentralizing the spinal posture data set to obtain an intermediate spinal posture data set; Determine the principal component direction of the intermediate spine posture data set, and project the intermediate spine posture data set to the principal component direction to update the spine posture data set; A movement type recognition module is used to input an updated set of spinal posture data within a set time period from the predicted moment into a pre-trained movement recognition model when the predicted moment is detected, so as to obtain a spinal movement type identification of the predetermined spinal part, wherein the pre-trained movement recognition model is configured to recognize at least two spinal movement types, and the at least two spinal movement types include a respiratory movement type.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the intraoperative spinal motion monitoring method described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the intraoperative spinal motion monitoring method described in any one of claims 1-4 when executed.

8. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the intraoperative spinal motion monitoring method according to any one of claims 1-4.

Images