Surgical information system and sequence provision method

Abstract

To improve convenience.SOLUTION: A surgical information system (2) comprises a first processing device (10) located in a working area (6) and a second processing device (12) located in a clean area (8). The first processing device (10) is configured to: a) provide and reproduce a 3D model of a body part; b) receive instructions defining a sequence of views on the 3D model in that every view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) store the sequence of views together with the 3D model; and d) transmit the sequence of views and the 3D model to the second processing device (12). The second processing device (12) is configured to e) reproduce the sequence of views on a display device (24) forming part of the second processing device (12).SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a surgical information system for providing a sequence of views on a 3D model of a body part. Further, the present invention relates to a sequence providing method for providing a sequence of views on a 3D model of a body part. The present invention also relates to an operating method of a surgical information system for providing a sequence of views on a 3D model of a body part.

Background Art

[0002] For example, in a surgical environment such as a hospital, a surgeon is often provided with an individual 3D model of a patient's body part for the preparation of a surgical procedure. The body part is, for example, a part including an organ or a plurality of organs for which a surgical procedure is planned. Related data is provided to the surgeon several days before the actual surgical procedure. Based on this, the surgeon examines and plans the individual schedule of the surgical procedure. This varies for each patient. The reasons are, on the one hand, the individuality of the case (for example, the location, number, and size of a tumor or other malignant tissue to be removed), as well as the individuality of the patient's anatomical structure. For example, the course of veins and arteries in the parenchymal tissue of the liver, or the general health condition, varies for each patient.

[0003] When planning a procedure, a surgeon memorizes or writes down the individual steps in their head. With this conventional method, the possibility of documenting the schedule is limited or non-existent. Further, in a surgical environment, the possibility of displaying a carefully considered schedule is extremely limited. Also, sharing the schedule with other procedure-related personnel is very limited.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005] [Non-Patent Document 1] PERALTA MAXIME ET AL:”Machine learning in deep brain stimulation:A systematic review” ARTIFICIAL INTELLIGENCE IN MEDICINE,ELSEVIER,NL,vol.122,18 October 2021(2021-10-18),XP086874634,ISSN:0933-3657,DOI:10.1016 / J.ARTMED.2021.102198[retrieved on 2021-10-18] [Overview of the project] [Problems that the invention aims to solve]

[0006] The object of the present invention is to provide a surgical information system, a sequence provisioning method, and an operating method that can overcome the shortcomings of the prior art and improve convenience. [Means for solving the problem]

[0007] This objective is solved by a surgical information system for providing a sequence of views on a 3D model of a body part in a surgical environment having a working area and a clean area, the surgical information system comprising a first processing unit located in the working area and a second processing unit located in the clean area, The first processing unit is a) Provide and reproduce 3D models of body parts, b) Receiving a command to define a sequence of views on a 3D model, such that each view defines the orientation of the 3D model and the sequence defines the chronological order of the views on the 3D model, c) Save the sequence of views along with the 3D model. d) The system is configured to transmit the view sequence and 3D model to a second processing unit located in a clean area. The second processing apparatus is e) The device is configured to play back a sequence of views on a display device that forms part of the second processing unit.

[0008] A surgical information system offers the advantage of transferring, for example, planning information related to surgical procedures from the work area to the clean area. The work area is an area within a surgical facility, such as a hospital, and is not necessarily sterile. It is generally undesirable to take any items, such as notebooks or scraps of paper, from this area to the clean area. The clean area is a sterile area, such as an operating room. Using a surgical information system eliminates the need to take any items from the work area to the clean area, thereby improving hygiene levels. Furthermore, the complete information on which the plan was made—namely, 3D models of body parts and corresponding views—can be transferred to the clean area. The sequence of views can serve as a procedure plan for the surgical procedure. The sequence is stored in the first and second processing units, thus making it possible to provide planning materials. By storing the sequence, those involved in the procedure do not need to memorize the individual steps of the plan. This is particularly advantageous because many plans are made several days before the actual procedure.

[0009] The first and second processing units are, for example, computers, workstations, or other suitable devices. The first processing unit includes a screen or monitor for displaying a 3D model of a body part.

[0010] Depending on the options, the surgical information system has the advantage of being able to switch views. For example, the view can be switched between posterior and anterior views. Optionally, individual views can also be skipped.

[0011] According to an advantageous embodiment of the present invention, the surgical information system is further enhanced in that the 3D model comprises a plurality of subparts forming a body part, and the command received in feature b) includes a command relating to the transparency value of at least one subpart. The first processing unit further, In feature c), the transparency values ​​of the sub-parts (or more) in the view sequence are additionally saved, and the view sequence is configured to be sent according to feature d). The second processing unit further, It is configured to play a sequence of views according to feature e).

[0012] It is advantageous for a view to define not only the orientation of a 3D model in, for example, a Cartesian coordinate system, but also the transparency value of at least one subpart of the 3D model. The transparency value is, for example, the opacity of the subpart. Setting an additional transparency value for at least one subpart can be advantageous in many configurations, for example, because the structure or part of an organ or complex tissue is hidden by other parts of the organ or tissue. However, detailed knowledge of the spatial configuration of all structures—both visible and hidden—can be crucial for successful treatment. Therefore, setting the opacity of a subpart to a value that reveals the underlying structure of another subpart can be very useful.

[0013] According to yet another advantageous embodiment, The first processing unit further, Each provides multiple virtual camera positions that define the viewing direction on the 3D model. Having received a command according to feature b), which includes a command relating to at least one selected virtual camera position, In accordance with feature c), the selected virtual camera position(s) are additionally saved to the sequence of views. It is configured to send a sequence of views according to feature d), The second processing unit further, d1) Receive a command indicating the selected virtual camera position, It is configured to play back a sequence of views from a view direction defined by the selected virtual camera position on a 3D model, according to feature e). In this respect, there is an even more enhanced surgical information system.

[0014] By defining multiple virtual camera positions, additional freedom is added to the display of the 3D model. The user can adjust or supplement the view sequence by adding some viewing directions. This further enhances the treatment plan that can be implemented in the surgical information system.

[0015] According to yet another advantageous embodiment, the surgical information system can be further enhanced in that the command for at least one selected virtual camera position includes the selection of the default virtual camera position. The first processing device further is configured to additionally save the default virtual camera position in the view sequence according to feature c), and to transmit the view sequence according to feature d). The second processing device further d2) receives a command indicating the selection of the default virtual camera position, and is configured to reproduce at least one view of the view sequence from the viewing direction on the 3D model defined by the selected default virtual camera position according to feature e).

[0016] By defining the default virtual camera position, the handling of the surgical information system is simplified. In the case where an unexpected situation occurs that causes the user to be confused or overly stressed in some way due to a large number of possible views, if there is an option to return to the default view, the situation will become very quickly clear. This option helps the user to quickly control the situation.

[0017] The first processing device further a0) receives input data indicating patient data, instrument data, and / or data regarding the surgeon's preferences, performs view classification using a computer-based clinical decision support system (CDSS) implemented in the first processing device, performs classification based on the received input data, As input features for an artificial intelligence (AI) model, multiple views are received via the CDSS input interface. The processor performs inference calculations, applies multiple views to the AI ​​model, and generates view classifications. The surgical information system can be further enhanced by being configured to perform view classification such that view classification is communicated via the user interface (UI) of a first processing unit as a pre-selection of views, and a sequence of views is defined on the 3D model based on this classification.

[0018] The application of computer-based clinical decision support systems (CDSS) opens up the possibility of performing pre-selection of views using artificial intelligence models. These AI models can be pre-trained, for example, based on data related to similar surgical procedures. Therefore, the AI ​​models can suggest a collection of views that have been useful in past cases. This pre-selection can assist or be helpful when manually selecting views to define a sequence of views that function as a treatment plan.

[0019] The first processing unit, The command received by feature b), which includes a command regarding the selection of at least one view as a milestone view in a sequence of views, In accordance with feature c), the milestone view(s) are saved in addition to the sequence of views. It is configured to transmit a sequence of views to a second processing unit according to feature d), The second processing unit further, d3) Receive a command indicating the selection of the milestone view, Configured to play the selected milestone view according to feature e), In this respect, surgical information systems can be further enhanced.

[0020] Many surgical procedures have milestones where specific standard procedures are performed. The surgical information system according to this embodiment can establish links between a milestone view and such milestones by defining a milestone view. The surgeon is automatically guided to the milestones identified during the planning of the procedure.

[0021] According to yet another advantageous embodiment, the surgical information system further includes a second processing unit, The system can be enhanced in that it receives signals from surgical instruments connected to a second processing unit, the signals indicating milestone actions or uses of the surgical instruments, and the signals trigger the selection of a milestone view as an instruction indicating the selection of a milestone view according to feature d3).

[0022] A surgical information system is advantageous when it is connected to or linked to surgical instruments and receives trigger signals indicating that a specific milestone in the procedure has been reached. For example, this could be the operation of an HF generator that supplies high voltage to a surgical tool. This tool is typically used at the end of a sequence of surgical steps, for example, after the removal of malignant tissue. Completing this sequence of steps marks a milestone, and the standard procedure of the surgery must be performed. The system automatically prompts the user for the requirements to perform the standard procedure.

[0023] This objective is further addressed by a sequence provisioning method for providing a sequence of views on a 3D model of a body part in a surgical environment, where the surgical environment includes a working area and a clean area, and the sequence provisioning method is a) A step of providing a 3D model of a body part to a first processing device located in the work area and reproducing it, b) The first processing unit receives an instruction to define a sequence of views on a 3D model, such that each view defines the orientation of the 3D model and the sequence defines the chronological order of the views on the 3D model. c) The step of saving the sequence of views together with the 3D model to the first processing unit, d) The step of transmitting the view sequence and 3D model from the first processing unit to a second processing unit located in a clean area, e) The step of playing a sequence of views on a display device that forms part of a second processing device.

[0024] Depending on the options, the sequence delivery method may include steps to switch views, for example, in a backward or forward direction. The sequence delivery method may also include any steps to skip one or more views.

[0025] Furthermore, the same or similar advantages described for surgical information systems also apply, in the same or similar manner, to sequence delivery methods that provide a sequence of views on a 3D model of a body part in a surgical environment. Therefore, these advantages and options for enhancing sequence delivery methods will not be repeated.

[0026] In an advantageous embodiment, the sequence delivery method is further enhanced in that the 3D model in step a) includes a plurality of subparts that form a body part, the command received in step b) includes a command relating to the transparency value of at least one subpart, the transparency values ​​of the subpart(s) are additionally saved in the sequence of views in step c), the sequence of views is transmitted in step d), and played back in step e).

[0027] In yet another advantageous embodiment of the present invention, the sequence providing method further includes step a1) providing a plurality of virtual camera positions, each defining a view direction on a 3D model, wherein the instruction received in step b) includes an instruction relating to at least one selected virtual camera position, the selected virtual camera position(s) are additionally stored in the sequence of views in step c), the sequence of views is transmitted in step d), the sequence providing method further includes step d1) receiving an instruction indicating the selected virtual camera position in a second processing unit, and step e) playing back the sequence of views from the view direction on the 3D model defined by the selected virtual camera position.

[0028] Depending on the option, the sequence delivery method may include the step of switching virtual cameras during sequence playback.

[0029] The sequence providing method can be further enhanced such that a command relating to at least one selected virtual camera position includes the selection of a default virtual camera position which is additionally saved in step c) the sequence of views, the sequence of views is transmitted in step d), and the sequence providing method further includes step d2) receiving a command indicating the selection of a default virtual camera position in a second processing unit, and step e) includes playing back at least one view of the sequence of views from a view direction on a 3D model defined by the selected default virtual camera position.

[0030] In another advantageous embodiment, the sequence delivery method further includes step a0) performing a view classification using a computer-based clinical decision support system (CDSS) implemented in a first processing unit, wherein the classification is based on the input data received in a further step a0), the instructions indicate data relating to patient data, instrument data, and / or surgeon preferences, and step a0) performing the view classification, The input features of the artificial intelligence (AI) model include receiving multiple views via the CDSS input interface, This involves performing inference calculations using a processor, applying multiple views to an AI model, and generating classifications of those views. The method includes communicating a view classification via a user interface (UI) of a first processing unit as a pre-selection of views, and performing step b) defining a sequence of views on a 3D model based on that classification.

[0031] The sequence providing method can be further enhanced in that the instruction received in step b) includes an instruction relating to the selection of at least one view as a milestone view in the sequence of views, the milestone view(s) are additionally saved to the sequence of views in step c), and the sequence of views is sent to a second processing unit in step d), the sequence providing method further includes step d3) receiving an instruction indicating the selection of a milestone view, and step e) includes playing back the selected milestone view.

[0032] In yet another embodiment, the sequence providing method further includes receiving a signal from a surgical instrument connected to a second processing unit, the signal indicating a milestone operation or use of the surgical instrument, and the signal triggers the selection of a milestone view in step d3).

[0033] This objective is further addressed by an operating method for operating a surgical information system for providing a sequence of views on a 3D model of a body part in a surgical environment having a working area and a clean area, wherein the surgical information system comprises a first processing unit located in the working area and a second processing unit located in the clean area, and the operating method is a) A step of providing a 3D model of a body part to a first processing device and reproducing it, b) The first processing unit receives an instruction to define a sequence of views on a 3D model, such that each view defines the orientation of the 3D model and the sequence defines the chronological order of the views on the 3D model. c) The step of saving the sequence of views together with the 3D model to the first processing unit, d) A step of transmitting the view sequence and 3D model from the first processing unit to the second processing unit, e) The step of playing a sequence of views on a display device that forms part of a second processing device.

[0034] The same or similar advantages described with respect to the surgical information system and the sequence provision method for providing a sequence of views on a 3D model also apply in the same or similar manner to the operation method for operating the surgical information system, and therefore will not be repeated.

[0035] Further features of the present invention will become apparent from the description of embodiments according to the present invention, along with the claims and included drawings. Embodiments according to the present invention can satisfy individual characteristics or combinations of several characteristics.

[0036] The following description will be based on exemplary embodiments without limiting the general spirit of the present invention, but all disclosures of details of the present invention not described in more detail herein will be explicitly referenced to the drawings. [Brief explanation of the drawing]

[0037] [Figure 1] This is a simplified surgical information system. [Figure 2] This flowchart shows a method for providing a sequence of views on a 3D model of a body part in a surgical environment. [Figure 3] A schematic diagram of a computer-based clinical decision support system (CDSS) that can be implemented in the first processing unit is shown. [Figure 4] This is an illustrative diagram of a 3D model of a body part. [Modes for carrying out the invention]

[0038] In the drawings, the same or similar types of elements or their corresponding parts are given the same reference number so that there is no need to introduce the items again.

[0039] Figure 1 shows a simplified surgical information system 2, configured to provide a sequence of views and 3D models of body parts in a surgical environment 4. The surgical environment 4 includes a work area 6, which is, for example, an office space. Furthermore, the surgical environment 4 includes a clean area 8, which is, for example, an operating room. The surgical information system 2 comprises a first processing unit 10 located in the work area 6 and a second processing unit 12 located in the clean area 8. The first processing unit 10 and the second processing unit 12 are, for example, computers.

[0040] The first processing unit 10 is configured to provide 3D models of body parts, which will be described in more detail when referring to Figure 4. The 3D models can be stored in the non-primary memory of the first processing unit 10 or downloaded via a network. The first processing unit 10 is further configured to play the 3D models on a screen, for example. The first processing unit 10 is also configured to receive commands via a user interface 14. The user interface includes, for example, a keyboard and mouse pointer as input devices and the display of the first processing unit 10 as an output device. The received commands define a sequence of views on the 3D model, such that each view defines the orientation of the 3D model and the sequence of views defines the chronological order of the views on the 3D model. To allow the user to select a desired view, the first processing unit 10 provides the user with functions to manipulate the 3D model. For example, the user can tilt, rotate, and move the 3D model, and zoom in and zoom out to enlarge or minimize the playback of the 3D model.

[0041] Figure 4 shows an example of a view 16 on a 3D model 18 of a body part, for example, the liver. The 3D model 18 of the body part comprises various sub-parts 20a, 20b, and 20c, such as veins and arteries (20a), organ channels (20b), or malignant tissue (20c). The view 16 defines the orientation of the 3D model 18 in space, which is defined, for example, using a Cartesian coordinate system. The sequence of views 16, input by the user via the user interface 14 of the first processing unit 10, defines not only the orientation of the 3D model 18 for each view 16, but also the sequence of views 16. This sequence of views 16 is a chronological arrangement of the views 16. The chronological order of views 16 defines a treatment plan for, for example, a surgical treatment performed on the body part shown in the 3D model 18. The sequence of views 16 is stored together with the 3D model 18, for example, in a non-temporary storage medium of the first processing unit 10.

[0042] Subsequently, the sequence of view 16, along with the 3D model 18, is transmitted or transmitted from the first processing unit 10 to the second processing unit 12. For data communication, the first processing unit 10 and the second processing unit 12 are connected via a data link 22, which can be implemented as a wired or wireless data link.

[0043] The second processing unit 12 is located in the clean area 8. Data communication via the data link 22 is advantageous in preventing users of the surgical information system 2 from taking items from the work area 6 to the clean area 8. This has the advantage of improving the hygiene level of the surgical information system 2.

[0044] The second processing unit 12 is configured to play back the sequence of view 16 on a display device 24 that forms part of the second processing unit 12. For example, the display device 24 can be the display of the second processing unit 12 and a computer. As an example, the playback of the sequence of view 16 serves as a procedure plan for the operator of the surgical instrument 26. The sequence of view 16 can guide the operator or surgeon through the surgical procedure.

[0045] As already mentioned, the 3D model 18 comprises a plurality of subparts 20a, 20b, 20c that form a body part illustrated in the 3D model 18. Instructions received via the user interface 14 and the first processing unit 10 may further include instructions relating to at least one transparency value of the subparts 20a, 20b, and / or 20c of the body part. The transparency value can define, for example, the opacity of the subparts 20a, 20b, 20c in view 16 on the 3D model 18. By reducing the opacity of at least one of the subparts 20a, 20b, 20c, the user can identify the structure of the part covered by the subpart in front. However, knowing the exact structure of all subparts 20a, 20b, 20c can be a crucial factor in the success of surgical treatment.

[0046] The first processing unit 10 can be further configured to additionally store transparency values ​​for at least one sub-part 20a, 20b, 20c in the sequence of views 16. Transparency values ​​can be defined for each individual view 16. This information is stored with the sequence of views 16, transmitted to the second processing unit 12, and displayed on the display device 24.

[0047] The first processing unit 10 can be further configured to provide multiple virtual camera positions, each defining a view direction on the 3D model 18. For example, the first view direction 28 can be oriented perpendicular to the drawing plane. An alternative virtual camera position can define, for example, a second view direction 30 oriented toward the drawing plane. Commands received by the first processing unit 10 include commands relating to the selected virtual camera position, and multiple virtual camera positions can be defined for each view 16. In other words, for each view 16 in a sequence of views 16, multiple individual virtual camera positions can be defined. The virtual camera positions are additionally stored along with the sequence of views 16. The virtual camera positions, along with the sequence of views 16, are transmitted from the first processing unit 10 to the second processing unit 12 and displayed on the display device 24 of the second processing unit 12.

[0048] The second processing unit 12 can be configured to receive commands indicating a selected virtual camera position. These commands can be received via the user interface 32 of the second processing unit 12. For example, the user interface 32 can be a keyboard or a microphone. The second processing unit 12 can be configured to perform speech recognition to capture commands indicating a selected virtual camera position. The user of the second processing unit 12 can select an appropriate view from multiple virtual camera positions.

[0049] The definition of the virtual camera position can include the definition of a default virtual camera position. This is, for example, the virtual camera position defined by the first view direction 28 shown in Figure 4. In the second processing unit 12, when the user interface 32 receives a corresponding command, the playback of the view 16 sequence switches to the default virtual camera position. This allows the user of the surgical information system 2 to regain their sense of direction as needed.

[0050] Figure 3 is a schematic diagram of a computer-based clinical decision support system (CDSS), which is implemented in the first processing unit 10 as an example. The CDSS implements an artificial intelligence model (AI model). The AI ​​model receives many input features through an input interface. The input features are sent to the input layer of the AI ​​model. The AI ​​model performs inference operations on the input features and provides an output in the output layer. The result is output to the output interface. A confidence score is also assigned to the output. The CDSS is configured to receive patient records as input features, among other things, from patient records. Patient data can be stored in a database. Another input feature may be instrument data, which is data about the operating parameters of a surgical instrument 26, for example. This can be live (time-dependent) operating parameters or static parameters. Another input feature may be data about the surgeon's or operator's preferences for the surgical instrument 26. Other data related to operator preferences may be related to the playback of views on the display device 24. Based on these input features, the CDSS performs a classification of views 16. For this purpose, the CDSS can recognize all possible views and / or virtual camera positions. This is shown in Figure 3 as a dataset with a total of 38 views. Based on the input features, CDSS can perform a pre-selection of views 16 as output data. This is shown by a pre-selected dataset of views 40. Based on the pre-selected views, the user can further select the desired views 16 in the user interface 14 of the first processing unit 10. More general and detailed information about CDSS is provided at the end of this description.

[0051] Many surgical procedures have milestones, and standard procedures are performed. The first processing unit 10 is configured to receive instructions containing information about the selection of at least one view 16 as a milestone view. Which view 16 will reach the goal is determined during the planning of the procedure. When the milestone view is displayed in the sequence of views 16, the user of the surgical information system 2 is prompted that a particular milestone has been reached in the surgical procedure.

[0052] Figure 1 shows that a surgical instrument 26 is connected to a second processing unit 12 via a data line 34. This data line 34 can be implemented as a wired or wireless connection. The second processing unit 12 is configured to receive a signal S from the surgical instrument 26 via the data line 34. This signal S indicates a milestone operation or milestone use of the surgical instrument 26. For example, the surgical instrument 26 may generate this trigger signal S when an HF generator (not shown) is operated. In many surgical procedures, the use of the HF generator indicates the completion of a particular procedure, which means that a milestone of the surgical procedure has been reached. The trigger signal S triggers the selection of a milestone view in the second processing unit 12.

[0053] Figure 2 shows a flowchart of a sequence provisioning method that provides a sequence of views 16 on a 3D model 18 of a body part in a surgical environment 4. The surgical environment 4 is shown and described above. This sequence provisioning method includes steps a), b), c), d), and e). In step a), the 3D model 18 of the body part is provided in a first processing unit 10 located in the work area 6. In step b), a command is received in the first processing unit 10, which defines a sequence of views 16 on the 3D model 18 such that each view 16 defines the orientation of the 3D model 18. The sequence further defines the chronological order of the views 16. In step c), the sequence of views 16 is saved together with the 3D model 18. In step d), the sequence of views 16 is sent together with the 3D model 18 to a second processing unit 12 located in the clean area 8. In step e), the sequence of views 16 is played back on the display device 24 of the second processing unit 12.

[0054] Steps indicated by dashed lines are optional steps in the sequence delivery method. Step a0) relates to performing view classification using a computer-based clinical decision support system. Step a1) refers to the selection of a virtual camera view and corresponds to step d1) which refers to displaying the view from the selected view direction of the virtual camera.

[0055] Steps d1), d2), and d3) do not necessarily have to be performed in the specified order. They can be performed as alternative steps or in any order. Step d2) refers to the selection of the view for the default virtual camera. Step d3) refers to the display of the view for the milestone. This can be triggered by signal S.

[0056] The following provides general and more detailed information regarding the implementation of CDSS (see Figure 3).

[0057] Figure 3 shows a schematic diagram of an exemplary computer-based clinical decision support system (CDSS) configured to provide pre-selected views based on input data, which may be patient data, instrument data, and / or data relating to surgeon preferences. In various embodiments, the CDSS includes an input interface 30 through which input data specific to a patient, for example, is provided as one of the input features to an artificial intelligence (AI) model. The AI ​​model runs on a processor 36, which performs inference operations to generate pre-selected views by applying the input data, such as patient data, instrument data, and / or data relating to surgeon preferences, to the AI ​​model. The pre-selected view dataset 40 can be output via an output interface, which may be a user interface 14 of a first processing unit 10. The system output is communicated to a user, such as a clinician.

[0058] In some embodiments, the input interface may be a direct data link between the CDSS and one or more medical devices, such as a surgical instrument 26 that generates at least some of the input features. For example, the input interface may transmit input data directly to the CDSS during therapeutic and / or diagnostic medical procedures. Additionally or alternatively, the input interface may be a classic user interface that facilitates interaction between the user and the CDSS. For example, the input interface may facilitate a user interface in which the user manually enters input data, such as surgeon preferences. The input interface may be the user interface 14 of the first processing unit 10. Additionally or alternatively, the input interface may provide the CDSS with access to an electronic patient record from which one or more input features can be extracted. In any of these cases, the input interface is configured to collect one or more of the input data relating to features relevant to a particular patient at or before the time the CDSS is used in order to evaluate the selection of views for providing a treatment plan.

[0059] Based on one or more input features, the processor 36 performs inference operations using the AI ​​model to generate a pre-selection of views. For example, an input interface may deliver input data to the input layer of the AI ​​model, and the data may be propagated through the AI ​​model as input features to the output layer. The AI ​​model can provide the computer system with the ability to perform tasks without being explicitly programmed by performing inferences based on patterns discovered in the analysis of the data. The AI ​​model explores the study and construction of algorithms (e.g., machine learning algorithms) that can learn from existing data and make predictions about new data. Such algorithms work by building an AI model from exemplary training data to make data-driven predictions or decisions, which are expressed as outputs or evaluations.

[0060] Machine learning (ML) has two common modes: supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., examples of associating inputs with outputs or results) to learn the relationships between inputs and outputs. The goal of supervised ML is to learn a function that best approximates the relationship between training inputs and outputs given a given set of training data, so that the ML model can implement the same relationship and produce a corresponding output when given an input. Unsupervised ML involves training an ML algorithm using unclassified and unlabeled information, so that the algorithm can act on that information without guidance. Unsupervised ML is useful for exploratory analysis because it can automatically identify the structure of the data.

[0061] Common tasks in supervised machine learning are classification and regression problems. Classification problems, also known as categorization problems, aim to classify items into one of several categorical values ​​(e.g., is this object an apple or an orange?). Regression algorithms aim to quantify several items (e.g., by assigning a score to a given input value). Some examples of commonly used supervised machine learning algorithms include logistic regression (LR), naive Bayes, random forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and support vector machines (SVM).

[0062] Some common tasks in unsupervised machine learning include clustering, representation learning, and density estimation. Some examples of commonly used unsupervised machine learning algorithms are K-means clustering, principal component analysis, and autoencoders.

[0063] Another type of machine learning is federative learning (also known as collaborative learning), which trains algorithms among multiple distributed devices that hold local data without exchanging data. This approach contrasts with traditional centralized machine learning techniques where all local datasets are uploaded to a single server, as well as more classical distributed methods that often assume local data samples are equally distributed. Federative learning makes it possible for multiple parties to build a common, robust machine learning model without sharing data, thus addressing important issues such as data privacy, data security, data access rights, and access to heterogeneous data.

[0064] In some examples, the AI ​​model may be trained continuously or periodically before the inference calculations are performed by the processor 36. Then, during the inference calculations, patient-specific input features provided to the AI ​​model may be propagated from the input layer through one or more hidden layers to the output layer corresponding to the view selection.

[0065] During and / or immediately following the inference operation, the view selection may be communicated to the user via the user interface (UI) and / or automatically caused the first processing unit 10 to display the pre-selected view.

[0066] All specified characteristics, including those obtained solely from the drawings, and individual characteristics disclosed in combination with other characteristics, are considered important to the present invention, both individually and in combination. Embodiments of the present invention can be realized by individual features or combinations of several features. Features combined with the expressions "in particular" or "especially" shall be treated as preferred embodiments. [Explanation of symbols]

[0067] 2. Surgical Information System 4 Surgical environment 6 Working area 8 Clean area 10 First processing unit 12. Second processing unit 14. User Interface (First Processing Unit) 16 views 18 3D models 20a, 20b, 20c Sub-parts 22 Data Links 24 Display device 26 Surgical instruments 28. First View Direction 30 Second View Direction 32. User Interface (Second Processing Unit) 34 data lines 36 processors 38 total views 40 pre-selected datasets S signal CDSS (Computer-Based Clinical Decision Support System) AI model, artificial intelligence model

Claims

1. A surgical information system for providing a sequence of views on a 3D model of a body part in a surgical environment having a working area and a clean area, The system comprises a first processing device located in the work area and a second processing device located in the clean area. The first processing apparatus is a) Provide and reproduce 3D models of body parts, b) Receiving an instruction to define a sequence of views on the 3D model, wherein each view defines the orientation of the 3D model and the sequence defines the chronological order of the views on the 3D model, the instruction includes an instruction relating to the selection of at least one of the views as milestone views in the sequence of views, c) Save the milestone view in addition to the sequence of the view, and save the sequence of the view together with the 3D model. d) The sequence of the view and the 3D model are transmitted to the second processing unit located in the clean area, The second processing apparatus is e) Replay the sequence of the view on a display device that forms part of the second processing apparatus, d3) Upon receiving a command indicating the selection of the milestone view, A surgical information system configured to play back the selected milestone view in accordance with e) above.

2. The 3D model comprises a plurality of sub-parts that form the body part, and the command includes a command relating to the transparency value of at least one of the sub-parts. The first processing apparatus further, The system is configured to additionally save the transparency values ​​of the subparts in the sequence of the view in step c) above, and to transmit the sequence of the view in accordance with step d), The surgical information system according to claim 1, wherein the second processing unit is further configured to reproduce the sequence of the view according to e).

3. The first processing apparatus further, Each provides a plurality of virtual camera positions that define the viewing direction on the 3D model, Having received a command according to b), which includes a command relating to at least one selected virtual camera position, In accordance with c) above, the selected virtual camera position is additionally saved to the sequence of the view, It is configured to transmit the sequence of views in accordance with d) above, The second processing apparatus further, d1) Receiving a command indicating the selected virtual camera position, The surgical information system according to claim 1, configured to play back the sequence of views from the viewing direction defined by the selected virtual camera position on the 3D model in accordance with e) above.

4. The command for the at least one selected virtual camera position includes the selection of a default virtual camera position. The first processing apparatus further, The system is configured to additionally save the default virtual camera position to the sequence of the view in accordance with c), and to transmit the sequence of the view in accordance with d), The second processing apparatus further, d2) Receiving a command indicating the selection of the default virtual camera position, The surgical information system according to claim 3, configured to reproduce at least one of the sequences of views from the viewing direction on the 3D model defined by the selected default virtual camera position, in accordance with e) above.

5. The first processing apparatus further, a0) Receive input data showing patient data, instrument data, and / or surgeon preferences, The classification of the view is performed using a computer-based clinical decision support system (CDSS) implemented in the first processing unit. Based on the received input data, the classification is performed. As input features for the artificial intelligence (AI) model, multiple views are received via the input interface of the CDSS. The processor performs inference calculations and applies the plurality of views to the AI ​​model to generate the classification of the views. The surgical information system according to claim 1, configured to perform the classification of a view such that the classification of the view is communicated via the user interface (UI) of the first processing unit as a pre-selection of the view, and the definition of the sequence of the view is performed on the 3D model based on thereon.

6. The second processing apparatus further, The surgical information system according to claim 1, configured to receive a signal from a surgical instrument connected to the second processing unit, the signal indicating a milestone operation or use of the surgical instrument, and the signal triggering the selection of the milestone view as a command indicating the selection of the milestone view in accordance with d3).

7. A sequence provision method for providing a sequence of views on a 3D model of a body part in a surgical environment, The aforementioned surgical environment includes a work area and a clean area. The first processing apparatus located in the aforementioned work area, a) A step of providing and reproducing a 3D model of a body part, b) Receiving an instruction to define a sequence of views on the 3D model, wherein each view defines the orientation of the 3D model and the sequence defines the chronological order of the views on the 3D model, the instruction includes an instruction relating to the selection of at least one of the views as a milestone view in the sequence of views; c) The steps of saving the milestone view in addition to the sequence of the view, and saving the sequence of the view together with the 3D model, d) The step of transmitting the sequence of views and the 3D model to a second processing unit located in the clean area, The second processing apparatus, e) A step of playing back the sequence of the view on a display device that forms part of the second processing apparatus, d3) The step of receiving an instruction indicating the selection of the milestone view, The steps include: playing the selected milestone view in accordance with e) above; and performing A method for providing sequences.

8. The 3D model in step a) includes a plurality of sub-parts that form the body part, The first processing apparatus, In step b), the system receives an instruction that defines the sequence of the view, which includes an instruction relating to the transparency value of at least one of the subparts. In step c), the transparency value of the subpart is additionally saved to the sequence of the view, In step d), the sequence of the view is transmitted to the second processing unit. The second processing apparatus, The sequence provision method according to claim 7, wherein in step e), the sequence of the view is played back.

9. The first processing apparatus, The step a1) is further performed, providing a plurality of virtual camera positions, each defining a view direction on the 3D model. In step b), the system receives an instruction that defines the sequence of the view, which includes an instruction relating to at least one selected virtual camera position. In step c), the selected virtual camera position is additionally saved to the sequence of the view. In step d), the sequence of the view is transmitted to the second processing unit. The second processing apparatus, The step d1) is further performed, which involves receiving a command indicating the selected virtual camera position, The sequence provision method according to claim 7, wherein in step e), the sequence of views is played back from the view direction on the 3D model defined by the selected virtual camera position.

10. The instruction relating to the at least one selected virtual camera position includes the selection of a default virtual camera position. The first processing apparatus, In step c), the selected default virtual camera position is additionally saved to the sequence of the view. In step d), the sequence of the view is transmitted to the second processing unit. The second processing apparatus, The step d2) is further performed, which involves receiving a command indicating the selection of the default virtual camera position, The sequence provision method according to claim 9, wherein in step e), at least one of the views of the sequence of views is played back from the view direction on the 3D model defined by the selected default virtual camera position.

11. The first processing apparatus, The sequence provision method according to claim 7, further comprising the steps of using a computer-based clinical decision support system (CDSS) implemented in the first processing unit, using patient data, instrument data, and / or data relating to surgeon preferences and a plurality of the views as input features of an artificial intelligence (AI) model, performing inference calculations by the processor to generate classifications of the views, and communicating the classifications of the views as pre-selected views via the user interface (UI) of the first processing unit.

12. The second processing apparatus, Further, the second processing unit receives signals from surgical instruments connected to it. The aforementioned signal indicates a milestone operation or use of the surgical instrument. The second processing apparatus, The sequence provision method according to claim 7, wherein in step d3), the signal triggers the selection of the milestone view.

Images