Surgical information system, sequence providing method, and operation method

Abstract

To improve convenience.SOLUTION: A surgical information system (2) comprises a first processing device (10) located in a working area (6) and a second processing device (12) located in a clean area (8). The first processing device (10) is configured to: a) provide and reproduce a 3D model of a body part; b) receive instructions defining a sequence of views on the 3D model in that every view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) store the sequence of views together with the 3D model; and d) transmit the sequence of views and the 3D model to the second processing device (12). The second processing device (12) is configured to e) reproduce the sequence of views on a display device (24) forming part of the second processing device (12).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a surgical information system for providing a sequence of views on a 3D model of a body part. Further, the present invention relates to a method for providing a sequence of views on a 3D model of a body part. The present invention also relates to a method of operating a surgical information system for providing a sequence of views on a 3D model of a body part. [Background technology]

[0002] In surgical environments, for example in hospitals, surgeons are often provided with individual 3D models of a patient's body part in preparation for the surgical procedure. This can be, for example, an organ or a region comprising several organs on which a surgical procedure is planned. The surgeon is provided with the relevant data several days before the actual surgical procedure. Based on this, the surgeon considers and plans an individual schedule for the surgical procedure. This varies from patient to patient. This is due, on the one hand, to the individuality of the case (e.g., the location, number, and size of tumors or other malignant tissues to be removed) and, on the other hand, to the individuality of the patient's anatomy. For example, the course of veins and arteries within the liver parenchyma or the general state of health varies from patient to patient.

[0003] When planning a procedure, surgeons memorize each individual step in their mind or write it out on paper. This traditional approach offers limited or no possibilities for documenting the schedule. Furthermore, the possibilities for displaying the carefully considered schedule in the surgical environment are very limited. The ability to share the schedule with other procedure participants is also very limited. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 051080 [Non-patent literature]

[0005] [Non-Patent Document 1] PERALTA MAXIME ET AL:”Machine learning in deep brain stimulation:A systematic review” ARTIFICIAL INTELLIGENCE IN MEDICINE,ELSEVIER,NL,vol.122,18 October 2021(2021-10-18),XP086874634,ISSN:0933-3657,DOI:10.1016 / J.ARTMED.2021.102198[retrieved on 2021-10-18] Summary of the Invention [Problem to be solved by the invention]

[0006] SUMMARY OF THE INVENTION An object of the present invention is to provide a surgical information system, a sequence providing method, and an operation method that can overcome the drawbacks of the prior art and improve convenience. [Means for solving the problem]

[0007] This object is solved by a surgical information system for providing a sequence of views on a 3D model (three-dimensional model) of a body part in a surgical environment having a working area and a clean area, the surgical information system comprising a first processing device located in the working area and a second processing device located in the clean area, The first processing device includes: a) Provide a 3D model of the body part to be reproduced; b) receiving instructions defining a sequence of views on the 3D model, where each view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) saving the sequence of views together with the 3D model, d) transmitting the sequence of views and the 3D model to a second processing device located in the clean area; The second processing device is e) playing the sequence of views on a display device forming part of the second processing device.

[0008] The surgical information system advantageously provides the possibility of transferring planning information, for example, for a surgical procedure, from a work area to a clean area. The work area is an area within a surgical facility, such as a hospital, that is not necessarily sterile. It is generally undesirable to transfer any items, such as notebooks or pieces of paper, from this area to a clean area. A clean area is particularly a sterile area, such as an operating room. With the surgical information system, there is no need to transfer any items from the work area to the clean area. This improves the level of hygiene. Furthermore, the complete information on which the planning was performed, i.e., a 3D model of the body part and the corresponding views, can be transferred to the clean area. The sequence of views can serve as a treatment plan for the surgical procedure. The sequence is stored in the first and second processing devices, thus providing a documentation of the plan. By storing the sequence, those involved in the procedure do not need to memorize the individual steps of the plan. This is particularly advantageous, since much planning is done several days before the actual procedure.

[0009] The first processing device and the second processing device may be, for example, a computer, a workstation, or other suitable device. The first processing device includes a screen or monitor for displaying the 3D model of the body part.

[0010] Advantageously, the surgical information system allows for options to switch between views, for example the view can be switched backward or forward, and optionally individual views can be skipped.

[0011] According to an advantageous embodiment of the invention, the surgical information system is further enhanced in that the 3D model comprises a plurality of sub-parts forming a body part, and the instructions received in feature b) comprise instructions regarding a transparency value of at least one sub-part, The first processing device further comprises: configured to additionally store transparency values ​​of subpart(s) in the sequence of views in feature c) and transmit the sequence of views according to feature d); The second processing device further comprises: configured to play the sequence of views according to feature e).

[0012] Advantageously, a view not only defines the orientation of the 3D model, for example, in a Cartesian coordinate system, but also defines a transparency value for at least one subpart of the 3D model. The transparency value, for example, is the opacity of the subpart. Additional setting of the transparency value for at least one subpart can be advantageous in many configurations, for example, when a structure or region of an organ or complex tissue is hidden by another region of the organ or tissue. However, detailed knowledge of the spatial arrangement of all structures, i.e., visible and hidden structures, can be crucial for successful treatment. Therefore, for example, setting the opacity of a subpart to a value that reveals the underlying structure of another subpart can be very useful.

[0013] According to yet another advantageous embodiment, The first processing device further comprises: providing a plurality of virtual camera positions, each of which defines a viewing direction on the 3D model; receiving instructions according to feature b), including instructions regarding at least one selected virtual camera position; additionally storing the selected virtual camera position(s) in the sequence of views according to feature c); transmitting the sequence of views according to feature d); The second processing device further comprises: d1) receiving an instruction indicating a selected virtual camera position; and reproducing a sequence of views from a view direction on the 3D model defined by the selected virtual camera position according to feature e). In this regard, there is an even more enhanced surgical information system.

[0014] Defining multiple virtual camera positions adds additional degrees of freedom to the display of the 3D model. Users can adjust or supplement the sequence of views by adding several view directions, further enhancing the treatment planning that can be performed with the surgical information system.

[0015] According to yet another advantageous embodiment, the surgical information system may be further enhanced in that the instructions for the at least one selected virtual camera position include selecting a default virtual camera position; The first processing device further comprises: additionally storing a default virtual camera position in the sequence of views according to feature c) and transmitting the sequence of views according to feature d); The second processing device further comprises: d2) receiving an instruction indicating a selection of a default virtual camera position; and reproducing at least one view of the sequence of views from a view direction on the 3D model defined by the selected default virtual camera position according to feature e).

[0016] Defining a default virtual camera position simplifies the handling of the surgical information system. In the unlikely event that the multiple possible views cause the user to become confused or overstressed in some way, having the option to return to the default view will clarify the situation very quickly. This option helps the user quickly regain control of the situation.

[0017] the first processing device further comprising: a0) receiving input data indicative of patient data, instrument data, and / or surgeon preference data; performing view classification using a computer-based clinical decision support system (CDSS) implemented on the first processing device; Classifying the received input data; receiving multiple views via an input interface of the CDSS as input features for an artificial intelligence (AI) model; performing an inference operation with a processor, applying the multiple views to an AI model to generate a classification of the views; The surgical information system can be further enhanced in that it is configured to perform a classification of views, such that the classification of views is communicated via a user interface (UI) of the first processing device as a pre-selection of views, based on which a definition of a sequence of views is performed on the 3D model.

[0018] The application of computer-based clinical decision support systems (CDSS) opens the possibility of performing view preselection using an artificial intelligence model. The artificial intelligence model can be pre-trained, for example, based on data from similar surgical procedures. Thus, the artificial intelligence model can suggest a collection of views that have been useful in previous cases. This preselection can assist or aid in the manual selection of views that define a sequence of views that serves as a treatment plan.

[0019] a first processing device, receiving instructions according to feature b) including instructions regarding the selection of at least one view as a milestone view in the sequence of views; incrementally storing the milestone view(s) in the sequence of views according to feature c); transmitting the sequence of views to a second processing device in accordance with feature d); the second processing device further comprising: d3) receiving an instruction indicating a selection of a milestone view; reproducing the selected milestone view according to feature e); In this respect, surgical information systems can be further strengthened.

[0020] Many surgical procedures have milestones where specific standard procedures are performed. By defining milestone views, surgical information systems according to embodiments can establish links between views and such milestones. Surgeons are automatically prompted to milestones identified during procedure planning.

[0021] According to yet another advantageous embodiment, the surgical information system further comprises: The device can be enhanced in that it is configured to receive a signal from a surgical instrument coupled to the second processing device, the signal indicating a milestone action or use of the surgical instrument, the signal triggering selection of the milestone view as an instruction indicating selection of the milestone view in accordance with feature d3).

[0022] The surgical information system is advantageously connected or coupled to the surgical instruments to receive a trigger signal indicating that a particular milestone in the procedure has been reached. For example, this can be the operation of an HF generator that supplies high voltage to a surgical tool. This tool is typically used at the end of a sequence of surgical steps, such as after removing malignant tissue. By completing this sequence of steps, a milestone has been reached and the standard procedure for surgery must be performed. The system automatically prompts the user with the requirements for performing the standard procedure.

[0023] The object is further solved by a sequence providing method for providing a sequence of views on a 3D model of a body part in a surgical environment, the surgical environment comprising a working area and a clean area, the sequence providing method comprising: a) providing a 3D model of a body part to a first processing device located in a working area for reproduction; b) receiving at a first processing device instructions defining a sequence of views on the 3D model, where each view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) storing the sequence of views together with the 3D model in the first processing device; d) transmitting the sequence of views and the 3D model from the first processing device to a second processing device located in the clean area; e) playing the sequence of views on a display device forming part of the second processing device.

[0024] Optionally, the sequence presentation method includes the step of switching views, for example in a backward or forward direction. The sequence presentation method may also include the optional step of skipping one or more views.

[0025] Additionally, the same or similar advantages as discussed with respect to the surgical information system also apply in the same or similar manner to a sequence presentation method for providing a sequence of views onto a 3D model of a body part in a surgical environment, and therefore, these advantages and options for enhancing the sequence presentation method will not be repeated.

[0026] According to an advantageous embodiment, the sequence providing method is further enhanced in that the 3D model in step a) comprises a plurality of sub-parts forming a body part, the instructions received in step b) comprise instructions regarding a transparency value of at least one sub-part, the transparency value of the sub-part(s) being additionally stored in the sequence of views in step c), and the sequence of views being transmitted in step d) and played back in step e).

[0027] In yet another advantageous embodiment of the present invention, the sequence providing method further comprises a step a1) of providing a plurality of virtual camera positions each defining a view direction on the 3D model, wherein the instructions received in step b) comprise instructions for at least one selected virtual camera position, wherein the selected virtual camera position(s) are additionally stored in the sequence of views of step c), and the sequence of views is transmitted in step d), wherein the sequence providing method further comprises a step d1) of receiving, in the second processing device, instructions indicating the selected virtual camera positions, and wherein step e) comprises a step of playing back the sequence of views from the view directions on the 3D model defined by the selected virtual camera positions.

[0028] Optionally, the method of providing a sequence includes switching virtual cameras during playback of the sequence.

[0029] The sequence providing method can be further enhanced in that the instructions regarding the at least one selected virtual camera position include a selection of a default virtual camera position which is additionally saved in the sequence of views in step c), and the sequence of views is transmitted in step d), the sequence providing method further comprising a step d2) of receiving, at the second processing device, instructions indicating the selection of the default virtual camera position, and step e) includes playing back at least one view of the sequence of views from a view direction on the 3D model defined by the selected default virtual camera position.

[0030] In another advantageous embodiment, the method for providing a sequence further comprises a step a0) of performing a classification of views using a computer-based clinical decision support system (CDSS) implemented in the first processing device, the classification being based on input data received in a further step a0), the instructions indicating patient data, instrument data, and / or data relating to surgeon preferences, and wherein the step a0) of performing a classification of views comprises: receiving a plurality of views via an input interface of the CDSS as input features for an artificial intelligence (AI) model; performing, by a processor, an inference operation, applying the plurality of views to an AI model to generate a classification of the views; and transmitting the classification of views via a user interface (UI) of the first processing device as a pre-selection of views, based on which step b) of defining a sequence of views on the 3D model is performed.

[0031] The sequence providing method may be further enhanced in that the instructions received in step b) include instructions regarding the selection of at least one view as a milestone view in the sequence of views, the milestone view(s) being additionally stored in the sequence of views in step c), and the sequence of views being transmitted to the second processing device in step d), the sequence providing method further comprising step d3) of receiving instructions indicating the selection of the milestone view, and step e) comprising playing the selected milestone view.

[0032] In yet another embodiment, the sequence providing method further includes receiving a signal from a surgical instrument coupled to the second processing device, the signal indicating a milestone operation or use of the surgical instrument, and the signal triggering selection of a milestone view in step d3).

[0033] This object is further solved by a method of operating a surgical information system for providing a sequence of views on a 3D model of a body part in a surgical environment having a working area and a clean area, the surgical information system comprising a first processing device located in the working area and a second processing device located in the clean area, the method comprising: a) providing a first processing device with a 3D model of the body part for reproduction; b) receiving at a first processing device instructions defining a sequence of views on the 3D model, where each view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) storing the sequence of views together with the 3D model in the first processing device; d) transmitting the sequence of views and the 3D model from the first to a second processing device; e) playing the sequence of views on a display device forming part of the second processing device.

[0034] The same or similar advantages as mentioned with respect to the surgical information system and with respect to the sequence providing method for providing a sequence of views on a 3D model also apply in the same or similar manner to the operating method for operating the surgical information system and will not be repeated.

[0035] Further features of the invention will become apparent from the description of the embodiments according to the invention, together with the claims and the included drawings. The embodiments according to the invention can fulfil individual characteristics or a combination of several characteristics.

[0036] The following description is based on exemplary embodiments, without limiting the general spirit of the invention, but explicit reference is made to the drawings for the disclosure of all details according to the invention not described in more detail in the text. [Brief explanation of the drawings]

[0037] [Figure 1] This is a simplified surgical information system. [Figure 2] 1 is a flowchart illustrating a method for providing a sequence of views on a 3D model of a body part in a surgical environment. [Figure 3] 1 illustrates a schematic representation of a computer-based clinical decision support system (CDSS) that can be implemented on a first processing device. [Figure 4] FIG. 1 is an exemplary illustration of a 3D model of a body part. DETAILED DESCRIPTION OF THE INVENTION

[0038] In the figures, the same or similar types of elements or respective corresponding parts are provided with the same reference numerals so as not to have to reintroduce the items.

[0039] 1 illustrates a simplified surgical information system 2 configured to provide a sequence of views and a 3D model of a body part in a surgical environment 4. The surgical environment 4 includes a work area 6, e.g., an office space. Additionally, the surgical environment 4 includes a clean area 8, e.g., an operating room. The surgical information system 2 includes a first processing device 10 located in the work area 6 and a second processing device 12 located in the clean area 8. The first processing device 10 and the second processing device 12 are, e.g., computers.

[0040] The first processing device 10 is configured to provide a 3D model of a body part, which will be described in more detail with reference to FIG. 4 . The 3D model can be stored in a non-temporary memory of the first processing device 10 or downloaded via a network. The first processing device 10 is further configured to, for example, reproduce the 3D model on a screen. The first processing device 10 is also configured to receive instructions via a user interface 14. The user interface includes, for example, a keyboard and a mouse pointer as input devices and a display of the first processing device 10 as an output device. The received instructions define a sequence of views on the 3D model, with each view defining an orientation of the 3D model and the sequence of views defining a chronological order of the views on the 3D model. To enable the user to select a desired view, the first processing device 10 provides the user with the ability to manipulate the 3D model. For example, the user can tilt, rotate, and move the 3D model and can zoom in and out to enlarge or minimize the reproduction of the 3D model.

[0041] FIG. 4 shows an example of views 16 on a 3D model 18 of a body part, e.g., a liver. The 3D model 18 of the body part includes various subparts 20a, 20b, and 20c, e.g., veins and arteries (20a), organ channels (20b), or malignant tissue (20c). The views 16 define the orientation of the 3D model 18 in space, e.g., using a Cartesian coordinate system. The sequence of views 16 input by a user via the user interface 14 of the first processing device 10 not only defines the orientation of the 3D model 18 for each view 16, but also defines the sequence of views 16. This sequence of views 16 is a chronological order of the views 16. The chronological order of the views 16 defines a treatment plan, e.g., a surgical treatment, to be performed on the body part shown in the 3D model 18. The sequence of views 16 is stored together with the 3D model 18, e.g., in a non-transitory storage medium of the first processing device 10.

[0042] The sequence of views 16, together with the 3D model 18, are then transmitted or communicated from the first processing device 10 to the second processing device 12. For data communication, the first processing device 10 and the second processing device 12 are coupled via a data link 22, which may be implemented as a wired or wireless data link.

[0043] The second processing device 12 is located in the clean area 8. Data communication via the data link 22 advantageously prevents users of the surgical information system 2 from removing items from the work area 6 to the clean area 8. This has the advantage of improving the hygiene level of the surgical information system 2.

[0044] The second processing device 12 is configured to play back the sequence of views 16 on a display device 24 that forms part of the second processing device 12. For example, the display device 24 is a display of the second processing device 12 and can be a computer. By way of example, the playback of the sequence of views 16 serves as a treatment plan for an operator of the surgical instrument 26. The sequence of views 16 can guide the operator or surgeon through a surgical procedure.

[0045] As previously mentioned, the 3D model 18 includes multiple subparts 20a, 20b, and 20c that form the body part depicted in the 3D model 18. The instructions received via the user interface 14 and the first processing device 10 may further include instructions regarding a transparency value for at least one of the subparts 20a, 20b, and / or 20c of the body part. The transparency value may, for example, define the opacity of the subparts 20a, 20b, and 20c in the view 16 on the 3D model 18. By reducing the opacity of at least one of the subparts 20a, 20b, and 20c, a user can identify the structure of the body part that is covered by the subpart in front of them. However, accurate knowledge of the structure of all subparts 20a, 20b, and 20c may be a critical factor for successful surgical treatment.

[0046] The first processing unit 10 may further be configured to additionally store a transparency value for at least one subpart 20a, 20b, 20c in the sequence of views 16. A transparency value may be defined for each individual view 16. This information is stored together with the sequence of views 16 and communicated to the second processing unit 12 for display on the display device 24.

[0047] The first processing unit 10 may further be configured to provide multiple virtual camera positions, each defining a view direction on the 3D model 18. As an example, the first view direction 28 may be oriented perpendicular to the drawing plane. An alternative virtual camera position may define a second view direction 30, for example, oriented toward the drawing plane. The instructions received by the first processing unit 10 may include instructions regarding a selected virtual camera position, and a number of virtual camera positions may be defined for every view 16. In other words, multiple individual virtual camera positions may be defined for each view 16 in the sequence of views 16. The virtual camera positions are additionally stored with the sequence of views 16. The virtual camera positions, together with the sequence of views 16, are communicated from the first processing unit 10 to the second processing unit 12 and displayed on the display device 24 of the second processing unit 12.

[0048] The second processing device 12 can be configured to receive instructions indicating a selected virtual camera position. These instructions can be received via a user interface 32 of the second processing device 12. For example, the user interface 32 can be a keyboard or a microphone. The second processing device 12 can be configured to perform voice recognition to capture the instructions indicating the selected virtual camera position. A user of the second processing device 12 can select an appropriate view from multiple virtual camera positions.

[0049] Defining the virtual camera position may include defining a default virtual camera position, such as the virtual camera position defined by the first view direction 28 shown in Figure 4. When a corresponding command is received by the user interface 32 in the second processing device 12, playback of the sequence of views 16 switches to the default virtual camera position, thereby allowing a user of the surgical information system 2 to be reoriented as needed.

[0050] FIG. 3 is a schematic diagram of a computer-based clinical decision support system (CDSS), implemented, for example, on the first processing device 10. The CDSS implements an artificial intelligence model (AI model). The AI ​​model receives a number of input features via an input interface. The input features are sent to the input layer of the AI ​​model. The AI ​​model performs inference operations on the input features and provides an output at the output layer, which is then output to the output interface. A confidence score is also assigned to the output. The CDSS is configured, among other things, to receive patient records as input features from a patient record. The patient data can be stored in a database. Another input feature can be instrument data, e.g., data related to the operating parameters of the surgical instrument 26. This can be live (time-dependent) operating parameters or static parameters. Another input feature can be data related to surgeon or operator preferences for the surgical instrument 26. Other data related to operator preferences can be related to the playback of the view on the display device 24. Based on these input features, the CDSS performs a classification of the view 16. For this purpose, the CDSS can recognize all possible views and / or virtual camera positions. This is shown in Figure 3 as a dataset totaling 38 views. Based on the input features, the CDSS can perform a preselection of views 16 as output data, shown by a preselected dataset of views 40. Based on the preselected views, the user can further select the desired views 16 in the user interface 14 of the first processing device 10. General and more detailed information about CDSSs is provided at the end of this description.

[0051] Many surgical procedures have milestones and standard procedures are followed. The first processing device 10 is configured to receive instructions including information regarding the selection of at least one view 16 as a milestone view. Which view 16 will reach the goal is defined during planning of the procedure. When a milestone view is displayed in the sequence of views 16, the user of the surgical information system 2 is prompted with the fact that a particular milestone has been reached in the surgical procedure.

[0052] FIG. 1 shows that the surgical instrument 26 is connected to the second processing device 12 via a data line 34. This data line 34 can be implemented as a wired or wireless connection. The second processing device 12 is configured to receive a signal S from the surgical instrument 26 via the data line 34. This signal S indicates a milestone operation or milestone use of the surgical instrument 26. For example, the surgical instrument 26 can generate this trigger signal S when an HF generator (not shown) is operated. In many surgical procedures, use of the HF generator indicates the end of a particular procedure, signifying that a milestone in the surgical procedure has been reached. The trigger signal S triggers the selection of a milestone view in the second processing device 12.

[0053] FIG. 2 shows a flowchart of a sequence provision method for providing a sequence of views 16 on a 3D model 18 of a body part in a surgical environment 4. The surgical environment 4 is shown and described above. The sequence provision method includes steps a), b), c), d), and e). In step a), a 3D model 18 of a body part is provided on a first processing device 10 located in the working area 6. In step b), instructions are received by the first processing device 10, the instructions defining a sequence of views 16 on the 3D model 18, with each view 16 defining an orientation of the 3D model 18. The sequence further defines a chronological order of the views 16. In step c), the sequence of views 16 is stored with the 3D model 18. In step d), the sequence of views 16, together with the 3D model 18, is transmitted to a second processing device 12 located in the clean area 8. In step e), the sequence of views 16 is played on a display device 24 of the second processing device 12.

[0054] The steps indicated by dashed lines are optional steps in the sequence provision method. Step a0) relates to performing view classification using a computer-based clinical decision support system. Step a1) refers to selecting a view for a virtual camera and corresponds to step d1) referring to displaying a view from a selected view direction of the virtual camera.

[0055] Steps d1), d2), and d3) do not necessarily have to be performed in the specified order. They can be performed as alternative steps or in any order. Step d2) refers to the selection of a default virtual camera view. Step d3) refers to the display of a view related to a milestone. This can be triggered by signal S.

[0056] Below we provide general and more detailed information on the implementation of a CDSS (see Figure 3).

[0057] FIG. 3 shows a schematic diagram of an exemplary computer-based clinical decision support system (CDSS) configured to provide a preselection of views based on input data, which may be patient data, instrument data, and / or surgeon preference data. In various embodiments, the CDSS includes an input interface 30 through which, for example, patient-specific input data is provided as one of the input features to an artificial intelligence (AI) model. The AI ​​model runs on a processor 36, which performs inference operations that apply the input data, such as, for example, patient data, instrument data, and / or surgeon preference data, to the AI ​​model to generate a preselection of views. A preselected dataset 40 of views can be output via an output interface, which may be the user interface 14 of the first processing device 10. The system output is communicated to a user, for example, a clinician.

[0058] In some embodiments, the input interface may be a direct data link between the CDSS and one or more medical devices, such as a surgical instrument 26, that generates at least some of the input features. For example, the input interface may transmit input data directly to the CDSS during a therapeutic and / or diagnostic medical procedure. Additionally or alternatively, the input interface may be a classic user interface that facilitates interaction between a user and the CDSS. For example, the input interface may facilitate a user interface through which a user manually enters input data, such as surgeon preferences. The input interface may be the user interface 14 of the first processing device 10. Additionally or alternatively, the input interface may provide the CDSS with access to an electronic patient record from which one or more input features can be extracted. In any of these cases, the input interface is configured to collect one or more of the input data regarding features associated with a particular patient at or before the CDSS is used to evaluate the selection of views to provide a treatment plan.

[0059] Based on one or more input features, processor 36 performs an inference operation using an AI model to generate a preselection of a view. For example, an input interface may deliver input data to an input layer of an AI model, which then propagates the data as input features to an output layer. AI models can provide computer systems with the ability to perform tasks without being explicitly programmed by making inferences based on patterns discovered in analyzing data. AI models explore the study and construction of algorithms (e.g., machine learning algorithms) that can learn from existing data and make predictions about new data. Such algorithms operate by constructing AI models from example training data to make data-driven predictions or judgments, which are expressed as outputs or evaluations.

[0060] There are two general modes of machine learning (ML): supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., examples that associate inputs with outputs or outcomes) to learn the relationship between inputs and outputs. The goal of supervised ML is to learn a function that best approximates the relationship between training inputs and outputs given some training data, so that an ML model can implement that same relationship given an input and generate a corresponding output. Unsupervised ML involves training an ML algorithm with unclassified, unlabeled information, allowing the algorithm to act on that information without guidance. Unsupervised ML is useful for exploratory analysis because it can automatically identify structure in data.

[0061] Common tasks in supervised ML are classification and regression problems. Classification problems, also known as categorization problems, aim to classify an item into one of several categorical values ​​(e.g., is this object an apple or an orange?). Regression algorithms aim to quantify some item (e.g., by assigning a score to the value of some input). Some examples of commonly used supervised ML algorithms include logistic regression (LR), naive Bayes, random forests (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and support vector machines (SVM).

[0062] Some common tasks in unsupervised ML include clustering, representation learning, and density estimation. Some examples of commonly used unsupervised ML algorithms are K-means clustering, principal component analysis, and autoencoders.

[0063] Another type of ML is federated learning (also known as collaborative learning), in which algorithms are trained across multiple distributed devices that maintain local data without exchanging data. This approach contrasts with traditional centralized machine learning techniques, in which all local datasets are uploaded to a single server, as well as more classical distributed approaches that often assume that local data samples are equally distributed. Federated learning enables multiple parties to build a common, robust machine learning model without sharing data, thus addressing important issues such as data privacy, data security, data access rights, and access to heterogeneous data.

[0064] In some examples, the AI ​​model may be trained continuously or periodically before the inference operation is performed by processor 36. Then, during the inference operation, the patient-specific input features provided to the AI ​​model may be propagated from an input layer, through one or more hidden layers, and finally to an output layer corresponding to a selection of views.

[0065] During and / or following the inference operation, the selection of the view may be communicated to the user via a user interface (UI) and / or automatically cause the first processing device 10 to display a pre-selected view.

[0066] All specified features, including those obtained solely from the drawings, and individual features disclosed in combination with other features, are considered to be essential to the invention both alone and in combination. Embodiments according to the invention can be realized by individual features or by a combination of several features. Features associated with the expressions "in particular" or "especially" shall be treated as preferred embodiments. [Explanation of symbols]

[0067] 2. Surgical Information System 4 Surgical environment 6 Working area 8 Clean area 10 First processing device 12 Second processing device 14 User Interface (First Processing Device) 16 Views 18 3D models 20a, 20b, 20c subparts 22 Data Link 24 Display device 26 Surgical instruments 28 First View Direction 30 Second View Direction 32 User Interface (Second Processing Device) 34 Data line 36 processors 38 total views A preselected dataset of 40 views S signal CDSS Computer-based clinical decision support system AI model Artificial intelligence model

Claims

1. 1. A surgical information system for providing a sequence of views on a 3D model of a body part in a surgical environment having a working area and a clean area, comprising: a first processing device located in the working area and a second processing device located in the clean area; The first processing device includes: a) providing a 3D model of a body part to be reproduced; b) receiving instructions to define a sequence of views on the 3D model, such that each view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) storing the sequence of views together with the 3D model; d) transmitting the sequence of views and the 3D model to the second processing device located in the clean area; The second processing device includes: e) a surgical information system configured to play back said sequence of views on a display device forming part of said second processing device.

2. the 3D model comprises a plurality of sub-parts forming the body part, and the instructions include instructions regarding a transparency value of at least one of the sub-parts; The first processing device further comprises: additionally storing the transparency values ​​of the sub-parts in the sequence of views in c) and transmitting the sequence of views according to d); The surgical information system of claim 1 , wherein the second processing device is further configured to play back the sequence of views in accordance with e).

3. The first processing device further comprises: providing a plurality of virtual camera positions, each defining a viewing direction on the 3D model; receiving instructions according to b), including instructions regarding at least one selected virtual camera position; additionally storing the selected virtual camera position in the sequence of views according to c); transmitting the sequence of views in accordance with d); The second processing device further comprises: d1) receiving an instruction indicating the selected virtual camera position; 2. The surgical information system of claim 1, configured to play back the sequence of views from the view direction on the 3D model defined by the selected virtual camera position in accordance with e).

4. the instructions for the at least one selected virtual camera position include selecting a default virtual camera position; The first processing device further comprises: additionally storing the default virtual camera position in the sequence of views in accordance with c) and transmitting the sequence of views in accordance with d); The second processing device further comprises: d2) receiving an instruction indicating a selection of said default virtual camera position; 4. The surgical information system of claim 3, configured to: play back at least one of the views of the sequence from the view direction on the 3D model defined by the selected default virtual camera position in accordance with e).

5. The first processing device further comprises: a0) receiving input data indicative of patient data, instrument data, and / or surgeon preference data; performing the classification of the views using a computer-based clinical decision support system (CDSS) implemented on the first processing device; performing the classification based on the received input data; receiving a plurality of said views via an input interface of said CDSS as input features for an artificial intelligence (AI) model; performing an inference operation by a processor to apply the plurality of views to the AI ​​model to generate the classification of views; 2. The surgical information system of claim 1, configured to perform the classification of the views such that the classification of the views is communicated via a user interface (UI) of the first processing device as a pre-selection of the views, and based thereon the definition of the sequence of views on the 3D model is performed.

6. The first processing device includes: receiving instructions according to b), including instructions regarding the selection of at least one of the views as a milestone view in the sequence of views; additionally storing the milestone view in the sequence of views according to c); transmitting the sequence of views to the second processing device in accordance with d); The second processing device further comprises: d3) receiving an instruction indicating a selection of said milestone view; The surgical information system of claim 1 , configured to reproduce the selected milestone view in accordance with e).

7. The second processing device further comprises:

7. The surgical information system of claim 6, configured to receive a signal from a surgical instrument coupled to the second processing device, the signal indicating a milestone operation or use of the surgical instrument, the signal triggering the selection of the milestone view as an instruction indicating the selection of the milestone view in accordance with d3).

8. 1. A method for providing a sequence of views on a 3D model of a body part in a surgical environment, comprising: the surgical environment includes a work area and a clean area; a) providing a 3D model of a body part to a first processing device located in the working area for reproduction; b) receiving at the first processing device instructions defining a sequence of views on the 3D model, such that each view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) storing said sequence of views together with said 3D model in said first processing device; d) transmitting the sequence of views and the 3D model from the first processing device to a second processing device located in the clean area; e) playing said sequence of views on a display device forming part of said second processing device.

9. 9. The method of claim 8, wherein the 3D model in step a) includes a plurality of sub-parts forming the body part, the instructions received in step b) include instructions regarding a transparency value of at least one of the sub-parts, the transparency value of the sub-part being additionally stored in the sequence of views in step c), and the sequence of views being transmitted in step d) and played back in step e).

10. 9. The method of claim 8, further comprising the step a1) of providing a plurality of virtual camera positions each defining a view direction on the 3D model, wherein the instructions received in step b) include instructions for at least one selected virtual camera position, the selected virtual camera position being additionally stored in the sequence of views of step c), and the sequence of views being transmitted in step d), wherein the method further comprises the step d1) of receiving, at the second processing device, instructions indicating the selected virtual camera positions, and wherein step e) includes playing back the sequence of views from the view directions on the 3D model defined by the selected virtual camera positions.

11. 11. The sequence providing method of claim 10, wherein the instructions regarding the at least one selected virtual camera position include a selection of a default virtual camera position that is additionally saved in the sequence of views in step c), and the sequence of views is transmitted in step d), the sequence providing method further comprising a step d2) of receiving, at the second processing device, instructions indicating the selection of the default virtual camera position, and step e) includes playing back at least one of the views of the sequence of views from the viewing direction on the 3D model defined by the selected default virtual camera position.

12. The method further includes a step a0) of performing classification of the views using a computer-based clinical decision support system (CDSS) implemented in the first processing device, the classification being based on input data received in the further step a0), the instructions indicating patient data, instrument data, and / or data related to surgeon preferences, and the step a0) of performing classification of the views includes: receiving a plurality of said views via an input interface of said CDSS as input features for an artificial intelligence (AI) model; 9. The method of claim 8, further comprising: performing an inference operation by a processor to apply the plurality of views to the AI ​​model to generate the classification of the views; and communicating the classification of the views via a user interface (UI) of the first processing device as a pre-selection of the views, based on which step b) of defining the sequence of the views on the 3D model is performed.

13. 9. The method for providing a sequence of claim 8, wherein the instructions received in step b) include instructions regarding the selection of at least one of the views as a milestone view in the sequence of views, the milestone view being additionally saved in the sequence of views in step c), and the sequence of views being transmitted to the second processing device in step d), the method for providing a sequence further comprising step d3) of receiving an instruction indicating the selection of the milestone view, and step e) including playing back the selected milestone view.

14. 14. The sequence providing method of claim 13, further comprising receiving a signal from a surgical instrument coupled to the second processing device, the signal indicating the milestone action or use of the surgical instrument, the signal triggering the selection of the milestone view in step d3).

15. 1. A method of operating a surgical information system to provide a sequence of views on a 3D model of a body part in a surgical environment having a working area and a clean area, the method comprising: the surgery information system includes a first processing device located in the work area and a second processing device located in the clean area; a) providing a 3D model of a body part to the first processing device for reproduction; b) receiving at the first processing device instructions defining a sequence of views on the 3D model, such that each view defines an orientation of the 3D model and the sequence defines a chronological order of the views on the 3D model; c) storing said sequence of views together with said 3D model in said first processing device; d) transmitting the sequence of views and the 3D model from the first processing device to the second processing device; e) playing said sequence of said views on a display device forming part of said second processing device.

Images