On-table interaction method and system for surgeon

Abstract

The invention discloses an on-table interaction method and system for a surgeon, and relates to the technical field of operation interaction, the system discloses an interaction instruction analysis module, an interaction instruction determination module and an interaction instruction display module, and is provided with the interaction instruction analysis module, the interaction instruction determination module and the interaction instruction display module. The operation process of a surgeon can be analyzed in a video frame extraction processing mode, the interaction instruction analysis sequence is set, each interaction analysis frame can be analyzed according to the instruction habit of the surgeon according to the instruction sequence, and when the instruction intention of each interaction analysis frame is judged, the interaction instruction of the surgeon can be analyzed. Instruction analysis is quickly performed on the next interactive analysis frame, analysis of all instruction types does not need to be performed on each interactive analysis frame, and three-dimensional comprehensive analysis is performed on instructions in the operation process.

Description

Technical Field

[0001] The present invention relates to the field of surgical interaction technology, and more specifically, to an on-stage interactive method and system for surgeons. Background Art

[0002] In a highly complex and tense surgical environment, instant and accurate communication between surgeons is essential. Usually, an operation involves a surgeon and multiple assistants and nurses. In order to facilitate interaction between doctors and nurses, an interactive surgical screen is usually provided during the operation, which can display the effects of the operation and the corresponding vital signs data. In order to keep the operation quiet, the surgeon usually does not speak during the operation, but expresses the corresponding instructions through some actions. When the assistants and nurses do not understand the instructions and execute the corresponding instructions in time, the progress of the operation will be delayed. Therefore, how to improve the efficiency of doctors' interaction during surgery is an urgent problem that needs to be solved. Summary of the invention

[0003] In view of the deficiencies in the prior art, an object of the present invention is to provide an on-stage interactive method and system for surgeons.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] An on-stage interactive system for surgeons, comprising an interactive instruction analysis module, an interactive instruction determination module, and an interactive instruction display module;

[0006] The interactive instruction analysis module obtains the interactive instruction analysis order of the surgeon before the operation begins, and during the operation, records a video of the surgeon's operation process, obtains an interactive analysis frame based on the video, and analyzes the instructions of the interactive analysis frame according to the interactive instruction analysis order;

[0007] The interactive instruction determination module obtains the instruction determination value of the interactive analysis frame each time an interactive instruction analysis log is generated, and determines whether to mark the on-stage interactive instruction based on the result of comparing the instruction determination value with its threshold value;

[0008] The interactive instruction display module is used to periodically mark the sorting display instructions, and sort all the sorting display instructions in descending order according to the numerical values ​​of the interactive instruction sorting values, and display the instruction names of all the sorting display instructions on the surgical interactive screen in the sorting order.

[0009] Furthermore, the analysis order of the surgeon's interactive instructions is obtained by the following method: obtaining all interactive instruction analysis logs of the surgeon before the current system time, obtaining the instruction type of the interactive instruction analysis log, marking the interactive instruction analysis logs of the same instruction type as same-finger analysis logs, summing up the instruction analysis values ​​of all same-finger analysis logs and taking the average to obtain the same-finger analysis mean SAP, obtaining the instruction stability times FZB, and using the formula The analysis order value LST of this type of instruction is obtained, where r1 is the mean coefficient of the same-finger analysis, and r2 is the instruction stability coefficient. All instruction types are sorted in descending order according to the analysis order value, and the interactive instruction analysis order is obtained according to the sorting order.

[0010] Furthermore, the instruction stabilization times FZB are obtained in the following manner: all same-finger analysis logs are sorted in order of analysis time, the instruction analysis values ​​of two adjacent same-finger analysis logs after sorting are calculated by difference and the absolute value is taken to obtain the same-finger analysis fluctuation value, and a same-finger analysis fluctuation threshold is set; when the same-finger analysis fluctuation value is greater than or equal to the same-finger analysis fluctuation threshold, no processing is performed; when the same-finger analysis fluctuation value is less than the same-finger analysis fluctuation threshold, the instruction stabilization times are increased by one and the instruction stabilization times are marked as FZB.

[0011] Furthermore, an interactive analysis frame is obtained based on the video, specifically: based on a preset time interval t, at each time node of the preset time interval, the video is subjected to frame extraction processing to obtain an interactive analysis frame.

[0012] Further, the instructions of the interactive analysis frame are analyzed according to the interactive instruction analysis order, specifically: the instructions of the interactive analysis frame are analyzed according to the interactive instruction analysis order, and each time an instruction is analyzed, an interactive instruction analysis log of the interactive instruction is generated; the interactive instruction analysis log includes the surgeon's work number, the interactive analysis frame number, the instruction type, the instruction analysis value, and the analysis time;

[0013] Each time an interactive instruction analysis log is generated, the instruction analysis value of the interactive instruction analysis log is obtained, and the instruction analysis threshold is set. When the instruction analysis value is greater than the instruction analysis threshold, the instruction type is marked as an interactive prominent instruction. When the instruction analysis value is less than or equal to the instruction analysis threshold, the instruction type is marked as an interactive non-present instruction.

[0014] Furthermore, the instruction analysis value of the interactive instruction analysis log is obtained in the following manner: obtain the instruction type of the interactive instruction analysis log, obtain the instruction analysis model of the instruction type, perform feature extraction on the interactive analysis frame to obtain the interactive analysis feature, input the interactive analysis feature into the instruction analysis model, and the instruction analysis model outputs the instruction analysis value.

[0015] Further, the instruction determination value of the interactive analysis frame is obtained in the following manner: all interactive prominent instructions and interactive non-appearing instructions of the interactive analysis frame are obtained, all interactive prominent instructions are grouped together in pairs, the instruction analysis values ​​of the two interactive prominent instructions in the same group are calculated by difference to obtain a prominent gap value, a prominent gap threshold is set, when the prominent gap value is greater than or equal to the prominent gap threshold, the number of prominent gaps is increased by one, when the prominent gap value is less than the prominent gap threshold, no processing is performed, and the number of prominent gaps is marked as SPK, all interactive non-appearing instructions are grouped together in pairs, the instruction analysis values ​​of the two interactive non-appearing instructions in the same group are summed to obtain an interactive non-appearing sum value, an interactive non-appearing sum threshold is set, when the interactive non-appearing sum value is greater than or equal to the interactive non-appearing sum threshold, no processing is performed, when the interactive non-appearing sum value is less than the interactive non-appearing sum threshold, the number of interactive non-appearing times is increased by one, and the number of interactive non-appearing times is marked as ZGP, and the formula is used. The instruction determination value qdz of the interactive analysis frame is obtained, wherein g1 is the coefficient of the number of prominent gaps, and g2 is the coefficient of the number of interactive non-appearances.

[0016] Furthermore, based on the comparison result between the instruction determination value and its threshold, it is determined whether to mark the on-stage interactive instruction, specifically: the instruction determination threshold is set, when the instruction determination value is greater than or equal to the instruction determination threshold, the interactive prominent instruction with the largest instruction analysis value in the interactive analysis frame is marked as the on-stage interactive instruction, the current time is marked as the on-stage interactive moment, the on-stage interactive moment is assigned to the on-stage interactive instruction, and the instructions of the next interactive analysis frame are prepared to be analyzed according to the interactive instruction analysis order; when the instruction determination value is less than the instruction determination threshold, the next instruction of the interactive analysis frame is analyzed according to the interactive instruction analysis order.

[0017] Furthermore, the sorting and displaying instructions are periodically marked, specifically: based on a preset time interval T, at each time node of the preset time interval, the interactive instruction sorting values ​​of various instructions are obtained, and an interactive instruction sorting threshold is set. When the interactive instruction sorting value is greater than or equal to the interactive instruction sorting threshold, the instruction is marked as a sorting and displaying instruction, and when the interactive instruction sorting value is less than the interactive instruction sorting threshold, no processing is performed;

[0018] The interactive instruction ranking value of the instruction is obtained in the following way: obtain the total number of times the same instruction is marked as an on-stage interactive instruction within a time length of T, and mark it as TSM; obtain the corresponding on-stage interactive moment of the instruction marked as an on-stage interactive instruction within a time length of T; sort all the on-stage interactive moments in chronological order; calculate the time difference between the two adjacent on-stage interactive moments after sorting and take the absolute value to obtain the on-stage interactive interval; sum up all the on-stage interactive intervals and take the average to obtain the on-stage interactive average interval SL; use the formula The interactive instruction ranking value GC of this type of instruction is obtained, wherein k1 is the total number of interactions on the stage coefficient, and k2 is the average interval coefficient of interactions on the stage.

[0019] Furthermore, a method for on-stage interaction for a surgeon includes the following steps:

[0020] Step 1: Before the operation begins, obtain the interactive command analysis order of the surgeon. During the operation, record a video of the surgeon's operation process, obtain an interactive analysis frame based on the video, and analyze the instructions of the interactive analysis frame according to the interactive command analysis order;

[0021] Step 2: when each interactive command analysis log is generated, the command determination value of the interactive analysis frame is obtained, and based on the result of comparing the command determination value with its threshold, it is determined whether to mark the on-stage interactive command;

[0022] Step 3: Periodically mark the sorting display instructions, and sort all the sorting display instructions in descending order according to the numerical values ​​of the interactive instruction sorting values, and display the instruction names of all the sorting display instructions on the surgical interactive screen in the sorting order.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. The method of the present invention can quickly and accurately determine the instructions that the surgeon wants to express, and display various interactive instructions according to priority on the surgical interactive screen, so that assistant doctors and nurses can execute relevant instructions in sequence;

[0025] 2. An interactive instruction analysis module, an interactive instruction determination module and an interactive instruction display module are set up. The surgical process of the surgeon can be analyzed by means of video frame extraction processing. The interactive instruction analysis order can be set up. Each interactive analysis frame can be analyzed in the order of instructions according to the surgeon's instruction habits. When the instruction intention of each interactive analysis frame is determined, the next interactive analysis frame can be quickly analyzed. There is no need to analyze all types of instructions for each interactive analysis frame, and a three-dimensional comprehensive analysis of the instructions during the operation can be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A flowchart of an on-stage interactive method for a surgeon;

[0027] Figure 2 A flowchart for determining whether to mark an interactive instruction on the stage;

[0028] Figure 3 A flowchart for obtaining instruction analysis values. DETAILED DESCRIPTION

[0029] Example 1

[0030] Reference Figure 1 , an on-stage interactive method for a surgeon, comprising the following steps:

[0031] Step 1: Before the operation begins, obtain the interactive command analysis order of the surgeon. During the operation, record a video of the surgeon's operation process, obtain an interactive analysis frame based on the video, and analyze the instructions of the interactive analysis frame according to the interactive command analysis order;

[0032] Step 2: when each interactive command analysis log is generated, the command determination value of the interactive analysis frame is obtained, and based on the result of comparing the command determination value with its threshold, it is determined whether to mark the on-stage interactive command;

[0033] Step 3: Periodically mark the sorting display instructions, and sort all the sorting display instructions in descending order according to the numerical values ​​of the interactive instruction sorting values, and display the instruction names of all the sorting display instructions on the surgical interactive screen in the sorting order.

[0034] Through the above method, the instructions that the surgeon wants to express can be quickly and accurately determined, and various interactive instructions can be displayed on the surgical interactive screen according to priority, making it convenient for assistant doctors and nurses to execute relevant instructions in sequence.

[0035] Example 2

[0036] Reference Figure 2-Figure 3 , an on-stage interactive system for surgeons, including an interactive instruction analysis module, an interactive instruction determination module, and an interactive instruction display module.

[0037] Interactive instruction analysis module: Before the operation begins, obtain the interactive instruction analysis order of the surgeon.

[0038] The analysis order of the surgeon's interactive instructions is obtained in the following way: obtain all interactive instruction analysis logs of the surgeon before the current system time, obtain the instruction type of the interactive instruction analysis log, mark the interactive instruction analysis logs of the same instruction type as same-finger analysis logs, sum and average the instruction analysis values ​​of all same-finger analysis logs to obtain the same-finger analysis mean SAP, sort all same-finger analysis logs in the order of analysis time, calculate the difference between the instruction analysis values ​​of two adjacent same-finger analysis logs after sorting and take the absolute value to obtain the same-finger analysis fluctuation value, set the same-finger analysis fluctuation threshold (system threshold, modified and adjusted according to actual needs), when the same-finger analysis fluctuation value is greater than or equal to the same-finger analysis fluctuation threshold, no processing is performed, when the same-finger analysis fluctuation value is less than the same-finger analysis fluctuation threshold, increase the instruction stability times by one, mark the instruction stability times as FZB, and use the formula The analysis order value LST of this type of instruction is obtained, where r1 is the mean coefficient of the same-finger analysis, r2 is the instruction stability coefficient, the value of r1 is 1.38, and the value of r2 is 1.02. All instruction types are sorted in descending order according to the numerical value of the analysis order value, and the interactive instruction analysis order is obtained according to the sorting order (the interactive instruction analysis order is shown by examples: 1. Scalpel transfer instruction; 2. Suturing instruction; 3. Surgical field clearing instruction; the above are just examples of the interactive instruction analysis order, and the specific interactive instruction analysis order includes but is not limited to the above).

[0039] During the operation, the surgical process of the surgeon is recorded, and based on the preset time interval t (t is the duration of each time interval), the video is frame-extracted at each time node of the preset time interval to obtain an interactive analysis frame, and the instructions of the interactive analysis frame are analyzed in the interactive instruction analysis order. Each time an instruction is analyzed, an interactive instruction analysis log of the interactive instruction is generated. The interactive instruction analysis log includes the surgeon's work number, the interactive analysis frame number (each interactive analysis frame corresponds to an independent and unique number), instruction type, instruction analysis value, and analysis time (generation time of the interactive instruction analysis log).

[0040] Each time an interactive instruction analysis log is generated, the instruction analysis value of the interactive instruction analysis log is obtained, and the instruction analysis threshold value (system threshold, modified and adjusted according to actual needs) is set. When the instruction analysis value is greater than the instruction analysis threshold value, the instruction type is marked as an interactive prominent instruction. When the instruction analysis value is less than or equal to the instruction analysis threshold value, the instruction type is marked as an interactive non-appearing instruction.

[0041] Interactive instruction determination module: each time an interactive instruction analysis log is generated, the instruction determination value of the interactive analysis frame is obtained, and an instruction determination threshold (system threshold, modified and adjusted according to actual needs) is set; when the instruction determination value is greater than or equal to the instruction determination threshold, the interactive prominent instruction with the largest instruction analysis value in the interactive analysis frame is marked as an on-stage interactive instruction, the current time is marked as the on-stage interactive moment, the on-stage interactive moment is assigned to the on-stage interactive instruction, and the instructions of the next interactive analysis frame are prepared to be analyzed in the interactive instruction analysis order; when the instruction determination value is less than the instruction determination threshold, the next instruction of the interactive analysis frame is analyzed in the interactive instruction analysis order (if all instructions in the interactive instruction analysis order are analyzed and the instruction determination value is less than the instruction determination threshold, it means that there is no instruction expression in the interactive analysis frame).

[0042] The instruction determination value of the interactive analysis frame is obtained in the following manner: obtain all interactive prominent instructions and interactive non-appearing instructions of the interactive analysis frame, group all interactive prominent instructions into one group, perform difference calculation on the instruction analysis values ​​of two interactive prominent instructions in the same group to obtain a prominent gap value, set a prominent gap threshold (system threshold, modified and adjusted according to actual needs), when the prominent gap value is greater than or equal to the prominent gap threshold, increase the number of prominent gaps by one, when the prominent gap value is less than the prominent gap threshold, do not process it, mark the number of prominent gaps as SPK, group all interactive non-appearing instructions into one group, sum the instruction analysis values ​​of two interactive non-appearing instructions in the same group to obtain an interactive non-appearing sum value, set an interactive non-appearing sum threshold (system threshold, modified and adjusted according to actual needs), when the interactive non-appearing sum value is greater than or equal to the interactive non-appearing sum threshold, do not process it, when the interactive non-appearing sum value is less than the interactive non-appearing sum threshold, increase the number of interactive non-appearing times by one, mark the number of interactive non-appearing times as ZGP, and use the formula The instruction determination value qdz of the interactive analysis frame is obtained, where g1 is the coefficient of the number of prominent gaps, g2 is the coefficient of the number of interactive non-appearances, the value of g1 is 0.82, and the value of g2 is 0.31.

[0043] The instruction analysis value of the interactive instruction analysis log is obtained in the following manner: obtain the instruction type of the interactive instruction analysis log, obtain the instruction analysis model of the instruction type, perform feature extraction on the interactive analysis frame to obtain the interactive analysis feature, input the interactive analysis feature into the instruction analysis model, and the instruction analysis model outputs the instruction analysis value.

[0044] The types of instructions include but are not limited to scalpel transfer instructions, suturing instructions, surgical field cleaning instructions, and operating table height adjustment instructions. Different types of instructions correspond to different instruction analysis models, but the construction method of each instruction analysis model is the same. It only needs to replace the training data corresponding to the model accordingly. The value range of instruction analysis values ​​of different instruction analysis models is the same. The following will take the scalpel transfer instruction as an example to introduce the construction method of the instruction analysis model: obtain j groups of interactive analysis features, which can be real collection or virtual settings, build a deep learning model, use the interactive analysis features as training data for the deep learning model, assign instruction analysis values ​​to each training data, divide the training data into training set and verification set in a ratio of 2:3, and iterate training through training and verification to obtain the instruction analysis model of the scalpel transfer instruction. The value range of the instruction analysis value is (1~10). The closer the instruction analysis value is to 10, the closer the surgeon's behavior is to expressing the scalpel transfer instruction. The closer the instruction analysis value is to 1, the less the surgeon's behavior is to expressing the scalpel transfer instruction.

[0045] Interactive instruction display module: based on the preset time interval T (T is the length of each time interval), at each time node of the preset time interval, the interactive instruction ranking value of various instructions is obtained, and the interactive instruction ranking threshold (system threshold, modified and adjusted according to actual needs) is set. When the interactive instruction ranking value is greater than or equal to the interactive instruction ranking threshold, the instruction is marked as a ranking display instruction. When the interactive instruction ranking value is less than the interactive instruction ranking threshold, no processing is performed. All ranking display instructions are sorted in descending order according to the numerical value of the interactive instruction ranking value, and the instruction names of all ranking display instructions are displayed on the surgical interactive screen in the sorting order (for example, the sorting order of the sorting display instructions is 1, scalpel transfer instruction; 2, surgical field clearing instruction; then the first line on the surgical interactive screen displays the scalpel transfer instruction, and the second line displays the surgical field clearing instruction); the assistant doctor can execute the corresponding instructions according to the instruction order on the surgical interactive screen.

[0046] The interactive instruction ranking value of the instruction is obtained in the following way: obtain the total number of times the same instruction is marked as an on-stage interactive instruction within a time length of T, and mark it as TSM; obtain the corresponding on-stage interactive moment of the instruction marked as an on-stage interactive instruction within a time length of T; sort all the on-stage interactive moments in chronological order; calculate the time difference between the two adjacent on-stage interactive moments after sorting and take the absolute value to obtain the on-stage interactive interval; sum up all the on-stage interactive intervals and take the average to obtain the on-stage interactive average interval SL; use the formula The interactive instruction ranking value GC of this type of instruction is obtained, where k1 is the total number of interactions on the stage coefficient, k2 is the average interval coefficient of interactions on the stage, the value of k1 is 0.63, and the value of k2 is 0.48.

[0047] By setting up an interactive instruction analysis module, an interactive instruction determination module and an interactive instruction display module, the surgical process of the surgeon can be analyzed by means of video frame extraction processing. By setting the interactive instruction analysis order, each interactive analysis frame can be analyzed in the instruction sequence according to the surgeon's instruction habits. When the instruction intention of each interactive analysis frame is determined, the next interactive analysis frame can be quickly analyzed. There is no need to analyze all types of instructions for each interactive analysis frame, and a three-dimensional comprehensive analysis of the instructions during the operation can be performed.

[0048] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0049] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0050] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0051] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0052] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0053] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0054] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0055] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An on-stage interactive system for surgeons, characterized in that: It includes an interactive instruction analysis module, an interactive instruction determination module, and an interactive instruction display module; The interactive instruction analysis module obtains the interactive instruction analysis order of the surgeon before the operation begins, and during the operation, records a video of the surgeon's operation process, obtains an interactive analysis frame based on the video, and analyzes the instructions of the interactive analysis frame according to the interactive instruction analysis order; The interactive instruction determination module obtains the instruction determination value of the interactive analysis frame each time an interactive instruction analysis log is generated, and determines whether to mark the on-stage interactive instruction based on the result of comparing the instruction determination value with its threshold value; The interactive instruction display module is used to periodically mark the sorting display instructions, and sort all the sorting display instructions in descending order according to the numerical values ​​of the interactive instruction sorting values, and display the instruction names of all the sorting display instructions on the surgical interactive screen in the sorting order.

2. The on-stage interactive system for surgeons according to claim 1, characterized in that: The analysis order of the surgeon's interactive instructions is obtained by the following method: obtain all the surgeon's interactive instruction analysis logs before the current system time, obtain the instruction type of the interactive instruction analysis log, mark the interactive instruction analysis logs of the same instruction type as same-finger analysis logs, sum up the instruction analysis values ​​of all same-finger analysis logs and take the average to obtain the same-finger analysis mean SAP, obtain the instruction stability number FZB, and use the formula The analysis order value LST of this type of instruction is obtained, where r1 is the mean coefficient of the same-finger analysis, and r2 is the instruction stability coefficient. All instruction types are sorted in descending order according to the analysis order value, and the interactive instruction analysis order is obtained according to the sorting order.

3. The on-stage interactive system for surgeons according to claim 2, characterized in that: The instruction stabilization times FZB are obtained in the following way: all the same-finger analysis logs are sorted in the order of analysis time, the instruction analysis values ​​of two adjacent same-finger analysis logs after sorting are calculated by difference and the absolute value is taken to obtain the same-finger analysis fluctuation value, and the same-finger analysis fluctuation threshold is set. When the same-finger analysis fluctuation value is greater than or equal to the same-finger analysis fluctuation threshold, no processing is performed. When the same-finger analysis fluctuation value is less than the same-finger analysis fluctuation threshold, the instruction stabilization times are increased by one and the instruction stabilization times are marked as FZB.

4. The on-stage interactive system for surgeons according to claim 1, characterized in that: The interactive analysis frame is obtained based on the video, specifically: based on the preset time interval t, at each time node of the preset time interval, the video is subjected to frame extraction processing to obtain an interactive analysis frame.

5. The on-stage interactive system for surgeons according to claim 1, characterized in that: Analyze the instructions of the interactive analysis frame according to the interactive instruction analysis order, specifically: analyze the instructions of the interactive analysis frame according to the interactive instruction analysis order, and generate an interactive instruction analysis log of the interactive instruction each time an instruction is analyzed; the interactive instruction analysis log includes the surgeon's work number, the interactive analysis frame number, the instruction type, the instruction analysis value, and the analysis time; Each time an interactive instruction analysis log is generated, the instruction analysis value of the interactive instruction analysis log is obtained, and the instruction analysis threshold is set. When the instruction analysis value is greater than the instruction analysis threshold, the instruction type is marked as an interactive prominent instruction. When the instruction analysis value is less than or equal to the instruction analysis threshold, the instruction type is marked as an interactive non-present instruction.

6. The on-stage interactive system for surgeons according to claim 5, characterized in that: The instruction analysis value of the interactive instruction analysis log is obtained in the following manner: obtain the instruction type of the interactive instruction analysis log, obtain the instruction analysis model of the instruction type, perform feature extraction on the interactive analysis frame to obtain the interactive analysis feature, input the interactive analysis feature into the instruction analysis model, and the instruction analysis model outputs the instruction analysis value.

7. The on-table interactive system for surgeons according to claim 1, characterized in that: The instruction determination value of the interactive analysis frame is obtained in the following manner: obtain all interactive prominent instructions and interactive non-appearing instructions of the interactive analysis frame, group all interactive prominent instructions into a group, perform difference calculation on the instruction analysis values ​​of the two interactive prominent instructions in the same group to obtain a prominent gap value, set a prominent gap threshold, when the prominent gap value is greater than or equal to the prominent gap threshold, increase the number of prominent gaps by one, when the prominent gap value is less than the prominent gap threshold, do not process it, mark the number of prominent gaps as SPK, group all interactive non-appearing instructions into a group, sum the instruction analysis values ​​of the two interactive non-appearing instructions in the same group to obtain an interactive non-appearing sum value, set an interactive non-appearing sum threshold, when the interactive non-appearing sum value is greater than or equal to the interactive non-appearing sum threshold, do not process it, when the interactive non-appearing sum value is less than the interactive non-appearing sum threshold, increase the number of interactive non-appearing times by one, mark the number of interactive non-appearing times as ZGP, and use the formula The instruction determination value qdz of the interactive analysis frame is obtained, wherein g1 is the coefficient of the number of prominent gaps, and g2 is the coefficient of the number of interactive non-appearances.

8. The on-table interactive system for surgeons according to claim 1, characterized in that: Based on the comparison result between the instruction determination value and its threshold, determine whether to mark the on-stage interactive instruction, specifically: set the instruction determination threshold, when the instruction determination value is greater than or equal to the instruction determination threshold, mark the interactive prominent instruction with the largest instruction analysis value in the interactive analysis frame as the on-stage interactive instruction, mark the current time as the on-stage interactive moment, assign the on-stage interactive moment to the on-stage interactive instruction, and prepare to analyze the instructions of the next interactive analysis frame according to the interactive instruction analysis order; when the instruction determination value is less than the instruction determination threshold, analyze the next instruction of the interactive analysis frame according to the interactive instruction analysis order.

9. The on-table interactive system for surgeons according to claim 1, characterized in that: Periodically mark the sorting and displaying instructions, specifically: based on the preset time interval T, at each time node of the preset time interval, obtain the interactive instruction sorting value of various instructions, set the interactive instruction sorting threshold, when the interactive instruction sorting value is greater than or equal to the interactive instruction sorting threshold, mark the instruction as a sorting and displaying instruction, when the interactive instruction sorting value is less than the interactive instruction sorting threshold, do not process it; The interactive instruction ranking value of the instruction is obtained in the following way: obtain the total number of times the same instruction is marked as an on-stage interactive instruction within a time length of T, and mark it as TSM; obtain the corresponding on-stage interactive moment of the instruction marked as an on-stage interactive instruction within a time length of T; sort all the on-stage interactive moments in chronological order; calculate the time difference between the two adjacent on-stage interactive moments after sorting and take the absolute value to obtain the on-stage interactive interval; sum up all the on-stage interactive intervals and take the average to obtain the on-stage interactive average interval SL; use the formula The interactive instruction ranking value GC of this type of instruction is obtained, wherein k1 is the total number of interactions on the stage coefficient, and k2 is the average interval coefficient of interactions on the stage.

10. An on-stage interactive method for surgeons, applied to an on-stage interactive system for surgeons as claimed in claim 1, characterized in that: The steps include: Step 1: Before the operation begins, obtain the interactive command analysis order of the surgeon. During the operation, record a video of the surgeon's operation process, obtain an interactive analysis frame based on the video, and analyze the instructions of the interactive analysis frame according to the interactive command analysis order; Step 2: when each interactive command analysis log is generated, the command determination value of the interactive analysis frame is obtained, and based on the result of comparing the command determination value with its threshold, it is determined whether to mark the on-stage interactive command; Step 3: Periodically mark the sorting display instructions, and sort all the sorting display instructions in descending order according to the numerical values ​​of the interactive instruction sorting values, and display the instruction names of all the sorting display instructions on the surgical interactive screen in the sorting order.

Images