Programs, information processing devices, methods, and systems

Abstract

This technology quantifies and visualizes the intuitive information that was previously the tacit knowledge of the practitioner, enabling objective decision-making support. [Solution] A program for operating a computer 20 comprising a processor 29 and memories 25 and 26, wherein the memories 25 and 26 store a trained model that takes acoustic data acquired in an environment where vascular treatment is being performed as input and outputs risk assessment information regarding vascular treatment procedures. The program causes the processor 29 to perform a first step of receiving acoustic data, a second step of inputting the acoustic data received in the first step into the trained model and acquiring risk assessment information output from this trained model, and a third step of outputting the risk assessment information acquired in the second step in a form that can be perceived by humans.

Description

Technical Field

[0001] The present disclosure relates to a program, an information processing apparatus, a method, and a system.

Background Art

[0002] As a treatment method for calcified lesions of blood vessels, a rotary atherectomy device such as a rotabrator is used. However, it highly depends on the experience of the operator, and the technique involves a risk of complications. In addition, the difficulty of the technique is high, and since it depends on tacit knowledge such as the operator's sense, it has been difficult to objectively evaluate risks and provide education.

[0003] On the other hand, a technique has been disclosed in which the driving sound of a rotabrator is recorded, the acoustic data is converted into a spectrogram and imaged, and analyzed using an autoencoder (AI) (Non-Patent Document 1). In this technique, it has been shown that unique frequency patterns are observed during normal times, during burr entrapment which is a complication, and during guidewire transection, respectively, suggesting the possibility of risk detection by acoustic analysis.

Prior Art Documents

Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, although the technique disclosed in Non-Patent Document 1 has shown the effectiveness of acoustic analysis, it remains at individual analysis, and there is no mechanism for aggregating and utilizing a plurality of technique data. Therefore, there have been the following problems. • Acoustic data of the procedures were not systematically accumulated and shared, making it difficult to continuously improve the AI ​​learning model and to objectively evaluate the procedures of each practitioner. • The procedure relied heavily on the surgeon's intuition, making it difficult to educate young doctors on specific risk avoidance methods and to standardize the technique.

[0006] The purpose of this disclosure is to quantify and visualize the intuitive information that has previously been the tacit knowledge of the practitioner, thereby enabling objective decision-making support. [Means for solving the problem]

[0007] To solve the above problems, a program according to one aspect of this disclosure is a program for operating a computer comprising a processor and memory, which supports a procedure related to vascular treatment using a rotary atherectomy device. The memory stores a trained model that takes acoustic data acquired in an environment where vascular treatment is being performed as input and outputs risk assessment information about the procedure related to vascular treatment. The program causes the processor to perform a first step of receiving acoustic data, a second step of inputting the acoustic data received in the first step into the trained model and acquiring risk assessment information output from this trained model, and a third step of outputting the risk assessment information acquired in the second step in a form that can be perceived by a human. [Effects of the Invention]

[0008] According to this disclosure, it is possible to quantify and visualize sensory information that was previously implicit knowledge of the operator, thereby providing objective support for decision-making. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows the general appearance of the rotary atherectomy device related to this disclosure. [Figure 2] This figure shows the general configuration of the rotary atherectomy device related to this disclosure. [Figure 3]This is a diagram illustrating the operation of the rotary atherectomy device related to this disclosure. [Figure 4] This figure illustrates the outline of the procedure using the rotary atherectomy device described herein. [Figure 5] This figure shows the overall configuration of a system according to one embodiment. [Figure 6] This figure shows the functional configuration of a terminal device according to one embodiment. [Figure 7] This figure shows the functional configuration of a server according to one embodiment. [Figure 8] This figure shows an example of the data structure of a patient database according to one embodiment. [Figure 9] This figure shows an example of the data structure of a surgical database according to one embodiment. [Figure 10] This figure shows an example of the data structure of an acoustic data database according to one embodiment. [Figure 11] A flowchart showing an example of the processing flow in a system according to one embodiment. [Figure 12] This flowchart shows another example of the processing flow in a system according to one embodiment. [Figure 13] This flowchart shows yet another example of the processing flow in a system according to one embodiment. [Figure 14] This flowchart shows yet another example of the processing flow in a system according to one embodiment. [Figure 15] This figure shows an example of a spectrogram image in a system according to one embodiment. [Figure 16] A block diagram showing the basic hardware configuration of Computer 90. [Modes for carrying out the invention]

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all the drawings for describing the embodiments, common components are denoted by the same reference numerals, and repeated descriptions are omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Also, not all of the components shown in the embodiments are essential components of the present disclosure. Further, each drawing is a schematic diagram and is not necessarily strictly illustrated.

[0011] Also, in the following description, a "processor" is one or more processors. At least one processor is typically a microprocessor such as a CPU (Central Processing Unit), but may also be another type of processor such as a GPU (Graphics Processing Unit). At least one processor may be single-core or multi-core.

[0012] Also, at least one processor may be a processor in a broad sense such as a hardware circuit (e.g., FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0013] Also, in the following description, expressions such as "xxx table" may be used to describe information from which an output is obtained for an input, but this information may be data of any structure or a learning model such as a neural network that generates an output for an input. Therefore, "xxx table" can be referred to as "xxx information".

[0014] Also, in the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0015] Furthermore, in the following explanation, the subject of the process may sometimes be "program," but since a program is executed by a processor and performs defined processes using the memory and / or interface as appropriate, the subject of the process may also be the processor (or a device such as a controller that has that processor).

[0016] The program may be installed on a device such as a computer, or it may reside on a program distribution server or a computer-readable (e.g., non-temporary) recording medium. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0017] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs, CPUs, conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0018] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0019] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0020] Furthermore, in the following explanation, identification numbers are used as identification information for various objects, but other types of identification information (for example, identifiers including letters or symbols) may also be used.

[0021] Furthermore, in the following explanations, when describing similar elements without distinction, a reference code (or a common code among reference codes) may be used, and when describing similar elements with distinction, the element's identification number (or reference code) may be used.

[0022] Furthermore, in the following explanation, only control lines and information lines deemed necessary for the explanation are shown, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0023] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the terminal device 10 and the server 20, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0024] <0 System Overview> The following outlines the system relating to this disclosure. However, the following explanation should not be interpreted restrictively, and the content of this disclosure should be understood based on the disclosures herein and the ordinary technical knowledge and common sense of those skilled in the art.

[0025] The system described herein is intended to support vascular treatment procedures using a rotary atherectomy device. Therefore, first, an overview of the rotary atherectomy device will be explained with reference to Figures 1 to 4 (Reference: Japanese Patent Publication No. 2021-517840, Japanese Patent Publication No. 2025-500642).

[0026] Figure 1 shows the external appearance of a surgical instrument called a rotablator, which is an example of a rotary atherectomy device, and Figure 2 is a diagram illustrating the overview of the rotablator. The rotablator widens the narrowed portion of the coronary artery by physically removing deposits called plaque, which are caused by arteriosclerosis in the coronary arteries of the heart, by scraping them off with a drill introduced into the coronary artery via a catheter.

[0027] Cardiovascular and peripheral artery diseases arise from the accumulation of atherosclerotic material on the inner walls of blood vessels, which can lead to a condition known as atherosclerosis. Atherosclerosis and other vascular deposits can restrict blood flow and cause ischemia in the patient's heart, the patient's leg vascular system, the patient's carotid arteries, etc. Such ischemia can cause pain, swelling, non-healing wounds, amputations, stroke, myocardial infarction, and / or other conditions.

[0028] Atherosclerosis can have a wide variety of properties; some deposits are relatively soft, while others are fibrous and / or calcified. In the latter case, the deposits are sometimes called plaques. Atherosclerosis occurs spontaneously as a result of aging, but can also be aggravated by factors such as diet, high blood pressure, genetics, and vascular damage. Atherosclerosis can be treated in a variety of ways, including various catheter-based approaches that may rely on medication, bypass surgery, and / or endovascular dilation or removal of atherosclerotic vessels or other substances that occlude the vessels. Atherectomy is a catheter-based medical intervention that can be used to treat atherosclerosis.

[0029] Atheromatosis is a medical intervention performed to restore blood flow through a portion of a patient's vascular system that is blocked by plaque or other material (e.g., blocked by an obstruction). In an atheromatosis procedure, a device at the end of a drive shaft is used to take in or remove (e.g., ablation, polishing, cutting, shaving, etc.) plaque or other material from the patient's blood vessels (such as arteries and veins). In some cases, the device at the end of the drive shaft may be able to polish, and / or otherwise, is configured to remove plaque from the vessel wall or other obstruction as the device rotates and engages with the plaque or other obstruction within the vessel.

[0030] Figure 1 shows a rotary atherectomy system 310. The rotary atherectomy system 310 may be electrically driven, pneumatically driven, and / or driven by one or more other suitable means. Additional or alternative components to those exemplified and described herein may be used in the operation of the rotary atherectomy system 310.

[0031] The rotary atherectomy system 310 may include a drive assembly 312 and a control unit 314 (e.g., a controller). The drive assembly 312 may have, among other elements, a forward assembly 316 and a rotary assembly 317. Although the control unit 314 is shown as separate from the drive assembly 312 in Figure 1, the functions of the control unit 314 and the drive assembly 312 may be incorporated into a single component (e.g., the forward assembly 316 or another suitable single component).

[0032] The rotating assembly 317 may comprise a drive shaft 318 (e.g., a flexible drive shaft or other suitable drive shaft, or an elongated member made of the same), a rotating device 320 (e.g., a rotating tip or other rotating device), and an elongated member 322 having a first end (e.g., a proximal end), a second end (e.g., a distal end), and an elongated member 322 extending from the first end to the second end and having a lumen for receiving the drive shaft 318. In some cases, the elongated member 322 may be an elongated tubular member. The rotating device 320 may have a rough or sharp surface to polish, ablate, cut, or shave plaque on the vessel wall or other obstructions within the vessel when it rotates.

[0033] The forward assembly 316 may include a knob 323, a housing 326, a drive mechanism, and / or one or more other suitable components. The housing 326 can at least partially house the drive mechanism, and the knob 323 is at least partially accessible from outside the housing 326. The drive mechanism may be, or include, a motor (e.g., an electric motor, an air motor, or other suitable motor) at least partially housed within the housing 326 and connected to the knob 323, the drive shaft 318, and the control unit 314. The knob 323 may be configured to move forward along a longitudinal path to advance the drive mechanism 34 and the rotary assembly 317 longitudinally. The drive mechanism may be coupled to the drive shaft 318 in a suitable manner, including but not limited to welding, clamp connections, adhesive, screwing, and / or other suitable couplings configured to withstand rotational speed and rotational force. Since the drive shaft 318 can rotate over a wide range of speeds, the coupling between the drive mechanism and the drive shaft 318 can be configured to withstand such rotational speeds and the associated forces.

[0034] The drive shaft 318 can be formed from one or more different materials. For example, the drive shaft 318 may be formed from one or more different materials, including steel, stainless steel, other metals, polymers, and / or other suitable materials.

[0035] The drive shaft 318 may have an appropriate diameter and / or length for passing through the patient's vascular structure. The diameter and / or length of the drive shaft 318 may depend on the dimensions of the lumen of the elongated member 322, the dimensions of the patient's blood vessels through which it passes, and / or one or more other appropriate factors.

[0036] The rotating device 320 may have an outer circumference equal to or greater than the distal diameter of the drive shaft 318 and / or the elongated member 322. Alternatively, the rotating device 320 may have an outer circumference smaller than the diameter of the drive shaft 318 and / or the elongated member 322. The rotating device 320 may have a symmetrical design to penetrate equally and sufficiently in both directions of rotation, but this is not required, and the rotating device 320 may be configured to penetrate in only one direction.

[0037] The rotating device 320 can be coupled to the drive shaft 318. If the drive shaft 318 has a first end (e.g., a proximal end portion) and a second end (e.g., a distal end portion), the rotating device 320 may be coupled to the drive shaft 318 at or near the second end. In some cases, the rotating device 320 may be located at or adjacent to the end of the second end of the drive shaft 318.

[0038] The rotating device 320 can be coupled to the drive shaft 318 in any way. For example, the rotating device 320 can be coupled to the drive shaft 318 by adhesive, screwing, welding, clamping, and / or other suitable connections configured to withstand rotational speed and force. As with respect to the connection between the drive shaft 318 and the drive mechanism described above, the drive shaft 318 and / or the rotating device 320 can rotate at speeds of zero (0) RPM to 250,000 RPM or more, so the drive shaft 318 and the rotating device 320 can be configured to withstand such rotational speed and associated forces.

[0039] In some cases, as shown in Figure 2, the forward assembly 316 may include a motor 326, such as an electrically driven motor, a pneumatically driven motor, a hydraulically driven motor, or even a winding-driven motor. The motor 326 may be located within the forward assembly 316, as indicated by dashed lines. In some cases, the motor 326 does not have to be located within the forward assembly 316, but instead may be located remotely within the console 328, with a flexible drive cable 330 extending from the motor 326 to the forward assembly 316.

[0040] The drive assembly 312 and the control unit 314 are capable of communication and may be located in the same housing, and / or in separate housings (e.g., the forward assembly housing 326 and the control unit housing 328, or other housings). Whether in the same housing or separate housings, the drive assembly 312 and the control unit 314 may communicate via a wired connection (e.g., via one or more electrical connectors 324 or other suitable electrical connectors) and / or a wireless connection. The wireless connection is made via one or more suitable communication protocols, including but not limited to cellular communication, ZigBee®, Bluetooth®, Wi-Fi®, Infrared Data Association (IrDA®), Dedicated Short-Range Communication (DSRC), EnOcean®, and / or other suitable general or proprietary wireless protocols.

[0041] Although not necessarily shown in Figure 1, the drive assembly 312 may include and / or house one or more operating structures. For example, among other components, the drive assembly 312 may include a motor (e.g., the above and / or other suitable motors), rubber feet, control electronics, drive circuits, and so on.

[0042] The control unit 314 may include several structures that may be separated from the drive assembly 312 (as shown in Figure 1, for example) or included in the drive assembly 312. For example, as shown in Figure 1, the control unit 314 may include a display 330 and control knobs 332 (e.g., motor speed (e.g., RPM or other speed) adjustment knob or other control knobs). Additionally or alternatively, the control unit 314 may include one or more other structures for controlling the drive mechanism and / or other structures of the drive assembly 312 (e.g., one or more states of the drive mechanism), including a processor, memory, input / output devices, speakers, volume control buttons, on / off power switch, motor start switch, timer, clock, and / or other functions.

[0043] In some cases, the control unit 314 may include one or more drive mechanism load output control mechanisms for controlling the operation of the rotary atherectomy system 310. An example of a drive mechanism load output control mechanism that may be included in the control unit 314 is a mechanism configured to set and / or adjust the forward load output (e.g., rotational speed) and / or the reverse load output from the drive mechanism 34. Additionally or alternatively, the control unit 314 may include other control and / or safety mechanisms for controlling the operation of the rotary atherectomy system 310 and mitigating risks to the patient.

[0044] For example, the rotating device 320 can be realized as a drill with a rough surface like a file. A fine diamond is embedded in the tip of the drill (rotating device 320), and as shown in Figure 3, when the drill 320 is introduced into the coronary artery 40, the drill 320 is driven to rotate, cutting the plaque 41 inside the coronary artery 40. The plaque 41 is sufficiently hard, while the inner surface of the coronary artery 40, where no plaque has accumulated, is sufficiently soft. Therefore, when the drill 320 rotates, the drill 320 does not damage the inner surface of the coronary artery 40. On the other hand, since the plaque 41 is sufficiently hard, when the drill 320 rotates, the plaque 41 is cut by the drill 320. Therefore, when the rotating assembly 317 is introduced into the coronary artery 40 of the human body, and the rotating device (drill) 320 of the rotating assembly 317 is rotated, and the knob 323 of the forward assembly 316 is operated to advance the rotating device 320, at least a portion of the plaque 41 that is forming a narrowing in the coronary artery 40 is cut away. As a result, blood flow in the coronary artery 40, which was obstructed by the plaque 41, is restored, and blood supply to the cardiac myocardium by the coronary artery 40 becomes smooth. This allows for treatment of angina pectoris and myocardial infarction.

[0045] As already outlined, the rotating assembly 317 is introduced into the arteries of the human body, for example, from the radial artery in the wrist in Figure 4 to the coronary arteries of the heart, as shown in Figure 4.

[0046] While treatment for calcified vascular lesions using rotary atherectomy devices such as rotablators involves real-time monitoring of the internal position of the drive shaft 318 and rotary device 320 using an X-ray projection device when introducing the rotary assembly 317 to the coronary artery, and sometimes injecting contrast agent into the coronary artery to confirm that blood flow in the coronary artery 40 has been restored by the rotary device (drill) 320, treatment with rotary atherectomy devices is highly dependent on the operator's experience, and the procedure carries a risk of complications. Furthermore, the procedure is difficult and relies on tacit knowledge such as the operator's intuition, making objective risk assessment and education difficult.

[0047] Therefore, the inventors discovered that by recording the driving sound of the rotator, converting the acoustic data into a spectrogram to create an image, and analyzing it using an autoencoder (AI), it is possible to analyze in real time which of the following three patterns the procedure using a rotary atherectomy device is. That is, 1) Normal time 2) Burr entrapment, that is, a condition in which the rotation of the rotating device 20 stops because the plaque 41 cannot be cut smoothly by the rotating device 20 during plaque cutting (this is a complication). 3) Guidewire transection, that is, when the drive shaft 318 is severed during rotational drive of the rotating device 20, or when introducing the rotating device 20 to the narrowed portion of the coronary artery (this is also a complication). The inventors' research has revealed that each of these exhibits a unique frequency pattern.

[0048] More specifically, 2) when the bars are stacked, a decrease in the frequency of the sound generated from the rotary atherectomy device occurs (for example, from 3000Hz to 2000Hz), and 3) when the guidewire is cut, high-frequency components are generated in the sound produced from the rotary atherectomy device, and a creaking sound is also generated.

[0049] 1) Under normal conditions, the sound produced by the rotary atherectomy device is centered around 3000Hz. 2) The sound produced when the bar is stacked is centered around 2000Hz, as previously explained. Furthermore, 3) the sound produced when the guidewire is cut is higher frequency than 3000Hz. On the other hand, the frequency range of sounds produced during the rotary atherectomy device procedure, for example, in the operating room, does not overlap with the frequency ranges of these sounds 1) to 3). For example, the frequency range of sounds produced by a heart rate monitor is centered around 1000Hz, and the frequency range of conversations among medical personnel, including the surgeon, is 500-2000Hz. The inventor's experience also confirms that the sound produced during the rotary atherectomy device procedure can be clearly distinguished from other sounds.

[0050] Therefore, in the system described in this disclosure, as shown in Figure 4, a device (terminal device 10) capable of acquiring (recording) acoustic data generated from a rotary atherectomy device is placed in the operating room or other location where the procedure using the rotary atherectomy device is performed. The acoustic data is acquired by this terminal device 10 and sent in real time to a cloud server (server 20) (not shown in Figure 4), where the server 20 analyzes the acoustic data. The acoustic data analyzed by the server 20 (acoustic data analysis results) is then input into a trained model stored in the server 20, and risk assessment information output from this trained model is acquired. The trained model has been pre-trained on the relationship between acoustic data (analysis results) and risk assessment information, and outputs risk assessment information when acoustic data (analysis results) is input.

[0051] In the system related to this disclosure, risk assessment information includes, for example, information on whether or not complications occurred, or the likelihood of them occurring. As mentioned above, examples of complications include bar stacking and guidewire breakage during procedures using a rotary atherectomy device.

[0052] Furthermore, the system related to this disclosure utilizes risk assessment information obtained from trained models in the following three patterns.

[0053] (1) Notification to the surgeon performing the procedure during vascular treatment If risk assessment information is obtained indicating a high probability of complications occurring (or having occurred), the operator is notified via a terminal device 10 located in the operating room or other designated area. For example, the notification may be made by displaying on the terminal device 10's screen that there is a high risk of complications occurring (preferably details of the complications, such as whether the bars stacked), or by emitting an alert sound from the terminal device 10.

[0054] (2) Feedback to the operator Acoustic data (analysis results) and / or risk assessment information are stored in server 20 along with identification information (ID) that identifies the operator. Server 20 then uses the identification information that identifies the operator to search for risk assessment information for past procedures and presents to the operator, in the form of graphs and scores, trends in risk assessment information for the past 10 cases, trends in the likelihood of complications occurring for the past 10 cases (e.g., a generally high probability of bar stacking), and comparisons of risk assessment information with other operators.

[0055] (3)Education and training The system displays and compares on the screen the acoustic data (analysis results) from when the resident is the surgeon and the acoustic data (analysis results) from when a model instructor is the surgeon. Furthermore, if there are significant differences in the acoustic data (analysis results), or if dangerous signs of noise are detected in the acoustic data (analysis results) generated during the resident's procedure, the system highlights these points and issues alerts. After the resident's procedure, the system presents an evaluation of the entire procedure based on the acoustic data (analysis results), displays a timestamp indicating when dangerous signs of noise occurred, and offers advice for improvement to the resident. Note that in (3), outputting risk assessment information from a pre-trained model is not necessarily required.

[0056] With the above configuration, sensory information that was previously tacit knowledge of the surgeon can be quantified and visualized, enabling objective judgment and support for procedures using a rotary atherectomy device.

[0057] <One Embodiment> <1 System Configuration Diagram> Figure 5 shows the overall configuration of the procedure support system (hereinafter simply referred to as "the system") 1 of this embodiment. As shown in Figure 5, the system 1 includes a plurality of terminal devices (in Figure 1, terminal devices 10A and 10B are shown; hereinafter collectively referred to as "terminal device 10"), a server 20, and an external server 30. The terminal devices 10, server 20, and external server 30 are connected to each other so as to be able to communicate with each other via a network 80. The network 80 is composed of a wired or wireless network. In this embodiment, server 20 is a server that functions as a web server (including a cloud server) and exchanges information with terminal devices 10 via web pages. In addition, a web page browser for viewing web pages is installed on terminal devices 10, but a dedicated application for providing services from server 20 may be installed and configured to be viewable by the dedicated application.

[0058] As previously described, the terminal device 10 is placed in an operating room or the like and records the sounds generated during the procedure using the rotary atherectomy device to acquire acoustic data, and transmits the acquired acoustic data to the server 20. The terminal device 10 also provides various notifications to the surgeon performing the procedure using the rotary atherectomy device. Therefore, the terminal device 10 is a device owned and preferably operated by medical personnel, including the surgeon performing the procedure using the rotary atherectomy device. In the system 1 of this embodiment, the terminal device 10 that acquires acoustic data generated by the procedure using the rotary atherectomy device and the terminal device 10 that provides various notifications to the surgeon performing the procedure using the rotary atherectomy device may be separate devices.

[0059] The terminal device 10 is, for example, a mobile device such as a tablet or smartphone that supports a mobile communication system. Alternatively, the terminal device 10 may be a stationary PC (Personal Computer), laptop PC, etc. Furthermore, the terminal device 10 may be a landline telephone, television receiver (TV), smart speaker, etc., that have communication functions such as the Internet. Although two terminal devices 10 are shown in Figure 1, there is no limit to the number of terminal devices 10 used in System 1, and multiple terminal devices 10 may be used simultaneously.

[0060] Terminal device 10 is connected to server 20 via network 80 in a communicative manner. Terminal device 10 connects to network 80 by communicating with communication equipment such as a wireless base station 81 that supports communication standards such as 4G, 5G, LTE (Long Term Evolution), IEEE 802.16-2004 (so-called WiMAX), and UWB (Ultra Wide Band), and a wireless LAN router 82 that supports wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11. In addition, terminal device 10 connects to network 80 by communicating with communication equipment that supports already implemented wireless communication standards, such as IEEE 802.15, represented by Bluetooth (registered trademark). Herein, the wireless communication standards that terminal device 10 can communicate with are not limited to those approved by IEEE, IEC (International Electrotechnical Commission), and ISO (International Organization for Standardization), but do not exclude wireless communication standards currently under consideration by these organizations. Furthermore, the terminal device 10 may be connected to the network 80 via a so-called wired LAN (IEEE 802.3) and a wired LAN router. As shown in Figure 2, the terminal device 10 includes a communication interface 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

[0061] The communication interface 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device (for example, a keyboard, touch panel, touchpad, mouse, or other pointing device) for receiving input operations from the user. The output device 14 is an output device (display, speaker, etc.) for presenting information to the user. The memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). The storage unit 16 is a storage device for saving data, such as flash memory or an HDD (Hard Disk Drive). The processor 19 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0062] Furthermore, the terminal device 10 that acquires acoustic data generated by the procedure using a rotary atherectomy device may be a portable IC recorder with communication capabilities, which is specialized in the function of acquiring acoustic data (recording sound).

[0063] Server 20 is managed by the administrator of System 1 in this embodiment, and the stored contents are modified, added, or deleted by this administrator as appropriate.

[0064] Server 20 is a computer connected to network 80. Server 20 includes a communication interface 22, an input / output interface 23, memory 25, storage 26, and a processor 29.

[0065] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices. Input / Output IF23 functions as an interface to an input device for receiving input operations from the user and an output device for presenting information to the user. Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory). Storage 26 is a storage device for saving data, such as flash memory or HDD (Hard Disk Drive). Processor 29 is hardware for executing the instruction set written in the program, and is composed of an arithmetic unit, registers, peripheral circuits, etc.

[0066] The external server 30 is a so-called cloud server located separately from the server 20. In System 1 of this embodiment, the external server 30 is not essential, but it is also possible to configure it so that the external server 30, which is a cloud server, mainly stores and analyzes acoustic data (including outputting risk assessment information based on trained models), while the server 20 is responsible for presenting and notifying medical personnel, mainly surgeons performing procedures with a rotary atherectomy device.

[0067] <1.1 Functional configuration of terminal device 10> Figure 6 is a block diagram showing an example of the functional configuration of the terminal device 10 shown in Figure 5. The terminal device 10 shown in Figure 6 can be implemented, for example, by a PC, a mobile terminal, or a wearable terminal. As shown in Figure 6, the terminal device 10 includes a first communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a storage unit 180, and a control unit 190. Each block included in the terminal device 10 is electrically connected, for example, by a bus.

[0068] The first communication unit 120 performs modulation and demodulation processing for the terminal device 10 to communicate with other devices such as the server 20 and the external server 30. The first communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to an external source (for example, the server 20). The first communication unit 120 performs reception processing on the signal received from an external source and outputs it to the control unit 190.

[0069] The input device 13 is a device for a user operating the terminal device 10 to input instructions or information. The input device 13 may be implemented as, for example, a keyboard, mouse, reader, etc. If the terminal device 10 is a mobile terminal, it may be implemented as a touch-sensitive device 131, etc., to which instructions are input by touching the operating surface. The input device 13 converts the instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may also include, for example, a receiving port that accepts electrical signals input from an external input device.

[0070] The output device 14 is a device for presenting information to the user operating the terminal device 10. The output device 14 is implemented, for example, by a display 141. The display 141 displays data according to the control of the control unit 190. The display 141 is implemented, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0071] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of the audio signal. The audio processing unit 17 converts the signal received from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is implemented, for example, by an audio processing processor. The microphone 171 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 17. The speaker 172 converts the audio signal received from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0072] The storage unit 180 stores data and programs used by the terminal device 10. For example, the storage unit 180 stores an application program 181 and acoustic data 182. The acoustic data 182 is acoustic data acquired by the acoustic data acquisition unit 195 of the control unit 190, which will be described later.

[0073] The control unit 190 is realized when the processor 19 reads the application program 181 stored in the memory unit 180 and executes the instructions contained in the application program 181. The control unit 190 controls the operation of the terminal device 10. By operating according to the application program 181 stored in the memory unit 180, the control unit 190 performs the functions of an input operation receiving unit 191, a transmitting / receiving unit 192, a data processing unit 193, a presentation control unit 194, and an acoustic data acquisition unit 195.

[0074] The input operation receiving unit 191 processes instructions or information input from the input device 13. Specifically, for example, the input operation receiving unit 191 receives information based on instructions input from a keyboard, mouse, etc.

[0075] The transmitting / receiving unit 192 performs processing to enable the terminal device 10 to send and receive data with external devices such as the server 20 and the external server 30 in accordance with the communication protocol.

[0076] The data processing unit 193 performs calculations on the data received as input by the terminal device 10 according to the application program 181 and outputs the calculation results to the storage unit 180.

[0077] The presentation control unit 194 controls the output device 14 in order to present the information generated by the data processing unit 193 to the user. Specifically, for example, the presentation control unit 194 displays the information generated by the data processing unit 193 on the display 141. The presentation control unit 194 also outputs the information generated by the data processing unit 193 from the speaker 172.

[0078] The acoustic data acquisition unit 195 converts the audio signal acquired by the microphone 171 of the audio processing unit 17 and converted from analog to digital by the audio processing unit 17 into a predetermined format, such as WAV format, and temporarily stores it in the storage unit 180 as acoustic data 182. Then, the acoustic data acquisition unit 195 sends the acoustic data 182 stored in the storage unit 180 to the server 20 (or external server 30) via the transmitting and receiving unit 192.

[0079] <1.2 Functional Configuration of Server 20> Figure 7 shows an example of the functional configuration of server 20. As shown in Figure 7, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0080] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0081] The memory unit 202 stores data and programs used by the server 20. The memory unit 202 stores the patient database (Database:DB) 2022, the surgical database 2023, the acoustic data database 2024, the acoustic data 2025, the trained model 2026, the screen data template 2027, and so on.

[0082] Patient DB2022 is a database for managing data about patients who have undergone a procedure using a rotary atherectomy device, the acoustic data and other information of which are managed by System 1 of this embodiment. Further details will be described later.

[0083] Surgical DB2023 is a database for managing data related to procedures using a rotary atherectomy device, for which acoustic data and other information are managed by System 1 of this embodiment. Further details will be described later.

[0084] The acoustic data DB2024 is a database for managing acoustic data acquired by the terminal device 10 managed by System 1 of this embodiment and sent to the server 20. Further details will be described later.

[0085] Acoustic data 2025 is acoustic data acquired by terminal device 10 and sent to server 20.

[0086] These patient databases (2022), surgery database (2023), acoustic data database (2024), and acoustic data (2025) serve as training data for generating the pre-trained model (2026), which will be discussed later.

[0087] When the trained model 2026 receives the acoustic data analysis results, which are analyzed by the acoustic data analysis module 2024 based on the acoustic data 2025 described later, it outputs risk assessment information for the procedure using a rotary atherectomy device that corresponds to these acoustic data analysis results.

[0088] The trained model 2026 is obtained by having a machine learning model perform machine learning according to the model learning program (not shown in the figure) based on the training data described above. The trained model 2026 according to this embodiment is, for example, a parameterized composite function composed of multiple functions. A parameterized composite function is defined by a combination of multiple tunable functions and parameters. The prediction model according to this embodiment may be any parameterized composite function that satisfies the above requirements, but it is assumed to be a multi-layer network model (hereinafter referred to as a multi-layer network). A prediction model using a multi-layer network has an input layer, an output layer, and at least one intermediate or hidden layer between the input and output layers. The prediction model is intended to be used as a program module that is part of artificial intelligence software.

[0089] As the multilayer network according to this embodiment, for example, a deep neural network (DNN), which is a multilayer neural network targeted by deep learning, may be used. As the DNN, for example, a convolutional neural network (CNN) that targets images may be used.

[0090] Furthermore, the above is merely an example of a prediction model, and a prediction model may have other configurations. For example, the prediction model may be a rule-based model described by a function in which acoustic data analysis results and risk assessment information are used as variables, and coefficients derived from past performance are attached to each variable.

[0091] Screen data template 2027 is template data for creating a screen that presents analysis results to the user of terminal device 10 (the surgeon who performed the rotational atherectomy device procedure) and the person who presents the analysis results from server 20 (who is also the surgeon who performed the rotational atherectomy device procedure).

[0092] The control unit 203 performs the functions shown in the various modules, namely the receive control module 2031, the transmit control module 2032, the acoustic data acquisition module 2033, the acoustic data analysis module 2034, the learning model generation module 2035, the risk assessment acquisition module 2036, the feedback result generation module 2037, the feedback result output module 2038, and the presentation control module 2039, by having the server 20's processor process according to the application program 2021 stored in the memory unit 202.

[0093] The receive control module 2031 controls the process by which the server 20 receives signals from external devices according to a communication protocol.

[0094] The transmission control module 2032 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

[0095] The acoustic data acquisition module 2033 acquires the acoustic data 185 that the terminal device 10 has acquired and sent, and stores it in the storage unit 202 as acoustic data 2025.

[0096] The acoustic data analysis module 2034 analyzes the acoustic data 2025 acquired by the acoustic data acquisition module 2033 and stored in the storage unit 202 to generate acoustic data analysis results. The acoustic data analysis results generated by the acoustic data analysis module 2034 are stored in the acoustic data DB 2024.

[0097] The acoustic data analysis module 2034 analyzes acoustic data 2025 and generates acoustic data analysis results in order to facilitate inference operations by the trained model 2026. Acoustic data 2025 is, for example, in WAV format, and it is preferable to perform feature analysis and other processes before performing machine learning in order to perform inference operations by the trained model 2026.

[0098] Therefore, the acoustic data analysis module 2034 generates a spectrogram of the acoustic data 2025 based on this data. The spectrogram plots the time progression of the acoustic data 2025 on the horizontal axis, the frequency analysis results on the vertical axis, and the amplitude or intensity of the frequency components as the density of the frequency analysis results. Thus, the spectrogram itself has three-dimensional components. In this case, the density of the frequency analysis results is often displayed as a color component. Such a spectrogram is generated and displayed as a color spectrogram image.

[0099] Figure 15 shows an example of a spectrogram image based on acoustic data acquired when performing a procedure using a rotary atherectomy device on a model simulating the inside of the body, as demonstrated by the inventor. While the actual spectrogram image is a color image, Figure 15 is a grayscale display of the color spectrogram image. In the figure, the spectrogram image labeled "normal rotablation" is a spectrogram image based on acoustic data acquired when the procedure using the rotary atherectomy device is performed normally, and the spectrogram image labeled "burr entrapment" is a spectrogram image based on acoustic data acquired when the bar (rotating device 20) was stacked. In the figure, the horizontal axis represents elapsed time, the vertical axis represents frequency, and the intensity (actually a color scale) represents frequency intensity.

[0100] However, spectrogram images are color images with three-dimensional components. To generate the trained model 2026 using these spectrogram images as training data would require complex training data, making efficient machine learning impossible. Therefore, the acoustic data analysis module 2034 compresses the spectrogram images to two-dimensional data using known dimensionality reduction techniques before using them as training data for the trained model 2026.

[0101] Dimensionality reduction techniques are well-known, and one example is the use of an autoencoder, a dimensionality reduction algorithm that utilizes neural networks in machine learning. Since autoencoders themselves are well-known, a detailed explanation will be omitted here. The autoencoder used by the acoustic data analysis module 2034 reduces the dimensionality of a color spectrogram image with three-dimensional components to data with two-dimensional components.

[0102] Here, the dimensionality reduction method is not limited to that using an autoencoder. Another example is the method of converting a color spectrogram image with three-dimensional components into a grayscale image. Although the grayscale image still has three-dimensional components, the amount of information can be significantly reduced compared to the color image, making it suitable as training data for generating a trained model.

[0103] The acoustic data analysis module 2034 acquires a dimensionality-reduced spectrogram image as the acoustic data analysis result and stores it in the acoustic data DB 2024. In this specification, the acoustic data analysis results and acoustic data may not be distinguished from each other.

[0104] The learning model generation module 2035 generates a pre-trained model 2026 using patient DB 2022, surgery DB 2023, and acoustic data DB 2024 as training data. The method used by the learning model generation module 2035 to generate the pre-trained model 2026 is publicly known, so a detailed explanation is omitted.

[0105] The risk assessment acquisition module 2036 inputs the acoustic data analysis results generated by the acoustic data analysis module 2034 into the trained model 2026 stored in the memory unit 202, and acquires the risk assessment information output as a result of the inference operation from this trained model 2026. The risk assessment acquisition module 2036 then stores the acquired risk assessment information in the acoustic data DB 2024.

[0106] The feedback result generation module 2037 generates feedback results for the surgeon who performed the procedure using the rotary atherectomy device, based on the acoustic data analysis results generated by the acoustic data analysis module 2034 and the risk assessment information acquired by the risk assessment acquisition module 2036. Then, the feedback result output module 2038 outputs the feedback results generated by the feedback result generation module 2037.

[0107] The feedback results generated by the feedback result generation module 2037 and output by the feedback result output module 2038 differ for each of the three patterns described above. While the types of feedback results generated have already been explained, we will explain them again.

[0108] (1) Notification to the surgeon performing the procedure during vascular treatment If risk assessment information is obtained indicating a high probability of complications occurring (or having occurred), the feedback result output module 2038 notifies the surgeon via a terminal device 10 located in the operating room or elsewhere. For example, the feedback result output module 2038 notifies the surgeon by displaying on the terminal device 10's display 141 that there is a high risk of complications occurring (preferably details of the complications, such as whether a bar stack occurred), or by emitting an alert sound from the terminal device 10.

[0109] (2) Feedback to the operator Acoustic data (analysis results) and / or risk assessment information are stored in the storage unit 202 of the server 20 along with identification information (ID) that identifies the operator. The feedback result output module 2038 then uses the identification information that identifies the operator to search for risk assessment information for past procedures and presents to the operator, in a visualized form using graphs and scores, trends in risk assessment information for the past 10 cases, trends in the likelihood of complications occurring for the past 10 cases (e.g., a generally high probability of bar stacking), and comparisons of risk assessment information with other operators. The graphs and scores can be displayed using a display device (not shown) on the server 20. Alternatively, these graphs and scores may be sent to the terminal device 10.

[0110] (3)Education and training The feedback result output module 2038 displays and compares the acoustic data (analysis results) when the resident is the operator and the acoustic data (analysis results) when the exemplary instructor is the operator on the display 141 of the terminal device 10. Furthermore, if there are significant differences in this acoustic data (analysis results), or if dangerous warning sounds are detected in the acoustic data (analysis results) generated during the procedure performed by the resident, the module highlights the data or issues an alert. In addition, after the procedure performed by the resident, the feedback result output module 2038 presents an evaluation of the entire procedure based on the acoustic data (analysis results) via the terminal device 10, displays a timestamp indicating when dangerous warning sounds occurred, and provides advice to the resident for improvement. Note that in (3), output of risk assessment information from the trained model 2026 is not necessarily required.

[0111] The display control module 2039 generates a display control signal to display a desired screen on the display 141 of the terminal device 10 and sends this display control signal to the terminal device 10. The display control module 2039 also generates a sound effect signal to be played by the speaker 172 of the terminal device 10, as needed, and sends this sound effect signal to the terminal device 10.

[0112] <2 Data Structure> Figures 8 to 10 show the data structure of the database stored by server 20. Note that Figures 8 to 10 are examples and do not exclude data that is not shown.

[0113] The databases shown in Figures 8 to 10 refer to relational databases, which are data sets called tables, structurally defined by rows and columns, and are used to manage and relate these tables to each other. In databases, tables are called tables, the columns of tables are called columns, and the rows of tables are called records. In relational databases, relationships can be established and linked between tables.

[0114] Typically, each table has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit 203 of the server 20 can instruct the processor 29 to add, delete, or update records in specific tables stored in the storage unit 202, according to various programs.

[0115] Figure 8 shows the data structure of the patient database 2022. As shown in Figure 8, each record in the patient database 2022 includes, for example, the fields "Patient ID", "Patient Name", "Patient Background", and "Case ID". Each field in the patient database 2022 is entered by the operator of the server 20 prior to the output of risk assessment information based on the acoustic data analysis results in system 1 of this embodiment. The information stored in the patient database 2022 can be changed and updated as needed.

[0116] The item "Patient ID" is an ID used to identify patients who are to be treated using the rotary atherectomy device procedure in System 1 of this embodiment. The item "Patient Name" is information about the name of the patient identified by the item "Patient ID". The item "Patient Background" is information about the background of the patient identified by the item "Patient ID", for example, information about the degree of coronary artery calcification in the patient. The item "Case ID" is information used to identify the case of the patient identified by the item "Patient ID".

[0117] Figure 9 shows the data structure of the surgical database 2023. As shown in Figure 9, each record in the surgical database 2023 includes, for example, the fields "Patient ID", "Patient Name", "Surgery ID", "Surgeon ID", and "Date and Time of Procedure". Each field in the surgical database 2023 is entered by the operator of the server 20 prior to the output of risk assessment information based on the acoustic data analysis results in system 1 of this embodiment. The information stored in the surgical database 2023 can be changed and updated as needed.

[0118] The "Patient ID" field is the same as the "Patient ID" field in Figure 8. The "Patient Name" field is also the same as the "Patient Name" field in Figure 8. The "Surgery ID" field is used to identify individual procedures when the patient identified by the "Patient ID" field has undergone a procedure using a rotary atherectomy device (i.e., surgery). The "Operator ID" field is used to identify the operator who performed the surgery identified by the "Surgery ID" field. The "Date and Time of Procedure" field contains information about the date and time the surgery identified by the "Surgery ID" field was performed.

[0119] Figure 10 shows the data structure of the acoustic data DB 2024. As shown in Figure 10, each record in the acoustic data DB 2024 includes, for example, the items "Patient ID", "Surgery ID", "Operator ID", "Acoustic Data", and "Risk Assessment". Each item in the acoustic data DB 2024 is entered by the operator of the server 20 prior to the output of risk assessment information based on the acoustic data analysis results in System 1 of this embodiment. Furthermore, each item in the acoustic data DB 2024 is entered based on the output of the acoustic data acquisition module 2033, the acoustic data analysis module 2034, and the risk assessment acquisition module 2036. The information stored in the acoustic data DB 2024 can be changed and updated as needed.

[0120] The item "Patient ID" is the same as the item "Patient ID" in Figure 8. The item "Surgery ID" is also the same as the item "Surgery ID" in Figure 9. The item "Operator ID" is also the same as the item "Operator ID" in Figure 9. The item "Acoustic Data" is the acoustic data 2025 itself acquired during the procedure (surgery) identified by the item "Surgery ID," or information used to identify acoustic data 2025. In addition, this item "Acoustic Data" also stores the acoustic data analysis results obtained by the acoustic data analysis module 2034 for the acoustic data identified by the item "Acoustic Data." The item "Risk Assessment" is information about risk assessment information regarding acoustic data 2025 identified by the item "Acoustic Data." The information stored in the "Risk Assessment" field is entered based on the judgment of the exemplary instructor (operator) before the inference operation by the trained model 2026 is performed. Once the inference operation by the trained model 2026 is performed, the results of this inference operation by the trained model 2026 are entered.

[0121] <3 Example of Operation> The following describes an example of the operation of the terminal device 10 and the server 20.

[0122] Figures 11 to 14 illustrate the output operation of risk assessment information based on acoustic data by the server 20 in System 1 of this embodiment.

[0123] Figure 11 is a diagram illustrating the operation of generating the trained model 2026 by the server 20. First, in step S1100, the control unit 203 of the server 20 acquires the acoustic data 185 sent from the terminal device 10 and stores it in the storage unit 202 as acoustic data 2025. More specifically, the acoustic data acquisition module 2023 of the control unit 203 acquires the acoustic data 185 sent from the terminal device 10 and stores it in the storage unit 202 as acoustic data 2025.

[0124] Next, in step S1101, the control unit 203 of the server 20 analyzes the acoustic data 2025 acquired in step S1100. More specifically, the acoustic data analysis module 2034 of the control unit 203 analyzes the acoustic data 2025 acquired in step S1100. The analyzed acoustic data (i.e., the acoustic data analysis results) is stored in the acoustic data DB 2024.

[0125] In step S1102, the control unit 203 of the server 20 acquires risk assessment information for the acoustic data analysis results analyzed in step S1101 and stores it in the acoustic data DB 2024. This risk assessment information in step S1102 is entered based on the judgment of the exemplary instructor (surgeon).

[0126] In step S1103, the control unit 203 of the server 20 generates a trained model 2026 using the patient DB 2022, surgery DB 2023, and acoustic data DB 2024 as training data, and stores the generated trained model 2026 in the storage unit 202. More specifically, the learning model generation module 2035 of the control unit 203 generates a trained model 2026 using the patient DB 2022, surgery DB 2023, and acoustic data DB 2024 as training data, and stores the generated trained model 2026 in the storage unit 202.

[0127] Next, Figure 12 is a diagram illustrating the operation of the server 20 when (1) providing notifications to the surgeon performing the procedure during vascular treatment.

[0128] The operation of steps S1200 to S1201 is the same as steps S1100 to S1101 in Figure 11, so the explanation is omitted. In step S1202, the control unit 203 of the server 20 inputs the acoustic data analysis results acquired in step S1201 into the trained model 2026, and in step S1203, it acquires risk assessment information from this trained model 2026. More specifically, the risk assessment acquisition module 2036 of the control unit 203 inputs the acoustic data analysis results acquired in step S1201 into the trained model 2026, and acquires risk assessment information from this trained model 2026.

[0129] In step S1204, the control unit 203 of the server 20 sends feedback results based on the risk assessment information acquired in step S1203 to the terminal device 10 and displays them on the display 141 of the terminal device 10. More specifically, the feedback result generation module 2037 of the control unit 203 generates feedback results based on the risk assessment information acquired in step S1203, and the feedback result output module 2038 sends the feedback results generated by the feedback result generation module 2037 to the terminal device 10.

[0130] Next, Figure 13 is a diagram illustrating the operation of the server 20 when providing feedback to the operator (2).

[0131] First, in step S1300, the control unit 203 of the server 20 accesses the acoustic data DB 2024, selects a surgeon ID to identify the surgeon to be provided with feedback in step S1301, and in step S1302 searches the acoustic data DB 2024 using the surgeon ID selected in step S1301 as the search key to obtain the acoustic data analysis results. More specifically, the feedback result generation module 2037 of the control unit 203 accesses the acoustic data DB 2024, selects a surgeon ID to identify the surgeon to be provided with feedback, searches the acoustic data DB 2024 using the selected surgeon ID as the search key to obtain the acoustic data analysis results.

[0132] In step S1303, the control unit 203 of the server 20 performs statistical processing on the acoustic data analysis results obtained in step S1302. More specifically, the feedback result generation module 2037 of the control unit 203 performs statistical processing on the acoustic data analysis results obtained in step S1302.

[0133] Then, in step S1304, the control unit 203 of the server 20 displays the results of the statistical processing of the risk assessment information performed in step S1303. The display may be a display device of the server 20 (not shown in the figure) or the display 141 of the terminal device 10. More specifically, the feedback result output module 2038 of the control unit 203 displays the results of the statistical processing of the risk assessment information performed in step S1303.

[0134] Next, Figure 14 is a diagram illustrating the operation of server 20 when (3) education and training is performed.

[0135] First, in step S1400, the control unit 203 of the server 20 accepts an input from the operator performing the education / training to select the education mode. The operation of steps S1401 to S1402 is the same as steps S1100 to S1101 in Figure 11, so the explanation is omitted. Also, the operation of step S1403 is the same as step S1300 in Figure 13, so the explanation is omitted.

[0136] In step S1404, the control unit 203 of the server 20 compares the acoustic data analysis results obtained from the rotational atherectomy device procedure performed by a resident physician with the acoustic data analysis results obtained from the rotational atherectomy device procedure performed by a model surgeon, using the acoustic data DB 2024. More specifically, the feedback result generation module 2037 of the control unit 203 compares the acoustic data analysis results obtained from the rotational atherectomy device procedure performed by a resident physician with the acoustic data analysis results obtained from the rotational atherectomy device procedure performed by a model surgeon, using the acoustic data DB 2024.

[0137] Then, in step S1405, the feedback result output module 2038 of the control unit 203 sends the comparison result performed in step S1404 to the terminal device 10.

[0138] <4 Summary> As described in detail above, according to System 1 of this embodiment, the acoustic data analysis results based on acoustic data generated by the procedure of the rotary atherectomy device are input into the trained model 2026, and risk assessment information is obtained from this trained model 2026. This makes it possible to quantify and visualize sensory information that was previously the tacit knowledge of the surgeon, thereby supporting objective decision-making. Similarly, according to System 1 of this embodiment, the acoustic data analysis results are compared, and statistical processing of the acoustic data analysis results is performed, so again, it makes it possible to quantify and visualize sensory information that was previously the tacit knowledge of the surgeon, thereby supporting objective decision-making.

[0139] <5 Variations> The embodiments described above are detailed explanations of the configuration for the purpose of clearly illustrating this disclosure, and are not necessarily limited to those comprising all the configurations described. Furthermore, some of the configurations in each embodiment can be added to, deleted from, or replaced with other configurations.

[0140] As an example, in the system 1 of the embodiment described above, acoustic data analysis results were obtained using a spectrogram image obtained from acoustic data, and risk assessment information was obtained based on these acoustic data analysis results. However, it is also possible to generate a trained model 2026 that takes the acoustic data itself as input and obtain risk assessment information from the acoustic data. As an example, waveform data for a certain period of time may be obtained from acoustic data (speech signals), and a trained model 2026 may be generated using the features of this waveform data.

[0141] <6.1 Basic Hardware Configuration of a Computer> Figure 15 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 901, main memory 902, auxiliary memory 903, and a communication interface IF991. These are electrically connected to each other by a communication bus 921.

[0142] The processor 901 is hardware for executing the instruction set written in a program. The processor 901 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0143] Main memory 902 is used to temporarily store programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0144] Auxiliary storage device 903 refers to a storage device for saving data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, and semiconductor memory.

[0145] The IF991 communication interface is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards.

[0146] A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0147] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0148] <6.2 Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 (Figure 15) will be explained. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0149] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0150] The control unit is realized when the processor 901 reads various programs stored in the auxiliary storage device 903, loads them into the main memory device 902, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0151] The memory unit is implemented by the main memory 902 and the auxiliary memory 903. The memory unit stores data, various programs, and various databases. The processor 901 can also reserve memory areas corresponding to the memory unit in the main memory 902 or the auxiliary memory 903 according to the program. The control unit can also cause the processor 901 to perform operations such as adding, updating, and deleting data stored in the memory unit according to the various programs.

[0152] A database, specifically a relational database, is used to manage and link together tabular data sets called masters, which are structurally defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters.

[0153] Typically, each table and master has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit can instruct the processor 901 to add, delete, or update records in specific tables and masters stored in the memory unit, according to various programs.

[0154] Furthermore, by storing data, various programs, and various databases in the memory unit, the information processing device and information processing system related to this disclosure can be considered to have been manufactured.

[0155] Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

[0156] The communication unit is implemented by the communication IF991. The communication unit provides the functionality to communicate with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0157] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0158] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0159] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0160] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0161] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0162] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0163] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0164] (Note) The details described in each of the above embodiments are noted below. (Note 1) A computer comprising a processor and memory, a program for operating a computer that supports procedures related to vascular treatment using a rotary atherectomy device, wherein the memory stores a trained model that takes acoustic data acquired in the environment in which vascular treatment is being performed as input and outputs risk assessment information about the procedure related to vascular treatment, and the program causes the processor to perform a first step of receiving acoustic data, a second step of inputting the acoustic data received in the first step into the trained model and obtaining risk assessment information output from this trained model, and a third step of outputting the risk assessment information obtained in the second step in a form that can be perceived by humans. (Note 2) In the third step, the program described in Appendix 1 outputs risk assessment information to the surgeon performing surgery during vascular treatment, the surgeon performing surgery after the procedure, and the surgeon performing surgery during a simulated vascular treatment. (Note 3) In the third step, when risk assessment information is output to the surgeon performing the vascular treatment surgery, the program as described in Appendix 2 includes at least one of the following: information regarding the risk of complications occurring as a result of the surgery, and information regarding the risk of the bars of the rotary atherectomy device becoming stuck during vascular treatment. (Note 4) In the third step, if risk assessment information is output to the surgeon who performed the surgery after the surgical procedure, the program as described in Appendix 2 or 3 includes at least one of the following: risk assessment information for a predetermined number of vascular treatments recently performed by the surgeon, the degree of tendency for the bars of the rotary atherectomy device to stack during vascular treatment, and a comparison of the acoustic data feature analysis between the surgeon and other surgeons. (Note 5) In step 3, if risk assessment information is output to the surgeon who performed the surgery during a simulated vascular treatment, the program, as described in any of Appendix 2 to 4, includes at least one of the following risk assessment information: a comparison of the acoustic data from the simulated surgery received in step 1 with the exemplary acoustic data; an alert display if there is a large difference between the acoustic data from the simulated surgery received in step 1 and the exemplary acoustic data, or if there are dangerous signs of sound in the acoustic data from the simulated surgery received in step 1; an evaluation of the entire simulated surgery; a timestamp if there was a dangerous operation with a rotary atherectomy device during the simulated surgery; and advice for improving the simulated surgery. (Note 6) The trained model stored in memory takes either acoustic data converted into a color spectrogram image, which is then mapped into a two-dimensional map using an autoencoder, or the spectrogram image converted into a grayscale image, as input and outputs risk assessment information regarding vascular treatment procedures. In the second step, the program converts the acoustic data received in the first step into a color spectrogram image, which is then mapped into a two-dimensional map using an autoencoder, or the spectrogram image converted into a grayscale image, as input to the trained model and obtains the risk assessment information output from this trained model, as described in any of the appendices 1 to 5. (Note 7) An information processing device comprising a processor and memory, which supports procedures related to vascular treatment using a rotary atherectomy device, wherein the memory stores a trained model that takes acoustic data acquired in an environment where vascular treatment is being performed as input and outputs risk assessment information about the procedure related to vascular treatment, and the processor performs a first step of receiving acoustic data, a second step of inputting the acoustic data received in the first step into the trained model and obtaining risk assessment information output from this trained model, and a third step of outputting the risk assessment information obtained in the second step in a form that can be perceived by humans. (Note 8) A computer comprising a processor and memory, wherein a method is performed by the computer to support a procedure related to vascular treatment using a rotary atherectomy device, the memory storing a trained model that takes acoustic data acquired in an environment in which vascular treatment is being performed as input and outputs risk assessment information about the procedure related to vascular treatment, and the processor performing a first step of receiving acoustic data, a second step of inputting the acoustic data received in the first step into the trained model and obtaining risk assessment information output from the trained model, and a third step of outputting the risk assessment information obtained in the second step in a form that can be perceived by a human. (Note 9) A system for supporting vascular treatment procedures using a rotary atherectomy device, comprising: a memory storing a trained model that takes acoustic data acquired in the environment where vascular treatment is being performed as input and outputs risk assessment information regarding the vascular treatment procedure; means for receiving acoustic data; means for inputting the acoustic data received by the means for receiving acoustic data into the trained model and acquiring the risk assessment information output from the trained model; and means for outputting the risk assessment information acquired by the means for acquiring the risk assessment information in a form that can be perceived by humans. [Explanation of symbols]

[0165] 1: System, 10: Terminal device, 20: Server, 25: Memory, 26: Storage, 29: Processor, 2021: Application program, 2022: Patient DB, 2023: Surgery DB, 2024: Acoustic data DB, 2025: Acoustic data, 2026: Trained model, 2033: Acoustic data acquisition module, 2034: Acoustic data analysis module, 2035: Trained model generation module, 2036: Risk assessment acquisition module, 2037: Feedback result generation module, 2038: Feedback result output module, 2039: Presentation control module

Claims

1. A computer comprising a processor and memory, and a program for operating the computer to support procedures related to vascular treatment using a rotary atherectomy device, The memory contains a trained model that takes acoustic data acquired in the environment where the vascular treatment is being performed as input and outputs risk assessment information regarding the procedure related to the vascular treatment. The program is provided to the processor: The first step is to accept the aforementioned acoustic data, The second step involves inputting the acoustic data received in the first step into the trained model and obtaining the risk assessment information output from this trained model, The system then performs a third step in which the risk assessment information acquired in the second step is output in a format that can be perceived by humans. The trained model stored in the memory takes either the acoustic data converted into a color spectrogram image and then mapped into a two-dimensional map using an autoencoder, or the spectrogram image converted into a grayscale image, as input and outputs risk assessment information regarding the vascular treatment procedure. A program that, in the second step, converts the acoustic data received in the first step into a color spectrogram image, inputs the resulting two-dimensional map of the spectrogram image using an autoencoder, or the spectrogram image converted into a grayscale image, into the trained model, and obtains the risk assessment information output from the trained model.

2. In the third step described above, the risk assessment information During the surgical procedure for the aforementioned vascular treatment, the surgeon performing the surgery, The surgeon who performed the aforementioned surgical procedure, and The program according to claim 1, which is output to the surgeon who performed the surgery during a simulated surgery for the vascular treatment.

3. In the third step, if the risk assessment information is output to the surgeon performing the surgery during the surgical procedure of the vascular treatment, the risk assessment information is: Information regarding the risk of complications arising from the aforementioned surgical procedure, and The program according to claim 2, which provides at least one piece of information relating to the risk of the bars of the rotary atherectomy device becoming stuck in the vascular treatment described above.

4. In the third step, if the risk assessment information is output to the surgeon who performed the surgery after the surgical procedure, the risk assessment information is: The risk assessment information for the most recent predetermined number of vascular treatments performed by the operator, In the aforementioned vascular treatment, the degree to which the bar of the rotary atherectomy device tends to get stuck, and The program according to claim 2, which is at least one of comparing the feature analysis of the acoustic data between the aforementioned operator and another operator.

5. In the third step, if the risk assessment information is output to the surgeon who performed the surgery during the simulated surgery for vascular treatment, the risk assessment information is: Comparison of the acoustic data received in the first step during the simulated surgery with the model acoustic data, If there is a large difference between the acoustic data from the simulated surgery received in the first step and the model acoustic data, or if there is a dangerous sound in the acoustic data from the simulated surgery received in the first step, an alert will be displayed, and The program according to claim 2, comprising at least one of the following: an evaluation of the entire simulated surgery; a timestamp indicating any dangerous operation by the rotary atherectomy device during the simulated surgery; and at least one of the following: advice for improvement of the simulated surgery.

6. An information processing device comprising a processor and memory, which is an information processing device that supports procedures related to vascular treatment using a rotary atherectomy device, The memory contains a trained model that takes acoustic data acquired in the environment where the vascular treatment is being performed as input and outputs risk assessment information regarding the procedure related to the vascular treatment. The aforementioned processor, The first step is to accept the aforementioned acoustic data, The second step involves inputting the acoustic data received in the first step into the trained model and obtaining the risk assessment information output from this trained model, The process involves performing a third step in which the risk assessment information acquired in the second step is output in a format that can be perceived by humans, The trained model stored in the memory takes either the acoustic data converted into a color spectrogram image and then mapped into a two-dimensional map using an autoencoder, or the spectrogram image converted into a grayscale image, as input and outputs risk assessment information regarding the vascular treatment procedure. An information processing device that, in the second step, converts the acoustic data received in the first step into a color spectrogram image, inputs the resulting two-dimensional map of the spectrogram image using an autoencoder, or the spectrogram image converted into a grayscale image, into the trained model, and obtains the risk assessment information output from the trained model.

7. A computer comprising a processor and memory, wherein a method is performed by the computer to assist in procedures related to vascular treatment using a rotary atherectomy device, The memory contains a trained model that takes acoustic data acquired in the environment where the vascular treatment is being performed as input and outputs risk assessment information regarding the procedure related to the vascular treatment. The aforementioned processor, The first step is to accept the aforementioned acoustic data, The second step involves inputting the acoustic data received in the first step into the trained model and obtaining the risk assessment information output from this trained model, The process involves performing a third step in which the risk assessment information acquired in the second step is output in a format that can be perceived by humans, The trained model stored in the memory takes either the acoustic data converted into a color spectrogram image and then mapped into a two-dimensional map using an autoencoder, or the spectrogram image converted into a grayscale image, as input and outputs risk assessment information regarding the vascular treatment procedure. A method comprising the second step of converting the acoustic data received in the first step into a color spectrogram image, inputting the resulting two-dimensional map of the spectrogram image using an autoencoder, or the spectrogram image converted into a grayscale image, into the trained model, and obtaining the risk assessment information output from the trained model.

8. A system that supports procedures related to vascular treatment using a rotary atherectomy device, A memory containing a trained model that takes acoustic data acquired in the environment where the aforementioned vascular treatment is performed as input and outputs risk assessment information regarding the procedure related to the aforementioned vascular treatment, The means for receiving the aforementioned acoustic data, The means for receiving the acoustic data inputs the received acoustic data into the trained model, and the means for obtaining the risk assessment information output from the trained model, The means for acquiring the risk assessment information includes means for outputting the acquired risk assessment information in a form that can be perceived by humans. The trained model stored in the memory takes either the acoustic data converted into a color spectrogram image and then mapped into a two-dimensional map using an autoencoder, or the spectrogram image converted into a grayscale image, as input and outputs risk assessment information regarding the vascular treatment procedure. The means for acquiring the risk assessment information is a system that converts the acoustic data received by the means for receiving the acoustic data into a color spectrogram image, inputs the resulting two-dimensional map of the spectrogram image using an autoencoder, or the resulting grayscale image of the spectrogram image, into the trained model, and acquires the risk assessment information output from the trained model.

Images