Diagnosis assistance device, ultrasound endoscope, diagnosis assistance method, and program

Abstract

This diagnosis assistance device is provided with a processor. The processor acquires a first ultrasound image generated by an ultrasonic module, and switches between a first operation mode and a second operation mode in accordance with whether or not reference information is synthesized in the first ultrasound image or whether or not the first ultrasound image is an image obtained in an auxiliary image mode, or in accordance with a setting value for specifying the image quality of the first ultrasound image, in which the first operation mode is such an operation mode that a specific region is detected on the basis of detection assistance information produced using a second ultrasound image obtained in a main image mode, and the second operation mode is such an operation mode that a specific region is detected but a result of the detection is not output or a specific region is not detected.

Description

Diagnosis support device, ultrasonic endoscope, diagnosis support method, and program

[0001] The technology disclosed herein relates to a diagnosis support device, an ultrasonic endoscope, a diagnosis support method, and a program.

[0002] International Publication No. 2020 / 036121 discloses an endoscopic system that includes an identification unit that identifies the type of image of a subject, a recognition unit that performs recognition processing that recognizes the subject using the image, and a notification unit that notifies whether the recognition processing works for the specific type of image identified by the identification unit.

[0003] Japanese Patent Application Laid-Open Publication No. 2021-035442 discloses an ultrasound diagnostic system that provides diagnostic assistance when the image mode is B mode or CF mode, but does not provide diagnostic assistance when the image mode is PW mode.

[0004] Japanese Patent Application Laid-Open Publication No. 2021-083699 discloses an ultrasound diagnostic device that includes a probe that is placed in contact with the breast and outputs a received signal by transmitting and receiving ultrasound waves to and from the breast; an image generation unit that generates an ultrasound image including a mammary gland image, a pectoralis major muscle image, and a boundary image between them based on the received signal; a tilt angle calculation unit that calculates the tilt angle of the boundary image based on the ultrasound image; and support image generation means that generates a support image that assists in operating the probe based on the tilt angle of the boundary image.

[0005] One embodiment of the technology disclosed herein provides a diagnostic support device, an ultrasound endoscope, a diagnostic support method, and a program that can prevent areas other than a specific area from being mistakenly detected as a specific area and prevent visualization of the detection results of a specific area from interfering with diagnosis.

[0006] A first aspect of the technology of the present disclosure is a diagnostic support device that includes a processor, which acquires a first ultrasound image that is generated by an ultrasound module and that shows a target region of observation, and switches between a first operating mode and a second operating mode depending on whether reference information referenced to diagnose the target region of observation is synthesized with the first ultrasound image, whether the first ultrasound image is an image obtained in an auxiliary image mode that is an image mode other than the primary image mode, or a setting value that determines the image quality of the first ultrasound image, wherein the first operating mode is an operating mode that detects a specific region from the first ultrasound image based on detection support information created using the second ultrasound image obtained in the primary image mode, and the second operating mode is an operating mode that either detects the specific region and does not output the detection result, or does not detect the specific region.

[0007] A second aspect of the technology of the present disclosure is a diagnostic support device according to the first aspect, in which the first operating mode is an operating mode used when reference information is not synthesized into the first ultrasound image, and the second operating mode is an operating mode used when reference information is synthesized into the first ultrasound image.

[0008] A third aspect of the technology of the present disclosure is a diagnostic support device according to the first or second aspect, in which the first operating mode is an operating mode used when the first ultrasound image is not an image obtained in auxiliary image mode, and the second operating mode is an operating mode used when the first ultrasound image is an image obtained in auxiliary image mode.

[0009] A fourth aspect of the technology of the present disclosure is a diagnostic assistance device according to any one of the first to third aspects, in which the first operating mode is an operating mode used when the setting value is within a specified range, and the second operating mode is an operating mode used when the setting value is not within the specified range.

[0010] A fifth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to fourth aspects, in which the reference information includes color information that expresses characteristics within the observation target area using color.

[0011] A sixth aspect of the technology of the present disclosure is a diagnostic support device according to the fifth aspect, in which the color information includes a plurality of chromatic color pixels, the first operating mode is an operating mode used when the number of chromatic color pixels among the plurality of chromatic color pixels having a saturation exceeding a first threshold is less than a second threshold, and the second operating mode is an operating mode used when the number is equal to or greater than the second threshold.

[0012] A seventh aspect of the technology of the present disclosure is a diagnosis support device according to any one of the first to sixth aspects, in which the reference information includes text information that assists in observing the observation target region.

[0013] An eighth aspect of the technology of the present disclosure is a diagnosis support device according to any one of the first to seventh aspects, in which the reference information includes a measurement line used for measurement within the observation target region.

[0014] A ninth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to eighth aspects, in which the reference information includes treatment support information that supports treatment using a puncture aspiration method.

[0015] A tenth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to ninth aspects, in which the auxiliary image mode is a first image mode that generates an ultrasound image using high-frequency components contained in reflected waves obtained by emitting ultrasound to an observation target area and reflecting it from the observation target area, or a second image mode that combines a B-mode image with another image.

[0016] An eleventh aspect of the technology of the present disclosure is a diagnostic support device according to the tenth aspect, in which the first image mode is a THI mode, a CH mode, or a CHI mode, and the second image mode is a Doppler mode or an elastography mode.

[0017] A twelfth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to eleventh aspects, in which the setting values ​​include a frequency parameter that adjusts the frequency of ultrasound emitted from the ultrasound module, a depth parameter that adjusts the depth represented in the first ultrasound image, a brightness parameter that adjusts the brightness of the first ultrasound image, a dynamic range parameter that adjusts the dynamic range of the first ultrasound image, and / or a magnification parameter that adjusts the magnification of the digital zoom on the first ultrasound image.

[0018] A thirteenth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to twelfth aspects, in which the ultrasound module has a setting value and the processor acquires the setting value from the ultrasound module.

[0019] A fourteenth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to twelfth aspects, in which a frame including a first ultrasound image is synthesized with a text image capable of identifying a setting value, and a processor identifies the setting value by performing image recognition processing on the text image, and switches between a first operating mode and a second operating mode depending on the identified setting value.

[0020] A fifteenth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to fourteenth aspects, in which a processor detects a specific region from a first ultrasound image using an AI method.

[0021] A sixteenth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to fifteenth aspects, in which the detection support information is a trained model obtained by training a model on training data including a second ultrasound image.

[0022] A seventeenth aspect of the technology of the present disclosure is a diagnostic support device according to any one of the first to sixteenth aspects, in which the processor varies the frequency with which a specific region is detected from the first ultrasound image, the accuracy with which a specific region is detected from the first ultrasound image, and / or the object to be detected as a specific region from the first ultrasound image depending on reference information, an auxiliary image mode, and / or a setting value.

[0023] An eighteenth aspect of the technique of the present disclosure is the diagnosis support device according to any one of the first to seventeenth aspects, in which the ultrasound module is an ultrasound endoscope.

[0024] A nineteenth aspect of the technology of the present disclosure is an ultrasound endoscope comprising a diagnostic support device according to any one of the first to eighteenth aspects and an ultrasound endoscope main body to which an ultrasound diagnostic device is connected.

[0025] A twentieth aspect of the technology of the present disclosure is a diagnostic support method that includes acquiring a first ultrasound image generated by an ultrasound module and showing an observation target area, and switching between a first operating mode and a second operating mode depending on whether reference information referenced to diagnose the observation target area is synthesized with the first ultrasound image, whether the first ultrasound image is an image obtained in an auxiliary image mode that is an image mode other than the primary image mode, or a setting value that determines the image quality of the first ultrasound image, wherein the first operating mode is an operating mode that detects a specific area from the first ultrasound image based on detection support information created using the second ultrasound image obtained in the primary image mode, and the second operating mode is an operating mode that either detects the specific area but does not output a detection result, or does not detect the specific area.

[0026] A 21st aspect of the technology of the present disclosure is a program for causing a computer to execute processing, the processing including acquiring a first ultrasound image generated by an ultrasound module and showing an observation target area, and switching between a first operating mode and a second operating mode depending on whether reference information referenced to diagnose the observation target area is synthesized with the first ultrasound image, whether the first ultrasound image is an image obtained in an auxiliary image mode that is an image mode other than the primary image mode, or a setting value that determines the image quality of the first ultrasound image, wherein the first operating mode is an operating mode that detects a specific area from the first ultrasound image based on detection support information created using the second ultrasound image obtained in the primary image mode, and the second operating mode is an operating mode that either detects the specific area and does not output the detection result, or does not detect the specific area.

[0027] 9A is a conceptual diagram showing an example of an aspect in which an endoscopic system is used. FIG. 9B is a conceptual diagram showing an example of the overall configuration of an endoscopic system. FIG. 9C is a block diagram showing an example of the configuration of an ultrasound endoscope. FIG. 9D is a conceptual diagram showing an example of an aspect in which a trained model is generated by having a model learn a group of B-mode images. FIG. 9E is a conceptual diagram showing an example of processing content of a generation unit. FIG. 9F is a conceptual diagram showing an example of processing content for switching between a detection mode and a non-detection mode. FIG. 9G is a conceptual diagram showing an example of processing content for detecting a lesion area from a B-mode image and displaying the B-mode image on the screen of a display device. FIG. 9H is a conceptual diagram showing an example of processing content for generating a Doppler image and displaying it on the screen of a display device. FIG. 9H is a flowchart showing an example of the flow of diagnostic support processing. FIG. 9H is a continuation of the flowchart shown in FIG. 9A. FIG. 9H is a flowchart showing an example of the flow of diagnostic support processing according to a first modified example. FIG. 9I is a conceptual diagram showing an example of processing content of a generation unit and a control unit according to a second modified example. FIG. 9I is a flowchart showing an example of the flow of diagnostic support processing according to the second modified example. FIG. 9I is a conceptual diagram showing an example of processing content of a generation unit and a control unit according to a third modified example. FIG. 9I is a conceptual diagram showing an example of processing content of a generation unit and a control unit according to a fourth modified example. FIG. 9I is a conceptual diagram showing an example of processing content of a control unit according to a fifth modified example. FIG. 9I is a conceptual diagram showing an example of processing content of a generation unit and a control unit according to a sixth modified example. FIG. 9I is a conceptual diagram showing an example of processing content of a control unit according to the sixth modified example. 13 is a flowchart showing an example of the flow of a diagnosis support process according to a seventh modified example.

[0028] Hereinafter, exemplary embodiments of a diagnosis support device, an ultrasonic endoscope, a diagnosis support method, and a program according to the techniques of the present disclosure will be described with reference to the accompanying drawings.

[0029] First, the terms used in the following description will be explained.

[0030] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". RAM is an abbreviation for "Random Access Memory". NVM is an abbreviation for "Non-volatile memory". EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". PC is an abbreviation for "Personal Computer". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network". AI is an abbreviation for "Artificial Intelligence". BLI is an abbreviation for "Blue Light Imaging". LCI is an abbreviation for "Linked Color Imaging". NN is an abbreviation for "Neural Network". CNN is an abbreviation for "Convolutional neural network". R-CNN is an abbreviation for "Region based Convolutional Neural Network".YOLO is an abbreviation for "You only Look Once". RNN is an abbreviation for "Recurrent Neural Network". FCN is an abbreviation for "Fully Convolutional Network". THI is an abbreviation for "Tissue Harmonic Imaging". CH is an abbreviation for "Compound Harmonic". CHI is an abbreviation for "Contrast Harmonic Imaging".

[0031] As an example, as shown in FIG. 1 , an endoscopic system 10 includes an ultrasonic endoscope 12 and a display device 14. The ultrasonic endoscope 12 is a convex type ultrasonic endoscope and includes an ultrasonic endoscope main body 16 and a processing device 18. The ultrasonic endoscope 12 is an example of an "ultrasound module" and an "ultrasound endoscope" according to the technology of the present disclosure. The processing device 18 is an example of a "diagnostic support device" according to the technology of the present disclosure. The ultrasonic endoscope main body 16 is an example of an "ultrasound endoscope main body" according to the technology of the present disclosure.

[0032] In this embodiment, a convex type ultrasonic endoscope is given as an example of the ultrasonic endoscope 12, but this is merely an example, and the technology of the present disclosure is also applicable to a radial type ultrasonic endoscope.

[0033] The ultrasonic endoscope main body 16 is used by, for example, a doctor 20. The processing device 18 is connected to the ultrasonic endoscope main body 16 and exchanges various signals with the ultrasonic endoscope main body 16. That is, the processing device 18 controls the operation of the ultrasonic endoscope main body 16 by outputting signals to the ultrasonic endoscope main body 16, and performs various signal processing on signals input from the ultrasonic endoscope main body 16.

[0034] The ultrasound endoscope 12 is a device for performing medical treatment (e.g., diagnosis and / or treatment) on a target area (e.g., an organ such as the pancreas) inside the body of a subject 22, and generates and outputs an ultrasound image 24 showing the observation area including the target area.

[0035] For example, when observing an observation region inside the body of the subject 22, the doctor 20 inserts the ultrasound endoscope body 16 into the body of the subject 22 through the mouth or nose (mouth in the example shown in FIG. 1 ) and emits ultrasound waves at a position such as the stomach or duodenum. The ultrasound endoscope body 16 emits ultrasound waves toward the observation region inside the body of the subject 22 and detects the reflected waves obtained when the emitted ultrasound waves are reflected from the observation region.

[0036] In the example shown in Figure 1, an upper gastrointestinal endoscopy is being performed, but the technology of the present disclosure is not limited to this and can also be applied to lower gastrointestinal endoscopy or bronchial endoscopy, etc.

[0037] The processing device 18 generates an ultrasonic image 24 based on the reflected waves detected by the ultrasonic endoscope body 16 and outputs it to the display device 14 or the like.

[0038] The display device 14 displays various information, including images, under the control of the processing device 18. Examples of the display device 14 include a liquid crystal display and an EL display. An ultrasound image 24 generated by the processing device 18 is displayed as a moving image on a screen 26 of the display device 14. In the example shown in FIG. 1 , the ultrasound image 24 on the screen 26 includes a lesion area 25 indicating a location corresponding to a lesion. In addition, as will be described in more detail below, in the example shown in FIG. 1 , a rectangular detection frame 27A is displayed on the screen 26, which can identify the position of the lesion area 25 in the ultrasound image 24.

[0039] By observing the ultrasound image 24 displayed on the screen 26, the doctor 20 determines whether or not a lesion is present in the observation area, and if a lesion is found, identifies the location of the lesion within the observation area by referring to the detection frame 27A in the ultrasound image 24.

[0040] 1 shows an example in which the ultrasound image 24 is displayed on the screen 26 of the display device 14, but this is merely an example and the ultrasound image 24 may be displayed on a display device other than the display device 14 (for example, the display of a tablet terminal). The ultrasound image 24 may also be stored in a computer-readable non-transitory storage medium (for example, a flash memory, a HDD, and / or a magnetic tape).

[0041] The display device 14 displays an ultrasound image 24 according to an image mode selected by the doctor 20. Here, the image mode refers to a display mode in which the reflected waves detected by the ultrasound endoscope main body 16 are imaged and displayed on the display device 14. In this embodiment, the image modes selected by the doctor 20 are two types: B-mode (Brightness mode) and Doppler mode. B-mode is an example of a "main image mode" according to the technology of the present disclosure, and Doppler mode is an example of an "auxiliary image mode, which is an image mode other than the main image mode" according to the technology of the present disclosure.

[0042] B-mode is an imaging mode that converts the intensity of reflected waves into brightness and displays them as a two-dimensional tomographic image (hereinafter referred to as a "B-mode image"). Doppler mode is an imaging mode that utilizes the Doppler effect to display identified hemodynamics as color information superimposed on a B-mode image. Hereinafter, an image generated in Doppler mode, i.e., an image obtained by superimposing color information indicating hemodynamics on a B-mode image, will be referred to as a "Doppler image." Furthermore, hereinafter, for convenience of explanation, when there is no need to distinguish between images generated in different imaging modes, such as a B-mode image and a Doppler image, they will simply be referred to as an "ultrasound image 24."

[0043] The ultrasound image 24 is a moving image including a plurality of frames generated at a frame rate determined according to the image mode. The frame rate of the Doppler mode is lower than the frame rate of the B mode. Note that although a moving image is illustrated here, this is merely an example, and the technology of the present disclosure also applies when the ultrasound image 24 is a still image.

[0044] As shown in FIG. 2 as an example, the ultrasound endoscope main body 16 includes a control section 28 and an insertion section 30. The insertion section 30 is tubular. The insertion section 30 has a distal end portion 32, a bending section 34, and a flexible section 36. The distal end portion 32, the bending section 34, and the flexible section 36 are arranged in this order from the distal end to the proximal end of the insertion section 30. The flexible section 36 is formed of a long, flexible material and connects the control section 28 and the bending section 34. The bending section 34 partially bends and rotates around the axis of the insertion section 30 when the control section 28 is operated. As a result, the insertion section 30 is advanced deeper into the hollow organ while bending and rotating around the axis of the insertion section 30 in accordance with the shape of the hollow organ (e.g., the shape of the duodenal tract).

[0045] The distal end portion 32 is provided with an ultrasonic probe 38 and a treatment tool opening 40. The ultrasonic probe 38 is provided on the distal side of the distal end portion 32. The ultrasonic probe 38 is a convex ultrasonic probe that emits ultrasonic waves and receives reflected waves obtained when the emitted ultrasonic waves are reflected from the observation target region.

[0046] The treatment tool opening 40 is formed closer to the base end of the distal end portion 32 than the ultrasonic probe 38. The treatment tool opening 40 is an opening for allowing a treatment tool 42 to protrude from the distal end portion 32. A treatment tool insertion port 44 is formed in the operation section 28, and the treatment tool 42 is inserted into the insertion section 30 from the treatment tool insertion port 44. The treatment tool 42 passes through the insertion section 30 and protrudes from the treatment tool opening 40 to the outside of the ultrasonic endoscope body 16. The treatment tool opening 40 also functions as a suction port for sucking blood, internal waste, and the like.

[0047] 2, a puncture needle is shown as the treatment tool 42. However, this is merely an example, and the treatment tool 42 may also be grasping forceps and / or a sheath, etc.

[0048] 2, an illumination device 46 and a camera 48 are provided at the distal end portion 32. The illumination device 46 emits light. Examples of the type of light emitted from the illumination device 46 include visible light (e.g., white light), invisible light (e.g., near-infrared light), and / or special light. Examples of the special light include light for BLI and / or light for LCI.

[0049] The camera 48 optically captures images of the inside of a hollow organ. An example of the camera 48 is a CMOS camera. The CMOS camera is merely an example, and other types of cameras such as a CCD camera may also be used. The image captured by the camera 48 may be displayed on the display device 14, on a display device other than the display device 14 (e.g., the display of a tablet terminal), or stored in a storage medium (e.g., a flash memory, a HDD, and / or a magnetic tape).

[0050] The ultrasonic endoscope 12 includes a processing device 18 and a universal cord 50. The universal cord 50 has a base end 50A and a tip end 50B. The base end 50A is connected to the operation unit 28. The tip end 50B is connected to the processing device 18. In other words, the ultrasonic endoscope body 16 and the processing device 18 are connected via the universal cord 50.

[0051] The endoscope system 10 includes a reception device 52. The reception device 52 is connected to the processing device 18. The reception device 52 receives instructions from a user. Examples of the reception device 52 include an operation panel having a plurality of hard keys and / or a touch panel, a keyboard, a mouse, a trackball, a foot switch, a smart device, and / or a microphone.

[0052] The processing device 18 performs various signal processing operations and transmits and receives various signals to and from the ultrasonic endoscope main body 16, etc., in accordance with instructions received by the receiving device 52. For example, the processing device 18 causes the ultrasonic probe 38 to emit ultrasonic waves in accordance with instructions received by the receiving device 52, and generates and outputs an ultrasonic image 24 (see FIG. 1 ) based on the reflected waves received by the ultrasonic probe 38.

[0053] The display device 14 is also connected to the processing device 18. The processing device 18 controls the display device 14 in accordance with instructions received by the reception device 52. As a result, for example, an ultrasound image 24 generated by the processing device 18 is displayed on the screen 26 of the display device 14 (see FIG. 1).

[0054] 3, the processing device 18 includes a computer 54, an input / output interface 56, a transmitting / receiving circuit 58, and a communication module 60. The computer 54 is an example of a "computer" according to the technology of the present disclosure.

[0055] The computer 54 includes a processor 62, a RAM 64, and an NVM 66. The input / output interface 56, the processor 62, the RAM 64, and the NVM 66 are connected to a bus 68.

[0056] The processor 62 controls the entire processing device 18. For example, the processor 62 has a CPU and a GPU, and the GPU operates under the control of the CPU and is mainly responsible for executing image processing. The processor 62 may be one or more CPUs that integrate a GPU function, or one or more CPUs that do not integrate a GPU function. The processor 62 may also include a multi-core CPU or a TPU. The processor 62 is an example of a "processor" according to the technology of the present disclosure.

[0057] The RAM 64 is a memory that temporarily stores information and is used as a work memory by the processor 62. The NVM 66 is a nonvolatile storage device that stores various programs, various parameters, and the like. Examples of the NVM 66 include flash memory (e.g., EEPROM) and / or an SSD. Note that flash memory and SSD are merely examples, and the NVM 66 may be other nonvolatile storage devices such as an HDD, or may be a combination of two or more types of nonvolatile storage devices.

[0058] The input / output interface 56 is connected to the reception device 52, and the processor 62 acquires instructions received by the reception device 52 via the input / output interface 56 and executes processing according to the acquired instructions.

[0059] A transmission / reception circuit 58 is connected to the input / output interface 56. The transmission / reception circuit 58 generates a pulsed ultrasonic emission signal 70 in accordance with instructions from the processor 62 and outputs the signal to the ultrasonic probe 38. The ultrasonic probe 38 converts the ultrasonic emission signal 70 input from the transmission / reception circuit 58 into ultrasonic waves and emits the ultrasonic waves toward an observation region 72 of the subject 22. The ultrasonic probe 38 receives reflected waves obtained when the ultrasonic waves emitted from the ultrasonic probe 38 are reflected by the observation region 72, converts the reflected waves into reflected wave signals 74, which are electrical signals, and outputs the reflected waves to the transmission / reception circuit 58. The transmission / reception circuit 58 digitizes the reflected wave signals 74 input from the ultrasonic probe 38 and outputs the digitized reflected wave signals 74 to the processor 62 via the input / output interface 56. The processor 62 generates an ultrasonic image 24 (see FIG. 1 ) showing the appearance of the observation region 72 based on the reflected wave signals 74 input from the transmission / reception circuit 58 via the input / output interface 56.

[0060] Although not shown in Fig. 3 , the lighting device 46 (see Fig. 2 ) is also connected to the input / output interface 56. The processor 62 controls the lighting device 46 via the input / output interface 56 to change the type of light emitted from the lighting device 46 and adjust the amount of light. Although not shown in Fig. 3 , the camera 48 (see Fig. 2 ) is also connected to the input / output interface 56. The processor 62 controls the camera 48 via the input / output interface 56 and acquires, via the input / output interface 56, images of the inside of the body of the subject 22 captured by the camera 48.

[0061] A communication module 60 is connected to the input / output interface 56. The communication module 60 is an interface including a communication processor, an antenna, etc. The communication module 60 is connected to a network (not shown) such as a LAN or a WAN, and controls communication between the processor 62 and external devices.

[0062] The display device 14 is connected to the input / output interface 56, and the processor 62 controls the display device 14 via the input / output interface 56 to cause the display device 14 to display various information.

[0063] The input / output interface 56 is connected to the reception device 52, and the processor 62 acquires instructions received by the reception device 52 via the input / output interface 56 and executes processing according to the acquired instructions.

[0064] The NVM 66 stores a diagnostic assistance program 76 and a trained model 78. The processor 62 performs diagnostic assistance processing by reading the diagnostic assistance program 76 from the NVM 66 and executing the read diagnostic assistance program 76 on the RAM 64. The diagnostic assistance processing is processing that detects a lesion from the observation target region 72 using an AI method and assists the doctor 20 (see FIG. 1 ) in making a diagnosis based on the detection results.

[0065] The processor 62 performs diagnostic assistance processing to detect a location corresponding to a lesion from the ultrasound image 24 (see FIG. 1 ) in accordance with the trained model 78, thereby detecting a lesion from the observation region 72. The diagnostic assistance processing is realized by the processor 62 operating as a generator 62A, a detector 62B, and a controller 62C in accordance with a diagnostic assistance program 76 executed on the RAM 64.

[0066] The diagnostic assistance program 76 is an example of a "program" according to the technology of the present disclosure. The trained model 78 is a trained model having a data structure used in the process of detecting a lesion from the ultrasound image 24. The trained model 78 is an example of "detection assistance information" and "trained model" according to the technology of the present disclosure.

[0067] As an example, as shown in FIG. 4 , a trained model 78 is generated by training an untrained model 80. A B-mode image group 82 is used as training data for training the model 80. The B-mode image group 82 is made up of a plurality of B-mode images 82A that are different from one another. The B-mode image 82A is an example of a "second ultrasound image" according to the technology of the present disclosure. The B-mode image group 82 is an example of "training data" according to the technology of the present disclosure.

[0068] An example of the model 80 is a mathematical model using a neural network (NN). Examples of the type of NN include YOLO, R-CNN, and FCN. The NN used in the model 80 may be, for example, YOLO, R-CNN, or a combination of FCN and RNN. RNN is suitable for learning multiple images obtained in a time series. Note that the types of NNs listed here are merely examples, and other types of NNs that enable object detection by learning images may also be used.

[0069] A plurality of B-mode images 82A show lesions. That is, the B-mode images 82A have lesion regions 84, which are locations corresponding to the lesions. Annotations 86 are attached to the B-mode images 82A. The annotations 86 are information that can identify the position of the lesion regions 84 within the B-mode images 82A (for example, information including a plurality of coordinates that can identify the position of a rectangular frame circumscribing the lesion regions 84).

[0070] For ease of explanation, information capable of identifying the position of the lesion region 84 in the B-mode image 82A is illustrated here as an example of the annotation 86, but this is merely an example. For example, the annotation 86 may include other types of information that identify the lesion shown in the B-mode image 82A, such as information capable of identifying the type of lesion shown in the B-mode image 82A.

[0071] For ease of explanation, the processing using the trained model 78 will be described below as processing that is actively performed by the trained model 78. That is, for ease of explanation, the trained model 78 will be described as a function that processes input information and outputs processing results. Also, for ease of explanation, a portion of the processing that trains the model 80 will be described below as processing that is actively performed by the model 80. That is, for ease of explanation, the model 80 will be described as a function that processes input information and outputs processing results.

[0072] Each B-mode image 82A included in the B-mode image group 82 is input to the model 80. In response to this, the model 80 predicts the position of the lesion region 84 from the input B-mode image 82A and outputs the prediction result. The prediction result includes information that can identify the position predicted by the model 80 as the position of the lesion region 84 within the B-mode image 82A. An example of the information that can identify the position predicted by the model 80 is information that includes a plurality of coordinates that can identify the position of a bounding box surrounding the region predicted as the position where the lesion region 84 is to be located (i.e., the position of the bounding box within the B-mode image 82A).

[0073] The model 80 is adjusted according to the error between the annotations 86 assigned to the B-mode images 82A input to the model 80 and the prediction results output from the model 80. That is, the model 80 is optimized by adjusting a plurality of optimization variables (e.g., a plurality of connection weights and a plurality of offset values) in the model 80 so as to minimize the error, thereby generating the trained model 78. That is, the data structure of the trained model 78 is obtained by having the model 80 learn a plurality of different B-mode images 82A to which the annotations 86 are assigned.

[0074] As described above, the trained model 78 is a mathematical model generated by training the model 80 with a group of B-mode images 82. Therefore, for example, the trained model 78 can be effectively used to detect a lesion area 25 (see FIG. 1 ) from a B-mode image generated as the ultrasound image 24. However, when detecting a lesion area 25 from a Doppler image generated as the ultrasound image 24, the likelihood of false detection is higher than when detecting a lesion area 25 from a B-mode image. This is because the model 80 has not learned the color information contained in the Doppler image. Furthermore, if the lesion area 25 is detected from the Doppler image and a detection frame 27A (see FIG. 1 ) is displayed in the Doppler image as a detection result, the color information contained in the Doppler image (i.e., colorized information indicating hemodynamics) and the detection frame 27A are visualized in a mixed state, which may hinder the doctor 20 from making a diagnosis.

[0075] Therefore, the processing device 18 according to this embodiment performs a diagnostic support process as shown in Figures 5 to 9B as an example. An example of the diagnostic support process will be specifically described below.

[0076] 5, when an instruction (hereinafter referred to as an "imaging mode instruction") to select a type of imaging mode (here, as an example, B-mode or Doppler mode) is received by the reception device 52, the generation unit 62A sets B-mode or Doppler mode in accordance with the imaging mode instruction. In this embodiment, the B-mode is used by the doctor 20 as the main imaging mode, and the Doppler mode is used by the doctor 20 as the sub-imaging mode (i.e., auxiliary imaging mode).

[0077] The generator 62A acquires a B-mode image 24A by acquiring a reflected wave signal 74 from the transmitter / receiver circuit 58 in B mode and generating the B-mode image 24A based on the acquired reflected wave signal 74. The B-mode image 24A shows an observation region 72. That is, the B-mode image 24A is an image showing a two-dimensional cross section of the observation region 72. In the example shown in FIG. 5 , the generator 62A generates a B-mode image 24A including a lesion region 25. However, the generator 62A may also generate a B-mode image 24A that does not show a lesion. The B-mode image 24A is an example of a "first ultrasound image" according to the technology of the present disclosure. Furthermore, the lesion region 25 is an example of a "specific region" according to the technology of the present disclosure.

[0078] The generator 62A acquires a reflected wave signal 74 from the transmitter / receiver circuitry 58 in Doppler mode and generates a Doppler image 24B based on the acquired reflected wave signal 74. The Doppler image 24B is an image obtained by superimposing color information 24B1, which expresses characteristics within the observation region 72 (see FIG. 3 ) in color (i.e., chromatic colors), on the B-mode image 24A. For example, the color information 24B1 is information indicating hemodynamics determined using the Doppler effect. The color information 24B1 is information referenced by the physician 20 to diagnose the observation region 72 (see FIG. 3 ).

[0079] The Doppler image 24B is an example of an "image obtained in auxiliary image mode" according to the technology of the present disclosure. The color information 24B1 is an example of "reference information," "color information," and "another image" according to the technology of the present disclosure. The superimposition of the color information 24B1 on the B-mode image 24A is an example of "combination" according to the technology of the present disclosure.

[0080] As an example, as shown in FIG. 6 , the control unit 62C switches between detection mode and non-detection mode depending on whether the ultrasound image 24 generated by the generation unit 62A is a B-mode image 24A or a Doppler image 24B. In the present embodiment, the control unit 62C determines whether the ultrasound image 24 generated by the generation unit 62A is a B-mode image 24A or a Doppler image 24B according to the image mode instruction received by the reception device 52, and switches between detection mode and non-detection mode depending on the determination result. In other words, the control unit 62C switches between detection mode and non-detection mode depending on the image mode set according to the image mode instruction received by the reception device. For example, the control unit 62C sets the detection mode in the case of B-mode and sets the non-detection mode in the case of Doppler mode.

[0081] The detection mode is an operation mode in which a lesion area 25 (see FIG. 5) is detected from a B-mode image 24A (see FIG. 5) using an AI method, i.e., an operation mode in which a lesion area 25 (see FIG. 5) is detected from the B-mode image 24A based on a trained model 78 created using a B-mode image group 82. The detection mode is used when color information 24B1 is not superimposed on the B-mode image 24A. In other words, the detection mode is used when a Doppler image 24B is not generated by the generation unit 62A (i.e., when a B-mode image 24A is generated by the generation unit 62A).

[0082] The non-detection mode is an operating mode in which the lesion area 25 is not detected from the ultrasound image 24. Here, an operating mode in which the lesion area 25 is not detected from the ultrasound image 24 is illustrated as an example of the non-detection mode, but the technology of the present disclosure is not limited to this. For example, the non-detection mode may be an operating mode in which the lesion area 25 is detected from the ultrasound image 24 but the detection result is not output (in other words, an operating mode in which the lesion area 25 is detected using an AI method in the background but the detection result is not visualized). The non-detection mode is used when color information 24B1 is superimposed on the B-mode image 24A. In other words, the non-detection mode is used when the Doppler image 24B is generated by the generation unit 62A.

[0083] As shown in FIG. 7 as an example, in the detection mode, the detector 62B detects a lesion from the B-mode image 24A generated by the generator 62A according to the trained model 78. That is, the detector 62B determines the presence or absence of a lesion area 25 in the B-mode image 24A according to the trained model 78, and generates position identification information 128 (e.g., information including multiple coordinates specifying the position of the lesion area 126) that identifies the position of the lesion area 25 if the lesion area 25 is present in the B-mode image 24A. Here, the lesion detection process by the detector 62B will be described mainly with reference to the trained model 78. When the B-mode image 24A generated by the generator 62A is input, the trained model 78 determines the presence or absence of a lesion area 25 in the input B-mode image 24A. When the trained model 78 determines that a lesion area 25 is present in the B-mode image 24A (i.e., when a lesion is detected in the B-mode image 24A), it outputs position identification information 27. Detection frame 27A is a rectangular frame corresponding to a bounding box (e.g., a bounding box with the highest reliability score) used when trained model 78 detects lesion region 25 from B-mode image 24A. In other words, detection frame 27A is a frame that surrounds lesion region 25 detected by trained model 78.

[0084] The detector 62B assigns a detection frame 27A to the B-mode image 24A corresponding to the position identification information 27 output from the trained model 78 (i.e., the B-mode image 24A input to the trained model 78 for output of the position identification information 27) in accordance with the position identification information 27. That is, the detector 62B assigns the detection frame 27A to the B-mode image 24A corresponding to the position identification information 27 output from the trained model 78 by superimposing the detection frame 27A on the B-mode image 24A so as to surround the lesion area 25. When the trained model 78 determines that a lesion area 25 exists in the B-mode image 27, the detector 62B outputs the B-mode image 24A with the assigned detection frame 27A to the controller 62C. When the trained model 78 determines that a lesion area 25 does not exist in the B-mode image 24A, the detector 62B outputs the B-mode image 24A without the assigned detection frame 27A to the controller 62C.

[0085] The control unit 62C displays the B-mode image 24A input from the detection unit 62B (i.e., the B-mode image 24A reflecting the detection result of the detection unit 62B) on the screen 26 of the display device 14. When a lesion is captured in the B-mode image 24A, the B-mode image 24A to which a detection frame 27A surrounding the lesion area 25 is added (i.e., the B-mode image 24A to which the detection frame 27A is superimposed) is displayed on the screen 26. On the other hand, when a lesion is not captured in the B-mode image 24A, the B-mode image 24A to which the detection frame 27A is not added (i.e., the B-mode image 24A output from the trained model 78) is displayed on the screen 26.

[0086] As an example, as shown in FIG. 8, in the non-detection mode, the control unit 62C acquires the Doppler image 24B generated by the generation unit 62A and displays the acquired Doppler image 24B on the screen 26 of the display device 14.

[0087] Next, the operation of the endoscope system 10 will be described with reference to FIGS. 9A and 9B.

[0088] 9A and 9B show an example of the flow of diagnostic support processing performed by the processor 62 of the processing device 18 on the condition that diagnosis using the endoscopic system 10 has started (for example, that ultrasound emission from the ultrasonic endoscope 12 has started). The flow of diagnostic support processing shown in Figures 9A and 9B is an example of a "diagnostic support method" according to the technology of the present disclosure.

[0089] In the diagnostic support process shown in Fig. 9A, first, in step ST10, the control unit 62C determines whether the currently set image mode is B-mode. If the currently set image mode is not B-mode (i.e., Doppler mode) in step ST10, the determination is negative, and the diagnostic support process proceeds to step ST26 shown in Fig. 9B. If the currently set image mode is B-mode in step ST10, the determination is positive, and the diagnostic support process proceeds to step ST12.

[0090] In step ST12, the control unit 62C determines whether the currently set operation mode is the non-detection mode. If the currently set operation mode is not the non-detection mode (i.e., the detection mode) in step ST12, the determination is negative, and the diagnostic support processing proceeds to step ST16. If the currently set operation mode is the non-detection mode in step ST12, the determination is positive, and the diagnostic support processing proceeds to step ST14.

[0091] In step ST14, the control unit 62C switches the operation mode from the non-detection mode to the detection mode. After the process of step ST14 is executed, the diagnostic support process proceeds to step ST16.

[0092] In step ST16, the generation unit 62A generates a B-mode image 24A based on the reflected wave signal 74 (see FIG. 5) input from the transmission / reception circuitry 58. After the processing of step ST16 is executed, the diagnostic support processing proceeds to step ST18.

[0093] In step ST18, the detection unit 62B inputs the B-mode image 24A generated in step ST16 into the trained model 78. After the processing of step ST18 is executed, the diagnostic support processing proceeds to step ST20.

[0094] In step ST20, the detection unit 62B determines whether or not a lesion is captured in the B-mode image 24A input to the trained model 78 in step ST18, using the trained model 78. If a lesion is captured in the B-mode image 24A, the trained model 78 outputs position identification information 27.

[0095] In step ST20, if a lesion is not captured in the B-mode image 24A, the determination is negative and the diagnostic support process proceeds to step ST24. In step ST20, if a lesion is captured in the B-mode image 24A, the determination is positive and the diagnostic support process proceeds to step ST22.

[0096] If the determination in step ST20 is affirmative, the detection unit 62B generates a detection frame 27A based on the position identification information 128 output from the trained model 78 and superimposes the detection frame 27A on the B-mode image 24A generated in step ST16 so as to surround the lesion region 25. Then, in step ST22, the control unit 62C displays the B-mode image 24A, in which the lesion region 25 is surrounded by the detection frame 27A, on the screen 26 of the display device 14. Because the lesion region 25 in the B-mode image 24A is surrounded by the detection frame 27A, the doctor 20 can visually determine where the lesion is located in the B-mode image 24A. After the processing in step ST22 is executed, the diagnostic support processing proceeds to step ST26 shown in FIG. 9B .

[0097] In step ST24, the control unit 62C displays the radial ultrasound image 24 generated in step ST16 on the screen 26 of the display device 14. In this case, since the detection frame 27A is not added to the B-mode image 24A, the doctor 20 can visually recognize that no lesion is visible in the B-mode image 24A. After the processing of step ST24 is executed, the diagnostic support processing proceeds to step ST26 shown in FIG. 9B.

[0098] 9B, the control unit 62C determines whether the currently set operation mode is the detection mode. If the currently set operation mode is not the detection mode (i.e., the non-detection mode) in step ST26, the determination is negative, and the diagnostic support process proceeds to step ST30. If the currently set operation mode is the detection mode in step ST26, the determination is positive, and the diagnostic support process proceeds to step ST28.

[0099] In step ST28, the control unit 62C switches the operation mode from the non-detection mode to the detection mode. After the process of step ST28 is executed, the diagnostic support process proceeds to step ST30.

[0100] In step ST30, the generation unit 62A generates a Doppler image 24B based on the reflected wave signal 74 (see FIG. 5) input from the transmission / reception circuitry 58. After the processing of step ST30 is executed, the diagnostic support processing proceeds to step ST32.

[0101] In step ST32, the control unit 62C displays the Doppler image 24B generated in step ST30 on the screen 26 of the display device 14. This allows the doctor 20 to visually recognize the hemodynamics from the color information 24B1 (see FIG. 8 ) included in the Doppler image 24B displayed on the screen 26. Furthermore, because the Doppler image 24B does not include the detection frame 27A, the detection frame 27A does not interfere with observation of the color information 24B1. After the processing of step ST32 is executed, the diagnostic support processing proceeds to step ST34.

[0102] In step ST34, the control unit 62C determines whether a condition for terminating the diagnostic support processing (hereinafter referred to as a "diagnostic support termination condition") has been satisfied. One example of the diagnostic support termination condition is that an instruction to terminate the diagnostic support processing has been accepted by the acceptance device 52. If the diagnostic support termination condition has not been satisfied in step ST34, the determination is negative, and the diagnostic support processing proceeds to step ST10 shown in FIG. 9A. If the diagnostic support termination condition has been satisfied in step ST34, the determination is positive, and the diagnostic support processing ends.

[0103] As described above, the endoscope system 10 can selectively switch between a detection mode and a non-detection mode. The detection mode is an operating mode in which the lesion area 25 is detected from the B-mode image 24A using an AI method that uses the trained model 78. The non-detection mode is an operating mode in which the lesion area 25 is not detected from the ultrasound image 24. The trained model 78 is created using the B-mode image group 82 as training data. Therefore, when the process of detecting the lesion area 25 from the Doppler image 24B is performed according to the trained model 78, there is a higher possibility that an area other than the lesion area 25 will be erroneously detected as the lesion area 25 compared to when the process of detecting the lesion area 25 from the B-mode image 24A is performed according to the trained model 78. Furthermore, if the process of detecting the lesion area 25 from the Doppler image 24B is performed according to the trained model 78 and a detection frame 27A is displayed in the Doppler image 24B as a detection result, the presence of the detection frame 27A may interfere with the physician 20's observation of the color information 24B1. Therefore, in the endoscope system 10, the detection mode and the non-detection mode are switched depending on whether the B-mode image 24A is generated. In other words, the detection mode and the non-detection mode are switched depending on whether the B-mode or the Doppler mode is set as the image mode. This prevents an area other than the lesion area 25 from being erroneously detected as the lesion area 25, and also prevents the visualization of the detection result of the lesion area 25 (i.e., the presence of the detection frame 27A) from interfering with the diagnosis.

[0104] In the endoscope system 10, the detection mode is an operating mode used when the color information 24B1 is not superimposed on the B-mode image 24A, and the non-detection mode is an operating mode used when the color information 24B1 is superimposed on the B-mode image 24A. In other words, the detection mode is an operating mode used when the imaging mode is not the Doppler mode (i.e., the B-mode), and the non-detection mode is an operating mode used when the imaging mode is the Doppler mode. Therefore, by selectively using the detection mode and the non-detection mode in the above manner, it is possible to prevent an area other than the lesion area 25 from being erroneously detected as the lesion area 25 due to the presence of the color information 24B1. In addition, in the non-detection mode, the color information 24B1 and the detection frame 27A are not displayed together on the screen 26, so that visualization of the detection result of the lesion area 25 (i.e., the presence of the detection frame 27A) is prevented from interfering with observation of the color information 24B1 included in the Doppler image 24B.

[0105] [First Modification] In the above embodiment, an example was described in which the detection mode and the non-detection mode were switched depending on whether the B mode or the Doppler mode was set as the imaging mode, but the technology of the present disclosure is not limited to this. For example, the detection mode and the non-detection mode may be switched depending on whether the color information 24B1 is superimposed on the image generated by the generation unit 62A.

[0106] In this case, for example, the diagnostic support process shown in Fig. 10 is performed by the processor 62. The flowchart shown in Fig. 10 differs from the flowchart shown in Fig. 9A in that the process of step ST100 is applied instead of the process of step ST10.

[0107] In step ST100 shown in FIG. 10 , the control unit 62C performs image analysis processing on the image generated by the generation unit 62A to determine whether color information 24B1 is superimposed on the image generated by the generation unit 62A. If color information 24B1 is not superimposed on the image generated by the generation unit 62A in step ST100, the determination is affirmative, and the diagnostic support processing proceeds to step ST12. If color information 24B1 is superimposed on the image generated by the generation unit 62A in step ST100, the determination is negative, and the processing proceeds to step ST26 shown in FIG. 9B . As a result, if color information 24B1 is not superimposed on the image generated by the generation unit 62A, the detection mode is set as the operating mode, and if color information 24B1 is superimposed on the image generated by the generation unit 62A, the non-detection mode is set as the operating mode. Therefore, the same effects as those of the above embodiment can be obtained.

[0108] [Second Modification] In the above embodiment, an example was given in which detection mode is set as the operating mode for a B-mode image 24A, and detection of the lesion area 25 is performed on the B-mode image 24A using an AI method. However, even if the image is a B-mode image 24A, non-detection mode may be set as the operating mode depending on the conditions.

[0109] As an example, as shown in FIG. 11 , when the generation unit 62A generates a B-mode image 24A including character information 88, a measurement line 90, and auxiliary treatment information 92, the control unit 62C sets the non-detection mode as the operating mode. The character information 88, the measurement line 90, and the auxiliary treatment information 92 are an example of "reference information" according to the technology of the present disclosure. The character information 88 is an example of "character information" according to the technology of the present disclosure. The measurement line 90 is an example of "measurement line" according to the technology of the present disclosure. The auxiliary treatment information 92 is an example of "auxiliary treatment information" according to the technology of the present disclosure.

[0110] Here, the text information 88 refers to, for example, a text image (i.e., an image expressing text) that assists in the observation of the observation target region 72 (see FIG. 3 ). Assisting in the observation of the observation target region 72 means, for example, that the text information 88 is useful for the physician 20 in identifying the presence or absence and location of a lesion from the B-mode image 24A. The concept of "text" according to this embodiment also includes numbers and symbols. Examples of text according to this embodiment include characters defined in Unicode. In the example shown in FIG. 11 , information visualizing the dimensions of the lesion region 25 is shown as an example of text information 88.

[0111] Measurement line 90 is, for example, a line used for measurements within observation region 72 (see FIG. 3 ). In the example shown in FIG. 11 , a dimension line that can visually identify the length in one direction of lesion region 25 is shown as an example of measurement line 90. Note that, although a dimension line is shown here, this is merely an example, and visualization information other than dimensionality (for example, a line used to measure a specified portion other than lesion region 25) may also be used.

[0112] The treatment auxiliary information 92 is information that assists treatment using the puncture aspiration method. In the example shown in Fig. 11, an arrow is shown indicating the direction in which a puncture needle (see Fig. 2) is inserted when the treatment tool 42 is used. Note that although an arrow is shown here, this is merely an example, and visualization information other than an arrow (for example, a dashed line or a blinking line) may also be used.

[0113] The control unit 62C performs image analysis processing on the image generated by the generation unit 62A to determine whether or not the image generated by the generation unit 62A is superimposed with text information 88, measurement lines 90, and / or auxiliary treatment information 92. In the example shown in Fig. 11 , the control unit 62C determines whether or not the B-mode image 24A generated by the generation unit 62A is superimposed with text information 88, measurement lines 90, and / or auxiliary treatment information 92.

[0114] Then, when the text information 88, the measurement line 90, and the auxiliary treatment information 92 are not superimposed on the B-mode image 24A generated by the generation unit 62A, the control unit 62C sets the detection mode as the operation mode. Furthermore, when the text information 88, the measurement line 90, and / or the auxiliary treatment information 92 are superimposed on the B-mode image 24A generated by the generation unit 62A, the control unit 62C sets the non-detection mode as the operation mode. In the example shown in FIG. 11 , the text information 88, the measurement line 90, and the auxiliary treatment information 92 are superimposed on the B-mode image 24A generated by the generation unit 62A, so the non-detection mode is set as the operation mode. In this case, the control unit 62C displays the B-mode image 24A with the text information 88, the measurement line 90, and the auxiliary treatment information 92 superimposed on it on the screen 26 of the display device 14.

[0115] An example of the flow of diagnostic support processing according to this second modification is shown in Fig. 12. The flowchart shown in Fig. 12 differs from the flowchart shown in Fig. 9A in that it includes processing of step ST200 between the processing of step ST16 and the processing of step ST18, and in that it includes processing of steps ST202 to ST206 following step ST24.

[0116] 12 , in step ST200, the control unit 62C determines whether the character information 88, the measurement line 90, and / or the auxiliary treatment information 92 are superimposed on the B-mode image 24A generated in step ST16. If the character information 88, the measurement line 90, and / or the auxiliary treatment information 92 are not superimposed on the B-mode image 24A generated in step ST16 in step ST200, the determination is negative, and the diagnostic support process proceeds to step ST18. If the character information 88, the measurement line 90, and / or the auxiliary treatment information 92 are superimposed on the B-mode image 24A generated in step ST16 in step ST200, the determination is positive, and the diagnostic support process proceeds to step ST202.

[0117] In step ST202, the control unit 62C determines whether the currently set operation mode is the detection mode. If the currently set operation mode is not the detection mode in step ST202, the determination is negative, and the diagnostic support processing proceeds to step ST206. If the currently set operation mode is the detection mode in step ST202, the determination is positive, and the diagnostic support processing proceeds to step ST204.

[0118] In step ST204, the control unit 62C switches the operation mode from the detection mode to the non-detection mode. After the process of step ST204 is executed, the diagnostic support process proceeds to step ST206.

[0119] In step ST206, the control unit 62C displays the B-mode image 24A generated in step ST16 (i.e., the B-mode image 24A without the position identification information 27) on the screen 26 of the display device 14. In this case, since the detection frame 27A is not added to the B-mode image 24A, the doctor 20 can visually recognize that no lesion is shown in the B-mode image 24A. After the processing of step ST206 is executed, the diagnostic support processing proceeds to step ST26 shown in FIG. 9B.

[0120] 4 , the trained model 78 is created using, as training data, a B-mode image group 82 consisting of a plurality of B-mode images 82A on which none of the character information 88, the measurement lines 90, and the treatment auxiliary information 92 is superimposed. Therefore, when a process for detecting a lesion region 25 is performed according to the trained model 78 from a B-mode image 24A on which the character information 88, the measurement lines 90, and / or the treatment auxiliary information 92 is superimposed, there is a high possibility that an area other than the lesion region 25 will be erroneously detected as the lesion region 25. In other words, there is a high possibility that an area other than the lesion region 25 will be erroneously detected as the lesion region 25 compared to when a process for detecting a lesion region 25 is performed according to the trained model 78 from a B-mode image 24A on which none of the character information 88, the measurement lines 90, and the treatment auxiliary information 92 is superimposed. Furthermore, if a process is performed to detect lesion area 25 from B-mode image 24A on which text information 88, measurement line 90, and / or auxiliary treatment information 92 are superimposed according to trained model 78, and detection frame 27A is displayed in B-mode image 24A as a detection result, there is a risk that the presence of detection frame 27A will interfere with observation of text information 88, measurement line 90, and / or auxiliary treatment information 92 by physician 20. Interference with observation of text information 88, measurement line 90, and / or auxiliary treatment information 92 means that it will interfere with the diagnosis of physician 20.

[0121] Therefore, in the second modified example, the detection mode and the non-detection mode are switched depending on whether the text information 88, the measurement line 90, and / or the auxiliary treatment information 92 are superimposed on the B-mode image 24A. That is, when the text information 88, the measurement line 90, and / or the auxiliary treatment information 92 are superimposed on the B-mode image 24A, the non-detection mode is set as the operation mode, and when none of the text information 88, the measurement line 90, and the auxiliary treatment information 92 are superimposed on the B-mode image 24A, the detection mode is set as the operation mode. This prevents an area other than the lesion area 25 from being erroneously detected as the lesion area 25 and prevents visualization of the detection result of the lesion area 25 (i.e., the presence of the detection frame 27A) from interfering with diagnosis.

[0122] [Third Modification] In the first modification (i.e., the example shown in FIG. 10 ), an example was given in which the detection mode and the non-detection mode were switched depending on the presence or absence of color information 24B1, but the technology of the present disclosure is not limited to this. For example, the detection mode may be switched depending on the number of specific chromatic color pixels superimposed on the ultrasound image 24 generated by the generation unit 62A (e.g., the number of chromatic color pixels included in the color information 24B1).

[0123] In this case, as shown in FIG. 13 as an example, the control unit 62C counts the number of chromatic color pixels whose saturation is equal to or greater than a first threshold value (hereinafter referred to as the "number of chromatic color pixels") from the ultrasound image 24 generated by the generation unit 62A. The control unit 62C then determines whether the number of chromatic color pixels is equal to or greater than a second threshold value. If the number of chromatic color pixels is less than the second threshold value, the control unit 62C sets the operation mode to detection mode, and if the number of chromatic color pixels is equal to or greater than the second threshold value, the control unit 62C sets the operation mode to non-detection mode.

[0124] The first threshold is a value set in advance as the minimum saturation value that causes a false detection in the detection mode when chromatic color pixels are included in the ultrasound image 24. The second threshold is a value set in advance as the minimum number of chromatic color pixels that causes a false detection in the detection mode when chromatic color pixels are included in the ultrasound image 24. Note that the first threshold and / or the second threshold may be variable values ​​that are changed according to instructions received by the reception device 52 and / or various parameters set for the ultrasound endoscope 12.

[0125] As described above, in the third modified example, the detection mode is set when the number of chromatic color pixels is less than the second threshold, and the non-detection mode is set when the number of chromatic color pixels is equal to or greater than the second threshold. This prevents regions other than the lesion region 25 from being mistakenly detected as the lesion region 25 due to the presence of chromatic color pixels, and also prevents the presence of chromatic color pixels from affecting the visibility of the ultrasound endoscope 24 and thereby hindering diagnosis.

[0126] [Fourth Modification] In the above embodiment, an example in which the non-detection mode is set as the operating mode in the Doppler mode has been described, but the technology of the present disclosure is not limited to this. For example, the non-detection mode may be set as the operating mode in the first image mode or the second image mode. The first image mode is an image mode in which an ultrasound image 24 is generated using high-frequency components contained in reflected waves (hereinafter simply referred to as "reflected waves") obtained by emitting ultrasound to the observation region 72 and reflecting it from the observation region 72. The second image mode is an image mode in which a B-mode image 24A is combined with another image.

[0127] The first image mode may be, for example, a THI mode, a CH mode, or a CHI mode. The second image mode may be a Doppler mode or an elastography mode. These image modes are well known, and therefore will not be described here.

[0128] 14 , the generator 62A selectively sets B-mode, Doppler mode, elastography mode, THI mode, CH mode, and CHI mode in accordance with the image mode instruction received by the reception device 52. The generator 62A then generates an ultrasound image 24 according to the set image mode. For example, the generator 62A generates a Doppler image 24B in the Doppler mode, an elastography image 24C in the elastography mode, a THI image 24D in the THI mode, a CH image 24E in the CH mode, and a CHI image 24F in the CHI mode.

[0129] The control unit 62C displays the ultrasound image 24 generated according to the set image mode on the screen 26 of the display device 14. That is, in the Doppler mode, a Doppler image 24B is displayed on the screen 26, in the elastography mode, an elastography image 24C is displayed on the screen 26, in the THI mode, a THI image 24D is displayed on the screen 26, in the CH mode, a CH image 24E is displayed on the screen 26, and in the CHI mode, a CHI image 24F is displayed on the screen 26. Note that the elastography image 24C, the THI image 24D, the CH image 24E, and the CHI image 24F are well-known ultrasound images, and therefore will not be described here.

[0130] The control unit 62C sets the non-detection mode as the operating mode when the first image mode or the second image mode is set (i.e., when the Doppler mode, elastography mode, THI mode, CH mode, or CHI mode is set).

[0131] In this way, when the first image mode is set (here, as an example, when the THI mode, CH mode, or CHI mode is set), the non-detection mode is set as the operating mode, which makes it possible to prevent an area other than lesion area 25 from being erroneously detected as lesion area 25 due to the presence of an image area corresponding to the high-frequency components contained in the reflected waves. Furthermore, when observing ultrasound image 24 (here, as an example, THI image 24D, CH image 24E, or CHI image 24F), it is possible to prevent the presence of an image area corresponding to the high-frequency components contained in the reflected waves from affecting the visibility of ultrasound image 24 and thereby hindering diagnosis.

[0132] Furthermore, when the second image mode is set (here, as an example, when the Doppler mode or the elastography mode is set), the non-detection mode is set as the operating mode, so that it is possible to prevent an area other than the lesion area 25 from being erroneously detected as the lesion area 25 due to the presence of another image (e.g., a chromatic image) to be combined with the B-mode image. Furthermore, when observing the ultrasound image 24 (here, as an example, the Doppler image 24B or the elastography image 24C), it is possible to prevent the presence of another image (e.g., a chromatic image) to be combined with the B-mode image from affecting the visibility of the ultrasound image 24 and thereby hindering diagnosis.

[0133] [Fifth Modification] In the above embodiment, an example is given in which the detection mode and the non-detection mode are switched depending on the image mode, but the technology of the present disclosure is not limited to this, and the detection mode and the non-detection mode may be switched depending on a setting value that determines the image quality of the ultrasound image 24.

[0134] 15 , for example, a plurality of parameters 94 are stored in the NVM 66 as setting values ​​that determine the image quality of the ultrasound image 24, and the control unit 62C acquires the plurality of parameters 94 from the NVM 66 and switches between the detection mode and the non-detection mode in accordance with the acquired plurality of parameters 94. The NVM 66 stores a frequency parameter 94A, a depth parameter 94B, a brightness parameter 94C, a dynamic range parameter 94D, and a magnification parameter 94E as the plurality of parameters 94.

[0135] Here, the multiple parameters 94 are an example of a "setting value" according to the technology of the present disclosure. The frequency parameter 94A is an example of a "frequency parameter" according to the technology of the present disclosure. The depth parameter 94B is an example of a "depth parameter" according to the technology of the present disclosure. The brightness parameter 94C is an example of a "brightness parameter" according to the technology of the present disclosure. The dynamic range parameter 94D is an example of a "dynamic range parameter" according to the technology of the present disclosure. The magnification parameter 94E is an example of a "magnification parameter" according to the technology of the present disclosure.

[0136] The frequency parameter 94A is a parameter that adjusts the frequency of the ultrasound emitted from the ultrasound probe 38 (see FIG. 2). The depth parameter 94B is a parameter that adjusts the depth represented by the ultrasound image 24. The brightness parameter 94C is a parameter that adjusts the brightness (i.e., gain) of the ultrasound image 24. The dynamic range parameter 94D is a parameter that adjusts the dynamic range of the ultrasound image 24. The magnification parameter 94E is a parameter that adjusts the magnification of the digital zoom for the ultrasound image 24.

[0137] The control unit 62C sets the operation mode to detection mode when the following conditions are satisfied: the frequency parameter 94A is within a first range, the depth parameter 94B is within a second range, the brightness parameter 94C is within a third range, the dynamic range parameter 94D is within a fourth range, and the magnification parameter 94E is within a fifth range (hereinafter referred to as "parameter conditions"). Furthermore, the control unit 62C sets the operation mode to non-detection mode when the parameter conditions are not satisfied. The first to fifth ranges are ranges designated in advance. The first to fifth ranges are examples of "designated ranges" according to the technology of the present disclosure.

[0138] The first range is an ideal range of the frequency of the ultrasound emitted from the ultrasound probe 38. An example of the ideal range of the frequency of the ultrasound is a frequency range in which an area other than the lesion area 25 is not erroneously detected as the lesion area 25 (for example, a frequency range applied to all B-mode images 82A included in the B-mode image group 82).

[0139] The second range is an ideal range of depths represented in the ultrasound image 24. An example of the ideal range of depths is a range of depths in which areas other than the lesion area 25 are not erroneously detected as the lesion area 25 (for example, a range of depths applied to all B-mode images 82A included in the B-mode image group 82).

[0140] The third range is an ideal range of brightness for the ultrasound image 24. An example of the ideal range of brightness is a range of brightness in which an area other than the lesion area 25 is not erroneously detected as the lesion area 25 (for example, a range of brightness applied to all B-mode images 82A included in the B-mode image group 82).

[0141] The fourth range is an ideal range of the dynamic range of the ultrasound image 24. An example of the ideal range of the dynamic range is a range of the dynamic range in which an area other than the lesion area 25 is not erroneously detected as the lesion area 25 (for example, a range of the dynamic range applied to all B-mode images 82A included in the B-mode image group 82).

[0142] The fifth range is an ideal range of digital zoom magnification for the ultrasound image 24. An example of the ideal range of digital zoom magnification is a range of digital zoom magnification in which an area other than the lesion area 25 is not erroneously detected as the lesion area 25 (for example, a range of digital zoom magnification applied to all B-mode images 82A included in the B-mode image group 82).

[0143] According to the fifth modification, the detection mode and the non-detection mode are switched in accordance with the plurality of parameters 94. That is, the detection mode is set when the parameter conditions are satisfied, and the non-detection mode is set when the parameter conditions are not satisfied. This prevents a region other than the lesion region 25 from being erroneously detected as the lesion region 25 due to the plurality of parameters 94 not being within ideal ranges, and also prevents the plurality of parameters 94 not being within ideal ranges from affecting the visibility of the ultrasound image 24, thereby hindering diagnosis.

[0144] Furthermore, according to the fifth modification, the detection mode is set when frequency parameter 94A is within the first range, and the non-detection mode is set when frequency parameter 94A is not within the first range. This prevents a region other than lesion region 25 from being erroneously detected as lesion region 25 due to frequency parameter 94A not being within the first range, and also prevents frequency parameter 94A not being within the first range from affecting the visibility of ultrasound image 24, thereby hindering diagnosis.

[0145] Furthermore, according to the fifth modification, the detection mode is set when depth parameter 94B is within the second range, and the non-detection mode is set when depth parameter 94B is not within the second range. This prevents a region other than lesion region 25 from being erroneously detected as lesion region 25 due to depth parameter 94B not being within the second range, and also prevents depth parameter 94B not being within the second range from affecting the visibility of ultrasound image 24, thereby hindering diagnosis.

[0146] Furthermore, according to the fifth modification, the detection mode is set when brightness parameter 94C is within the third range, and the non-detection mode is set when brightness parameter 94C is not within the third range. This prevents an area other than lesion area 25 from being erroneously detected as lesion area 25 due to brightness parameter 94C not being within the third range, and also prevents brightness parameter 94C not being within the third range from affecting the visibility of ultrasound image 24, thereby hindering diagnosis.

[0147] Furthermore, according to the fifth modification, the detection mode is set when dynamic range parameter 94D is within the fourth range, and the non-detection mode is set when dynamic range parameter 94D is not within the fourth range. This prevents a region other than lesion region 25 from being erroneously detected as lesion region 25 due to dynamic range parameter 94D not being within the fourth range, and also prevents dynamic range parameter 94D not being within the fourth range from affecting the visibility of ultrasound image 24 and hindering diagnosis.

[0148] Furthermore, according to the fifth modification, the detection mode is set when the magnification parameter 94E is within the fifth range, and the non-detection mode is set when the magnification parameter 94E is not within the fifth range. This prevents an area other than the lesion area 25 from being erroneously detected as the lesion area 25 due to the magnification parameter 94E not being within the fifth range, and also prevents the magnification parameter 94E not being within the fifth range from affecting the visibility of the ultrasound image 24 and impeding diagnosis.

[0149] Furthermore, according to the fifth modified example, a plurality of parameters 94 are stored in the NVM 66, and the control unit 62C switches between the detection mode and the non-detection mode in accordance with the plurality of parameters 94 in the NVM 66. Therefore, even if the plurality of parameters 94 required to determine whether to switch between the detection mode and the non-detection mode is not manually input each time an ultrasound image 24 is generated, the operation mode can be set in accordance with the plurality of parameters 94.

[0150] Although multiple parameters 94 are exemplified here, the technology of the present disclosure is not limited to this, and the detection mode and non-detection mode may be switched depending on whether at least one of the multiple parameters 94 is within a pre-specified range (i.e., the ideal range described above).

[0151] [Sixth Modification] In the above-described fifth modification, an example is given in which the control unit 62C acquires the multiple parameters 94 stored in the NVM 66, and the control unit 62C switches between the detection mode and the non-detection mode in accordance with the multiple parameters 94. However, the technology of the present disclosure is applicable even in cases in which the multiple parameters 94 are not stored in the NVM 66.

[0152] In this case, for example, as shown in Fig. 16 , the generation unit 62A generates a frame 96 in B mode. The frame 96 includes a B-mode image 24A, and a text image 98 is superimposed on the frame 96. The text image 98 is an image in which multiple parameters 94 are displayed as text. The control unit 62C displays the frame 96 on the screen 26 of the display device 14. As a result, the B-mode image 24A and the text image 98 are displayed on the screen 26, allowing the doctor 20 to observe the B-mode image 24A and understand the multiple parameters 94 indicated by the text image 98.

[0153] 17 as an example, the control unit 62C identifies an area 100 in which a text image 98 exists from a frame 96, and extracts the identified area 100 from the frame 96. The control unit 62C then performs image recognition processing (e.g., AI-based image recognition processing or non-AI-based image recognition processing using template matching or the like) on the area 100 extracted from the frame 96, thereby recognizing and acquiring multiple parameters 94 from the text image 98 included in the area 100. The control unit 62C then switches between detection mode and non-detection mode according to the multiple parameters 94 in the same manner as in the fifth modified example. This provides the same effect as the fifth modified example.

[0154] [Seventh Modification] In the above embodiment, regardless of the image mode, the frequency with which lesion area 25 is detected from ultrasound image 24 (hereinafter referred to as "detection frequency"), the accuracy with which lesion area 25 is detected from ultrasound image 24 (hereinafter referred to as "detection accuracy"), and the target to be detected as lesion area 25 from ultrasound image 24 (hereinafter referred to as "detection target") are not changed. However, processor 62 may vary the detection frequency, detection accuracy, and / or detection target depending on the image mode.

[0155] When the detection frequency is varied depending on the image mode, for example, the detection frequency is varied between when B mode is used as the primary image mode and when an image mode other than B mode (e.g., elastography mode, etc.) is used as the primary image mode. In this case, the detection frequency in B mode is made higher than the detection frequency in image modes other than B mode (hereinafter referred to as "other image modes"). For example, the detection frequency in B mode is set to every frame, and the detection frequency in other image modes is set to every several frames (e.g., every two frames).

[0156] When the detection accuracy is varied depending on the image mode, for example, the detection accuracy is varied between when B mode is used as the primary image mode and when other image modes are used as the primary image mode. A first example of a method for varying the detection accuracy is to vary the amount of learning performed on the model 80 to obtain the trained model 78 used in B mode from the amount of learning performed on the model to obtain the trained model used in the other image mode. A second example of a method for varying the detection accuracy is to vary the number of intermediate layers of the trained model 78 used in B mode from the number of intermediate layers of the trained model used in the other image mode.

[0157] When the detection target is made different depending on the image mode, for example, the detection target is made different when B mode is used as the primary image mode and when another image mode is used as the primary image mode. One method for making the detection target different is to generate trained model 78 used in B mode by training model 80 to detect a first type of lesion, and generate trained model 78 used in another image mode by training model 80 to detect a second type of lesion (i.e., a type of lesion different from the first type).

[0158] An example of the flow of diagnostic support processing according to the seventh modification is shown in Fig. 18. The flowchart shown in Fig. 18 differs from the flowchart shown in Fig. 9A in that it includes processing of steps ST300 to ST304 between the processing of step ST16 and the processing of step ST18.

[0159] 18 , in step ST300, the control unit 62C determines whether or not the detection frequency, detection accuracy, and detection target for B-mode have been set. If the detection frequency, detection accuracy, and detection target for B-mode have been set in step ST300, the determination is affirmative, and the diagnostic support process proceeds to step ST304. If the detection frequency, detection accuracy, and detection target for B-mode have not been set in step ST300, the determination is negative, and the diagnostic support process proceeds to step ST302.

[0160] In step ST302, the control unit 62C sets the detection frequency, detection accuracy, and detection target for B-mode. After the process of step ST302 is executed, the diagnostic support process proceeds to step ST304.

[0161] In step ST304, control unit 62C determines whether the timing for detecting lesion area 25 has arrived. The timing for detecting lesion area 25 is, for example, a timing separated by a time interval defined by the reciprocal of the frame rate. If the timing for detecting lesion area 25 has not arrived in step ST304, the determination is negative, and the diagnostic support processing proceeds to step ST24. If the timing for detecting lesion area 25 has arrived in step ST304, the determination is positive, and the diagnostic support processing proceeds to step ST18.

[0162] As described above, according to the seventh modification, the detection frequency is varied depending on the image mode, thereby preventing the number of times that lesion area 25 is detected from becoming too many or too few depending on the image mode. Furthermore, according to the seventh modification, the detection accuracy is varied depending on the image mode, thereby preventing a decrease in the accuracy of detecting lesion area 25 depending on the image mode. Furthermore, according to the seventh modification, the detection target is varied depending on the image mode, thereby preventing detection of a portion unintended as lesion area 25.

[0163] In the seventh modification, an example in which the detection frequency, detection accuracy, and / or detection target are varied depending on the image mode has been described, but the technology of the present disclosure is not limited to this. For example, the detection frequency, detection accuracy, and / or detection target may be varied depending on whether reference information (e.g., text information 88, measurement lines 90, and / or treatment auxiliary information 92) is superimposed on the B-mode image 24A, or on setting values ​​(e.g., multiple parameters 94) that determine the image quality of the ultrasound image 24.

[0164] [Other Modifications] In the above embodiment, an example in which the detection mode is set as the operating mode in the case of B-mode has been described, but the technology of the present disclosure is not limited to this. For example, the detection mode may be set as the operating mode in the case of A-mode or M-mode instead of B-mode, or the detection mode may be set as the operating mode in the case of an image mode that is predetermined as a single primary image mode. The primary image mode may be an image mode that corresponds to the type of ultrasound image 24 that the model 80 is trained on. For example, if the ultrasound image 24 that the model 80 is trained on is an A-mode image, the primary image mode is A-mode, and if the ultrasound image 24 that the model 80 is trained on is an M-mode image, the primary image mode is M-mode.

[0165] In the above embodiment, an example of a form in which the lesion area 25 is detected has been described, but the technology of the present disclosure is not limited to this, and a specific area other than the lesion area 25 (for example, a specific organ) may be detected together with the lesion area 25 or instead of the lesion area 25.

[0166] In the above embodiment, an example of detecting lesion area 25 using an AI method (i.e., an example of detecting lesion area 25 according to trained model 78) has been described, but the technology of the present disclosure is not limited to this, and lesion area 25 may be detected using a non-AI method. An example of a non-AI detection method is a detection method using template matching. In this case, the template used in template matching is an example of "detection assistance information" according to the technology of the present disclosure.

[0167] In the above embodiment, the ultrasonic endoscope 12 is exemplified, but the technology of the present disclosure can also be applied to an extracorporeal ultrasonic diagnostic device.

[0168] In the above embodiment, an example was given in which the ultrasound image 24 and detection frame 27A generated by the processing device 18 are displayed on the screen 26 of the display device 14. However, the ultrasound image 24 with the detection frame 27A may be transmitted to various devices such as a server, a PC, and / or a tablet terminal and stored in the memory of the various devices. Furthermore, the ultrasound image 24 with the detection frame 27A may be recorded in a report. Furthermore, the position identification information 27 may also be stored in the memory of various devices or recorded in a report. It is preferable that the ultrasound image 24, the detection frame 27A, and / or the position identification information 27 be stored in the memory or recorded in a report for each subject 22.

[0169] In the above embodiment, an example in which the diagnostic support processing is performed by the processing device 18 has been described, but the technology of the present disclosure is not limited to this. The diagnostic support processing may be performed by the processing device 18 and at least one device provided external to the processing device 18, or may be performed only by at least one device provided external to the processing device 18 (for example, an auxiliary processing device connected to the processing device 18 and used to expand the functions of the processing device 18).

[0170] An example of the at least one device provided outside the processing device 18 is a server. The server may be realized by cloud computing. Cloud computing is merely an example, and network computing such as fog computing, edge computing, or grid computing may also be used. Furthermore, the server given as the at least one device provided outside the processing device 18 is merely an example, and instead of a server, at least one PC and / or at least one mainframe may be used, or at least one server, at least one PC, and / or at least one mainframe may be used.

[0171] In the above embodiment, an example has been given in which the ultrasound image 24 with the detection frame 27A superimposed thereon is displayed on the screen 26 of the display device 14, but this is merely one example. For example, the ultrasound image 24 with the detection frame 27A superimposed thereon and the ultrasound image 24 without the detection frame 27A superimposed thereon (i.e., the ultrasound image 24 in which the results of detecting the lesion area 25 are not visualized) may be displayed on separate screens.

[0172] In the above embodiment, the doctor 20 is made aware of the presence or absence of a lesion and the location of the lesion, but the doctor 20 may also be made aware of the type of lesion and / or the progression of the lesion, etc. In this case, the ultrasound image (B-mode image 82A in the example shown in FIG. 4) may be used as training data for learning the model 80, with the annotation 86 including information that can identify the type of lesion and / or the progression of the lesion, etc.

[0173] In the above embodiment, an example in which the diagnostic assistance program 76 is stored in the NVM 66 has been described, but the technology of the present disclosure is not limited to this. For example, the diagnostic assistance program 76 may be stored in a portable storage medium such as an SSD or a USB memory. The storage medium is a non-transitory computer-readable storage medium. The diagnostic assistance program 76 stored in the storage medium is installed in the computer 54. The processor 62 executes diagnostic assistance processing in accordance with the diagnostic assistance program 76.

[0174] In the above embodiment, the computer 54 is exemplified, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 54. Furthermore, instead of the computer 54, a combination of a hardware configuration and a software configuration may be used.

[0175] The hardware resources for executing the diagnostic support processing described in the above embodiments can be various processors, as listed below. Examples of processors include general-purpose processors that function as hardware resources for executing diagnostic support processing by executing software, i.e., programs. Examples of processors include dedicated electronic circuits, such as FPGAs, PLDs, or ASICs, which are processors with circuit configurations designed specifically for executing specific processing. Each processor has built-in or connected memory, and uses the memory to execute the diagnostic support processing.

[0176] The hardware resource that executes the diagnostic support processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a processor and an FPGA). Also, the hardware resource that executes the diagnostic support processing may be a single processor.

[0177] As an example of a system configured with a single processor, first, one processor is configured by combining one or more processors with software, and this processor functions as a hardware resource that executes the diagnostic support processing. Second, there is a system in which a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute the diagnostic support processing, on a single IC chip, as typified by SoC. In this way, the diagnostic support processing is realized using one or more of the various processors described above as hardware resources.

[0178] Furthermore, the hardware structure of these various processors can be, more specifically, electronic circuits that combine circuit elements such as semiconductor devices. The above diagnostic support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0179] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0180] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0181] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processor is provided. The processor, Obtaining a first ultrasound image generated by an ultrasound module and including the region of interest; switching between a first operation mode and a second operation mode according to whether reference information referred to for diagnosing the observation target region is synthesized with the first ultrasonic image, whether the first ultrasonic image is an image obtained in an auxiliary image mode that is an image mode other than a main image mode, or a setting value that determines the image quality of the first ultrasonic image; the first operation mode is an operation mode for detecting a specific region from the first ultrasound image based on detection support information created using a second ultrasound image acquired in the primary image mode; The second operation mode is an operation mode in which the specific area is detected and a detection result is not output, or the specific area is not detected. Diagnostic aids.

2. the first operation mode is an operation mode used when the reference information is not synthesized with the first ultrasound image; The second operation mode is an operation mode used when the reference information is combined with the first ultrasound image. The diagnosis support device according to claim 1 .

3. the first operation mode is an operation mode used when the first ultrasound image is not an image obtained in the auxiliary image mode; The second operation mode is an operation mode used when the first ultrasonic image is an image obtained in the auxiliary image mode. The diagnosis support device according to claim 1 .

4. the first operation mode is an operation mode that is used when the setting value is within a specified range, The second operation mode is an operation mode that is used when the set value is not within the specified range. The diagnosis support device according to claim 1 .

5. The reference information includes color information that expresses characteristics within the observation target area by color. The diagnosis support device according to any one of claims 1 to 4.

6. the color information includes a plurality of chromatic pixels; the first operation mode is an operation mode used when a number of chromatic color pixels having a saturation exceeding a first threshold among the plurality of chromatic color pixels is less than a second threshold; The second operation mode is an operation mode used when the number is equal to or greater than the second threshold value. The diagnosis support device according to claim 5.

7. The reference information includes text information that assists in observing the observation target area. The diagnosis support device according to claim 1 .

8. The reference information includes a measurement line used for measuring the observation target area. The diagnosis support device according to claim 1 .

9. The reference information includes treatment assistance information that assists a treatment using a fine needle aspiration method. The diagnosis support device according to claim 1 .

10. The auxiliary image mode is a first image mode in which an ultrasound image is generated by utilizing high-frequency components contained in a reflected wave obtained by emitting ultrasound to the observation region and reflecting it from the observation region, or a second image mode in which a B-mode image is synthesized with another image. The diagnosis support device according to claim 1 .

11. the first image mode is a THI mode, a CH mode, or a CHI mode; The second imaging mode is a Doppler mode or an elastography mode. The diagnosis support device according to claim 10.

12. The set values ​​include a frequency parameter for adjusting the frequency of the ultrasound emitted from the ultrasound module, a depth parameter for adjusting the depth represented by the first ultrasound image, a brightness parameter for adjusting the brightness of the first ultrasound image, a dynamic range parameter for adjusting the dynamic range of the first ultrasound image, and / or a magnification parameter for adjusting the magnification of a digital zoom for the first ultrasound image. The diagnosis support device according to claim 1 .

13. The ultrasonic module has the set value, The processor obtains the setting value from the ultrasound module. The diagnosis support device according to claim 1 .

14. a text image capable of identifying the setting value is synthesized in a frame including the first ultrasound image; The processor, Identifying the setting value by performing an image recognition process on the text image; The first operation mode and the second operation mode are switched in accordance with the specified setting value. The diagnosis support device according to claim 1 .

15. The processor detects a specific region from the first ultrasound image using an AI method. The diagnosis support device according to claim 1 .

16. The detection assistance information is a trained model obtained by training a model on training data including the second ultrasound image. The diagnosis support device according to claim 1 .

17. The processor varies a frequency of detecting the specific region from the first ultrasonic image, an accuracy of detecting the specific region from the first ultrasonic image, and / or an object to be detected as the specific region from the first ultrasonic image, depending on the reference information, the auxiliary image mode, and / or the setting value. The diagnosis support device according to claim 1 .

18. The ultrasonic module is an ultrasonic endoscope. The diagnosis support device according to claim 1 .

19. The diagnosis support device according to claim 1 ; and an ultrasonic endoscope body to which the ultrasonic module is connected. Ultrasound endoscope.

20. acquiring a first ultrasound image generated by an ultrasound module, the first ultrasound image including the region of interest; and switching between a first operation mode and a second operation mode according to whether reference information referred to for diagnosing the observation region is synthesized in the first ultrasonic image, whether the first ultrasonic image is an image obtained in an auxiliary image mode that is an image mode other than a main image mode, or a setting value that determines the image quality of the first ultrasonic image; the first operation mode is an operation mode for detecting a specific region from the first ultrasound image based on detection support information created using a second ultrasound image acquired in the primary image mode; The second operation mode is an operation mode in which the specific area is detected and a detection result is not output, or the specific area is not detected. Diagnostic aid methods.

21. A program for causing a computer to execute a process, The process comprises: acquiring a first ultrasound image generated by an ultrasound module, the first ultrasound image including the region of interest; and switching between a first operation mode and a second operation mode according to whether reference information referred to for diagnosing the observation region is synthesized in the first ultrasonic image, whether the first ultrasonic image is an image obtained in an auxiliary image mode that is an image mode other than a main image mode, or a setting value that determines the image quality of the first ultrasonic image; the first operation mode is an operation mode for detecting a specific region from the first ultrasound image based on detection support information created using a second ultrasound image acquired in the primary image mode; The second operation mode is an operation mode in which the specific area is detected and a detection result is not output, or the specific area is not detected. program.