Lesion detection device, lesion detection method, and ultrasound diagnostic device

Abstract

To improve the accuracy of lesion detection when detecting multiple types of lesions. [Solution] The lesion detection device according to the embodiment includes a lesion detection unit that detects a lesion in each of a plurality of captured frames based on a detection threshold for detecting a lesion corresponding to the type of lesion; a frame result acquisition unit that acquires the detection results of lesions in the plurality of frames; and a threshold update unit that determines the type of lesion based on at least one of the continuity and frequency of appearance of lesion types in the detection results of the plurality of frames, and updates the detection threshold according to the determined type of lesion, and the lesion detection unit detects a lesion based on the updated detection threshold.

Description

[Technical Field]

[0001] Embodiments disclosed herein and in the drawings relate to lesion detection devices, lesion detection methods, and ultrasound diagnostic devices. [Background technology]

[0002] Image-based lesion detection technology is a local examination method that detects organs or tissues being measured in images. Image-based lesion detection technology is widely applied in medical fields such as medical image analysis, auxiliary diagnosis, image management, and lesion detection and classification. Common applications of image-based lesion detection technology include colonoscopy, gastroscopy, endoscopic ultrasound, and in vitro ultrasound.

[0003] Conventional image-based lesion detection technologies typically display detection frames within the image to inform users (doctors and technicians) of the detection results. However, when using image-based lesion detection technologies as a supplementary diagnostic method, too many incorrect detection frames can negatively impact the user's observation and judgment.

[0004] Furthermore, conventional image lesion detection techniques typically filter out false detection frames by setting a detection threshold to control the frequency and number of false detection frames. In this case, only detection frames that exceed the set detection threshold in confidence are output as detection results. However, if multiple types of lesions are present in an image, the same detection threshold must be set for all types of lesions. Therefore, the ability to detect multiple types of lesions may be limited, and it may not be possible to reduce the number of false detection frames for a particular type of lesion. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Chinese Patent Application Publication No. 109754361 Specification [Overview of the project] [Problems that the invention aims to solve]

[0006] One of the problems that the embodiments disclosed herein and in the drawings aim to solve is to improve the accuracy of lesion detection when detecting multiple types of lesions. However, the problems that the embodiments disclosed herein and in the drawings aim to solve are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0007] The lesion detection device according to this embodiment includes a lesion detection unit that detects a lesion in each of a plurality of captured frames based on a detection threshold for detecting a lesion corresponding to the type of lesion; a frame result acquisition unit that acquires the detection results of lesions in the plurality of frames; and a threshold update unit that determines the type of lesion based on at least one of the continuity and frequency of appearance of lesion types in the detection results of the plurality of frames, and updates the detection threshold according to the determined type of lesion. The lesion detection unit then detects the lesion based on the updated detection threshold. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a block diagram showing an example configuration of a lesion detection device according to the first embodiment. [Figure 2] Figure 2 is a schematic diagram showing the detection results across multiple frames. [Figure 3] Figure 3 is a schematic diagram showing the grouping of detection results across multiple frames. [Figure 4] Figure 4 is a schematic diagram illustrating the updating of the detection threshold. [Figure 5A] Figure 5A is a schematic diagram showing an example of the display content of the display according to the first embodiment. [Figure 5B] Figure 5B is a schematic diagram showing an example of the display content of the display according to the first embodiment. [Figure 6] Figure 6 is a flowchart of the lesion detection method according to the first embodiment. [Figure 7A]Figure 7A shows the detection results obtained by the lesion detection device related to the comparative example. [Figure 7B] Figure 7B shows the detection results obtained by the lesion detection device according to the first embodiment. [Figure 8] Figure 8 is a block diagram showing an example configuration of a lesion detection device according to the second embodiment. [Figure 9] Figure 9 is a block diagram showing an example configuration of an ultrasound diagnostic apparatus according to the third embodiment. [Modes for carrying out the invention]

[0009] The ultrasound diagnostic apparatus according to this embodiment includes: a lesion detection unit that detects a lesion in each of a plurality of captured frames based on a detection threshold for detecting a lesion corresponding to a lesion type; a frame result acquisition unit that acquires the detection results of the lesion in the plurality of frames; and a threshold update unit that determines the lesion type based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of frames, and updates the detection threshold according to the determined lesion type, wherein the lesion detection unit detects the lesion based on the updated detection threshold.

[0010] The lesion detection device according to this embodiment comprises: a lesion detection unit that detects a lesion in each of a plurality of captured frames based on a detection threshold for detecting a lesion corresponding to a lesion type; a frame result acquisition unit that acquires the detection results of the lesion in the plurality of frames; and a threshold update unit that determines the lesion type based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of frames, and updates the detection threshold according to the determined lesion type, wherein the lesion detection unit detects the lesion based on the updated detection threshold.

[0011] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment determine the lesion type based on the continuity and / or frequency of occurrence of lesion types in the detection results of multiple frames, and update the detection threshold according to the lesion type. Therefore, the detection threshold can be updated according to the lesion type in multiple frames, and without increasing the total number of detection frames, the detection threshold for lesion types in multiple frames and the detection threshold for lesion types other than lesion types in multiple frames can be set separately, thereby improving the accuracy of lesion detection. Here, "lesion type in the current frame" refers to, for example, a lesion type whose probability of presence in the current frame is above a predetermined threshold. Also, "lesion type other than lesion type in the current frame" refers to, for example, a lesion type whose probability of presence in the current frame is below a predetermined threshold.

[0012] Furthermore, compared to setting the same or fixed detection threshold, it is possible to distinguish between lesion types in multiple frames and lesion types other than lesion types in multiple frames and assign different detection thresholds. Therefore, the possibility of detecting lesions that do not exist in multiple frames is reduced, the number of erroneous detection frames is reduced, and the accuracy of lesion detection can be further improved.

[0013] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the frame result acquisition unit acquires the detection results of lesions in a plurality of previous frames prior to the current frame, the threshold update unit determines the lesion type in the current frame based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of previous frames, updates the detection threshold according to the determined lesion type in the current frame, and the lesion detection unit detects the lesion in the current frame based on the updated detection threshold.

[0014] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment determine the lesion type in the current frame based on the continuity and / or frequency of occurrence of lesion types in the detection results of the previous frame, update the detection threshold according to the lesion type, and detect the lesion in the current frame based on the updated detection threshold. Therefore, it is possible to update the detection threshold according to the lesion type present in the previous frame and detect the lesion in the current frame based on the updated detection threshold, thereby distinguishing and setting detection thresholds for lesion types present in the current frame and those not present in the current frame without increasing the number of erroneous detection frames, and further improving the accuracy of lesion detection. In addition, it is possible to reduce the possibility of detecting lesions that are not present in the current frame, reduce the number of erroneous detection frames, and further improve the accuracy of lesion detection.

[0015] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the threshold update unit is further characterized in that it determines the lesion type in the current frame based on the reliability of the lesion type in the detection result of the previous frame.

[0016] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment determine the lesion type in the current frame based on the reliability of the lesion type in the detection result of the previous frame. Therefore, the accuracy of determining the lesion type in the current frame can be improved, and the accuracy of lesion detection can be further improved by updating the detection threshold according to the lesion type in the current frame.

[0017] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the threshold update unit updates the detection threshold to lower it when at least one of the continuity and frequency of occurrence and the reliability satisfy a preset condition.

[0018] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the threshold update unit is characterized by setting a lower limit for the updated detection threshold.

[0019] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment set a lower limit by lowering the detection threshold for lesion types present in the current frame. Therefore, it is possible to increase the probability of detecting lesions present in the current frame while avoiding a decrease in detection accuracy of lesions present in the current frame due to an excessively low threshold, thereby further improving the accuracy of lesion detection.

[0020] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the threshold update unit updates the detection threshold to increase it when at least one of the continuity and frequency of occurrence and the reliability satisfy a preset condition.

[0021] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the threshold update unit is characterized by setting an upper limit on the updated detection threshold.

[0022] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment set an upper limit by raising the detection threshold for lesion types that are not present in the current frame. This reduces the possibility of detecting lesions that are not present in the current frame, while avoiding the failure to detect any lesions in the current frame due to an excessively high threshold, thereby further improving the accuracy of lesion detection.

[0023] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the lesion detection unit is characterized in that, when a lesion is detected based on an updated detection threshold, it displays a detection frame indicating the detected lesion on the display.

[0024] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the lesion detection unit is characterized in that it displays the detection frame on the display such that the color of the detection frame changes according to the type of lesion detected.

[0025] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the lesion detection unit is characterized by displaying the detection frame on the display along with the reliability of the detected lesion type.

[0026] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the lesion detection unit is characterized in that, when the detection threshold is updated, it displays information related to the update on the display.

[0027] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment display information on the display, such as the detection frame of the detected lesion, the reliability of the lesion type, and information regarding the update of the detection threshold. Therefore, the user can easily check the update status of the detection threshold and the detection status of lesions.

[0028] In the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment, the threshold update unit updates the detection threshold in the current frame using the detection threshold output from the trained model, which is trained by the learning unit to output the detection threshold in the current frame based on the detection result in the previous frame.

[0029] As a result, the ultrasound diagnostic apparatus and lesion detection apparatus according to this embodiment update the detection threshold for the current frame by inputting the detection result from the previous frame to the trained model. Therefore, the detection threshold can be appropriately updated, and the accuracy of lesion detection can be further improved.

[0030] The lesion detection method according to this embodiment includes: a lesion detection step in which a lesion is detected in each of a plurality of captured frames based on a detection threshold for detecting a lesion corresponding to a lesion type; a frame result acquisition step in which the detection results of the lesion in the plurality of frames are acquired; and a threshold update step in which the lesion type is determined based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of frames, and the detection threshold is updated according to the determined lesion type, wherein the lesion detection step is characterized in that the lesion is detected based on the updated detection threshold.

[0031] Hereinafter, embodiments of the lesion detection device, lesion detection method, and ultrasound diagnostic device according to the embodiment will be described with reference to the drawings.

[0032] In the following description, the lesion detection device and lesion detection method according to the embodiment will be used to detect lesions such as hepatic cysts (Cyst), hepatic hemangiomas (Hema), metastatic liver cancer (Meta), and hepatocellular carcinoma (HCC) in the liver of a subject in images (videos) captured by ultrasound examination performed by an ultrasound diagnostic device. However, it goes without saying that the lesion detection device and lesion detection method according to the embodiment may also be used to detect lesions in images captured by other medical imaging equipment (other medical imaging diagnostic devices), or to detect other lesions in organs or tissues other than the liver of a subject.

[0033] (First Embodiment) Referring to Figure 1, an example of the configuration of the lesion detection device 1 according to the first embodiment will be described. Figure 1 is a block diagram showing an example of the configuration of the lesion detection device 1 according to the first embodiment.

[0034] As shown in Figure 1, the lesion detection device 1 comprises a processing circuit 11, a memory 12, a communication interface 13, and a display 14. Each component is connected to the others via a common signal transmission path, which is a bus.

[0035] Memory 12 is a device for storing various data and information. Memory 12 may be a storage medium readable by the processor (for example, a magnetic storage medium, an electromagnetic storage medium, an optical storage medium, or a semiconductor memory), or it may be a drive device that reads and writes data and information to and from the storage medium. Memory 12 stores the computer programs that enable the processing circuit 11 to implement each functional unit.

[0036] Communication IF13 is an interface for communicating various types of data and information between each component.

[0037] The processing circuit 11 is a circuit that controls the overall operation of the lesion detection device 1. The processing circuit 11 includes at least one processor. A processor refers to a circuit such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), or Programmable Logic Device (e.g., Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), Field Programmable Gate Array (FPGA)). If the processor is a CPU, the CPU realizes each function by reading and executing each program stored in memory 12. If the processor is an ASIC, each function is directly incorporated as a logic circuit within the ASIC's circuitry. The processor may be configured as a single circuit or as a combination of multiple independent circuits. The processing circuit 11 realizes the functions described later.

[0038] In this embodiment, the processing circuit 11 includes a lesion detection function 111, a frame result acquisition function 112, and a threshold update function 113. The following describes each functional part of the processing circuit 11.

[0039] The lesion detection function 111 is a function for detecting lesions in images. The lesion detection function 111 detects lesions in each of the multiple captured frames based on a detection threshold for detecting lesions that corresponds to the type of lesion.

[0040] In the following explanation, "lesion type" includes, for example, the type of lesion (type of lesion) such as liver cyst, hepatic hemangioma, metastatic liver cancer, and hepatocellular carcinoma. "Detection threshold" is, for example, a preset threshold corresponding to the lesion type, and is used to detect lesions of the corresponding type in the image. For example, the "detection threshold" is the "confidence level" of the corresponding type of lesion detected in the image. The display 14 of the lesion detection device 1, described later, displays detection frames where the confidence level exceeds the detection threshold, and does not display detection frames where the confidence level is below the detection threshold. For example, the lesion detection function 111 displays detection frames for lesion types where the confidence level exceeds the detection threshold on the display 14, and does not display detection frames for lesion types where the confidence level is below the detection threshold. In this way, the lesion detection function 111 detects lesion types where the confidence level exceeds the detection threshold and displays the detection frames for the detected lesion types on the display 14. Here, "confidence level" indicates the likelihood that the detection result detected in the image is correct; the higher the "confidence level," the higher the likelihood that the detection result is correct. Furthermore, the lesion detection function 111 can detect lesions in the image using, for example, various known detection algorithms.

[0041] Referring to Figure 2, the detection results in multiple frames will be explained, and referring to Figure 3, the determination of lesion type by the threshold update function 113 will be explained. Figure 2 is a schematic diagram showing the detection results in multiple frames according to the first embodiment. Figure 3 is a schematic diagram showing the grouping of detection results in multiple frames according to the first embodiment. Figures 2 and 3 schematically show the detection results in frames 29 to 38 in the image. In this specification, video includes one or more frames. Also, a frame refers to an image (image data). That is, one frame refers to one image.

[0042] Specifically, the lesion detection device 1 acquires video footage captured from a medical image generation device (medical image diagnostic device, modality) (not shown), and the acquired video footage contains multiple frames. The lesion detection function 111 detects lesions contained in each of the multiple frames from each of the multiple frames. The multiple frames include the current frame and previous frames. The current frame refers, for example, to the most recent frame at the present time. The previous frame refers to one or more frames prior to the current frame.

[0043] The frame result acquisition function 112 is a function for acquiring detection results in multiple frames. The frame result acquisition function 112 generates and acquires detection results for lesions in multiple frames detected by the lesion detection function 111. Here, "detection results" include information such as the detection frame number, frame number, lesion type, confidence level, and detection frame coordinates of the detection frames in multiple frames detected by the lesion detection function 111, as shown in Figure 2. The frame result acquisition function 112 generates such detection results. The detection frame number is, for example, a number for uniquely identifying the detection frame. The frame number is, for example, a number for uniquely identifying the frame. The lesion type is, for example, information indicating the type (kind) of the lesion. The detection frame coordinates are the coordinates of two diagonally opposite vertices (x1, y1) and (x2, y2) of the four vertices of the rectangular detection frame combined into one. It becomes possible to calculate the area of ​​the detection frame region from the two vertices. Furthermore, as mentioned above, confidence indicates the likelihood that the detection result in the image is correct. The higher the "confidence," the more likely the detection result is to be correct.

[0044] The frame result acquisition function 112 sends the detection result from the previous frame to the memory 12, which stores the detection result. The memory 12 stores structures such as queues and lists. If the memory 12 stores a queue, this queue is used to store the detection results from the previous and current frames. The length of the queue can be determined according to the frame rate of the image (video), and the higher the frame rate of the image, the longer the queue will be. For example, if the frame rate of the image is 22 frames / second, the queue length will be 10.

[0045] The frame result acquisition function 112 may update the queue of detection results by sending the detection result for the current frame to the memory 12 and adding the detection result for the current frame to the queue of detection results stored in the memory 12.

[0046] The lesion detection function 111 may perform the same processing as the frame result acquisition function 112. For example, the lesion detection function 111 may generate a detection result. In this case, the lesion detection function 111 or the frame result acquisition function 112 may send the generated detection result to the memory 12 to store the detection result in the memory 12.

[0047] The threshold update function 113 is a function that updates the detection threshold. The threshold update function 113 determines the lesion type based on at least one of the continuity and frequency of occurrence of lesion types in the detection results of multiple frames, and updates the detection threshold according to the determined lesion type. Specifically, the threshold update function 113 determines the lesion type in the current frame (the lesion type that is considered to be highly likely to be present in the current frame) based on at least one of the continuity and frequency of occurrence of lesion types in the detection results of multiple previous frames, and updates the detection threshold according to the determined lesion type.

[0048] First, the threshold update function 113 groups the detection results in the queue of detection results stored in memory 12. The threshold update function 113 calculates the degree of overlap IoU (Intersection over Union) between two detection frames with the same lesion type in adjacent frames. Adjacent frames refer to two adjacent frames when the frames are arranged in the order in which they were captured (imaging) (in chronological order). The degree of overlap IoU is calculated, for example, by the following formula (1).

number

[0049] Here, IoU indicates the degree of overlap between two detection frames with the same lesion type in adjacent frames, Intersection(Annotation, Detection) indicates the intersection region of the two detection frames (the area where the two detection frames (the two regions of the two detection frames) overlap), and Union(Annotation, Detection) indicates the union region of the two detection frames. Note that the union region of the two detection frames refers to, for example, the sum of the areas of the two regions of the two detection frames if the two detection frames do not overlap, and the area of ​​the region indicated by the two detection frames when the two detection frames overlap.

[0050] Furthermore, the threshold update function 113 sets the threshold T of the overlap degree IoU. IoU The threshold T is set in advance. IoU This may be set in advance. For example, a threshold T may be set in memory 12 in advance. IoU Information indicating this is stored, and the threshold update function 113 retrieves this information from memory 12, and the threshold T indicated by the retrieved information is stored. IoU You may use the following to perform the process described below. In the following description, the threshold T IoU Let T be 0.9, but the threshold T IoU This can be any other appropriate value.

[0051] Next, the threshold update function 113 groups the detection results such that the intersection over union (IoU) between two detection frames with the same lesion type in adjacent frames within the same group exceeds the threshold T IoU For example, when arranging in order (in chronological order) a plurality of frames in which the same lesion type is imaged, the threshold update function 113 determines that the IoU between two detection frames in two adjacent frames exceeds the threshold T IoU for a plurality of detection results, and groups these detection results as one group. Note that when only one detection result is obtained for a certain lesion type among a plurality of detection results, the threshold update function 113 sets this detection result as one group.

[0052] Taking the detection frames (detection frames 1, 3, 5, 8, 10, 12, 14, 15, 17, 19) in FIG. 2 where the lesion type is "Cyst (hepatic cyst)" as an example, when the threshold T IoU = 0.9, the IoU between detection frames 1 and 3 with the same lesion type in adjacent frames 1、3 , the IoU between detection frames 3 and 5 3、5 , the IoU between detection frames 5 and 8 5、8 , the IoU between detection frames 8 and 10 8、10 , the IoU between detection frames 10 and 12 10、12 , the IoU between detection frames 12 and 14 12、14 , the IoU between detection frames 14 and 15 14、15 , the IoU between detection frames 15 and 17 15、17 , and the IoU between detection frames 17 and 19 17、19 all exceed the threshold T IoU . Therefore, as shown in FIG. 3 for example, detection frames 1, 3, 5, 8, 10, 12, 14, 15, 17, 19 are grouped into group G1. Note that a group is represented by "G k ", and "k" is an identifier for identifying the group and is an integer of 1 or more.

[0053] Similarly, for the detection frames 2, 4, 6, 9, 11, 13, 16 of "Hema (hepatic hemangioma)" in adjacent frames, the IoU between each pair of two detection frames all exceeds the threshold T IoUSince it exceeds the limit, detection frames 2, 4, 6, 9, 11, 13, and 16 are divided into group G2.

[0054] Furthermore, the queue of detection results shown in Figure 2 contains only one frame (frame 31) with a detection frame (detection frame 7) for the lesion type "HCC (hepatocellular carcinoma)," so this detection frame 7 is separated into group G3.

[0055] Similarly, the detection result queue shown in Figure 2 contains only one frame (frame 37) with a lesion type of "Meta (metastatic liver cancer)" (detection frame 18), so this detection frame 18 is divided into group G4.

[0056] As a result, the threshold update function 113 groups the detection results in the queue of detection results stored in memory 12 into groups G1, G2, G3, and G4 based on the degree of overlap IoU between two detection frames with the same lesion type in adjacent frames.

[0057] Here, the overlap degree IoU reflects the "continuity" between two detection frames of a certain lesion type. In other words, the overlap degree IoU is information that indicates such "continuity". The overlap degree IoU between two detection frames of a certain lesion type in adjacent frames is the threshold T. IoU If the threshold is exceeded, the continuity between detection frames for that lesion type is considered high, and the threshold update function 113 groups them into one group. In the above explanation, the threshold update function 113 calculated the "degree of overlap IoU" between two detection frames as information indicating "continuity" and grouped the detection results, but it may also be possible to group them using various other information such as other parameters or indicators that can reflect the continuity between two detection frames for a certain lesion type in adjacent frames.

[0058] Next, the threshold update function 113 updates each group G k Number of detection frames N(G) k ) count the number of detection frames N(G k ) threshold T N The threshold T is set in advance. NThis may be set in advance. In this case, as shown in Figure 3, the number of detection frames in group G1 is N(G1)=9, the number of detection frames in group G2 is N(G2)=7, the number of detection frames in group G3 is N(G3)=1, and the number of detection frames in group G4 is N(G4)=1. Note that in the following explanation, the threshold T N Let's set it to 5, but the threshold T N This can be any other appropriate value.

[0059] The threshold update function 113 updates each group G k Number of detection frames N(G) k ) and threshold T N Compare this with the number of detection frames N(G k ) is the threshold T N Group G surpassed k Leave the number of detection frames N(G) k ) is the threshold T N Discard groups that do not exceed the threshold T. For example, in the example shown in Figure 3, the number of detection frames N(G1) in group G1 and the number of detection frames N(G2) in group G2 are equal to the threshold T. N Because the threshold is exceeded, the detection limits in groups G1 and G2 will remain, but the detection limits in groups G3 and G4 will be discarded.

[0060] Here, each group G k Number of detection frames N(G) k ) reflects the "frequency of occurrence" of detection frames for a certain lesion type. That is, the number of detection frames N(G k ) is information indicating the "frequency of occurrence" within the detection frame for a certain lesion type. Group G k Number of detection frames N(G) k ) is the threshold T N If it exceeds that, then Group G k Since the frequency of detection frames for the corresponding lesion type is considered to be high, group G k The detection frame remains.

[0061] Next, the threshold update function 113 determines the lesion type corresponding to the remaining group as the lesion type in the current frame. For example, the threshold update function 113 determines the lesion type Cyst corresponding to the remaining group G1 and the lesion type Hema corresponding to group G2 as the lesion types in the current frame. Here, "lesion type in the current frame" refers to, for example, lesion types whose probability of existence in the current frame is above a predetermined threshold. Also, "lesion types other than those in the current frame" refers to, for example, lesion types whose probability of existence in the current frame is below a predetermined threshold.

[0062] As a result, the lesion detection device 1 according to this embodiment can determine the lesion type in the current frame based on the continuity and frequency of appearance of lesion types in the detection results of the previous frame, thereby improving the accuracy of determining the lesion type in the current frame.

[0063] Furthermore, in the above explanation, the threshold update function 113 detects multiple groups G based on the degree of overlap IoU (continuity). k After dividing, the number of detection frames N(G) k Multiple groups G based on (frequency of occurrence) k An example was described in which a portion of the group is retained, and then the lesion type corresponding to the remaining group is determined as the lesion type in the current frame. However, the threshold update function 113 may determine the lesion type based on either the continuity or frequency of occurrence of lesion types in the detection results.

[0064] For example, the threshold update function 113 may divide the detection results into multiple groups based on the degree of overlap IoU (continuity), and then determine the lesion type corresponding to each of the groups as the lesion type in the current frame. Alternatively, the threshold update function 113 may determine the lesion type in the current frame based on the number of detection frames (frequency of occurrence) in the detection results without grouping. For example, the threshold update function 113 may compare the number of detection frames (frequency of occurrence) in the detection results with a predetermined threshold for each lesion type without grouping. Then, the threshold update function 113 determines that the lesion types in which the number of detection frames exceeds the predetermined threshold are the lesion types in the current frame.

[0065] Furthermore, the threshold update function 113 may determine the lesion type in the current frame based on the reliability of the lesion type in the detection result of the previous frame.

[0066] Specifically, the threshold update function 113 updates the confidence threshold T conf and the number of detection frames N conf (G k ) threshold T Nconf The threshold is set in advance. The threshold update function 113 sets the threshold T conf Set this for each type of lesion. Note that the threshold T conf and threshold T Nconf This may be set in advance. For example, threshold T conf This may be set for each lesion type. In the following explanation, the threshold T for lesion type Cyst is used. conf The threshold T for lesion type Hema is 0.708. conf Set to 0.615, threshold T Nconf Let's set it to 3.

[0067] Next, the threshold update function 113 updates the confidence level to threshold T. conf Number of detection frames exceeding N conf (G k ) are counted. At this time, as shown by the thick underline in Figure 3, the confidence level in group G1 is the threshold T. conf Number of detection frames exceeding N conf(G1)=5 (detection frames 1, 3, 14, 15, 17), and the confidence level in group G2 is the threshold T. conf Number of detection frames exceeding N conf (G2) = 4 (detection frames 2, 4, 9, 11).

[0068] Next, the threshold update function 113 updates the confidence level in each group to threshold T. conf Number of detection frames exceeding N conf (G k ) and threshold T Nconf Compare this with the number of detection frames N conf (G k ) is the threshold T Nconf The lesion type corresponding to the group exceeding a certain threshold is determined to be the lesion type in the current frame.

[0069] Here, the confidence level in group G1 is the threshold T. conf Number of detection frames exceeding N conf (G1), and the confidence level in group G2 is threshold T conf Number of detection frames exceeding N conf (G2) is the threshold T Nconf It exceeds [a certain threshold]. Therefore, lesion type Cyst, corresponding to group G1, and lesion type Hema, corresponding to group G2, are determined to be the lesion types in the current frame.

[0070] As a result, the lesion detection device 1 according to this embodiment can also determine the lesion type in the current frame based on the reliability of the lesion type in the detection result of the previous frame, thereby improving the accuracy of determining the lesion type in the current frame.

[0071] Furthermore, in image-based lesion detection, detection frames in which lesions are correctly detected often appear continuously or at regular intervals, resulting in high reliability. Therefore, the lesion detection device 1 according to this embodiment can improve the accuracy of determining the lesion type in the current frame by determining the lesion type in the current frame based on the continuity and / or frequency and reliability of lesion types in the detection results of the previous frame.

[0072] The update of the detection threshold will be explained with reference to Figure 4. Figure 4 is a schematic diagram showing the update of the detection threshold in the first embodiment.

[0073] The threshold update function 113 updates the detection threshold based on the type of lesion in the current frame that has been determined.

[0074] Specifically, the threshold update function 113 may update the detection threshold for a lesion type in the current frame that has been determined, so as to lower the detection threshold for that lesion type. For example, the threshold update function 113 may update the detection threshold for a lesion type in the current frame that has been determined, so as to lower the detection threshold by a predetermined value. Alternatively, the threshold update function 113 may update the detection threshold for a lesion type that does not exist in the current frame, so as to raise the detection threshold for that lesion type. For example, the threshold update function 113 may update the detection threshold for a lesion type that does not exist in the current frame, so as to raise the detection threshold by a predetermined value.

[0075] For example, as shown in Figure 4, if it is determined that the lesion types in the current frame are Cyst and Hema, and that there are no lesion types HCC and Meta in the current frame, the threshold update function 113 updates the detection thresholds so as to lower the detection thresholds corresponding to lesion type Cyst and lesion type Hema, and raise the detection thresholds corresponding to lesion type HCC and lesion type Meta.

[0076] Next, the lesion detection function 111 detects lesions in the current frame based on the updated detection threshold. For example, the lesion detection function 111 displays detection frames for lesion types whose confidence level exceeds the updated detection threshold on the display 14, and does not display detection frames for lesion types whose confidence level is below the updated detection threshold on the display 14.

[0077] As a result, the lesion detection device 1 according to this embodiment can update the detection threshold according to the lesion type in the current frame, lowering the detection threshold corresponding to the lesion type present in the current frame to increase the probability of detecting a lesion of that type, and raising the detection threshold corresponding to the lesion type not present in the current frame to decrease the probability of detecting a lesion of that type not present in the current frame. Therefore, the lesion detection device 1 according to this embodiment can update the detection threshold corresponding to the lesion type present in the current frame without increasing the number of erroneous detection frames, thereby improving the accuracy of lesion detection.

[0078] Furthermore, the threshold update function 113 may set an upper or lower limit for the updated detection threshold. Note that an upper or lower limit may already be set for the updated detection threshold. For example, the threshold update function 113 may set an upper limit for the updated detection threshold when increasing the detection threshold, and a lower limit when decreasing the detection threshold.

[0079] This will be explained using lesion types HCC and Meta as examples. The threshold update function 113 sets upper limits on the detection thresholds corresponding to lesion type HCC and lesion type Meta to prevent the detection of malignant diseases such as metastatic liver cancer (Meta) and hepatocellular carcinoma (HCC) in the current frame from decreasing due to excessively high detection thresholds.

[0080] Furthermore, the threshold update function 113 may set lower limits for the detection threshold corresponding to lesion type Hema and the detection threshold corresponding to lesion type Cyst, so as to prevent the detection accuracy in the current frame from decreasing due to the detection threshold being too low.

[0081] The display contents of the display 14 will be described with reference to Figures 5A and 5B. Figures 5A and 5B are schematic diagrams showing an example of the display contents of the display 14 according to the first embodiment. The display 14 displays various images and various information under the control of the lesion detection function 111. That is, the lesion detection function 111 causes various images and various information to be displayed on the display 14.

[0082] As shown in Figure 5A, the display 14 displays a detection frame (dashed line frame in Figure 5A) indicating a detected lesion when a lesion is detected based on the detection threshold updated by the lesion detection function 111. The display 14 may also display the detection frame in a way that changes the color of the frame depending on the type of lesion detected (not shown). For example, the lesion detection function 111 changes the color of the detection frame displayed on the display 14 to blue when the detected lesion type is Cyst, and changes the color of the detection frame displayed on the display 14 to green when the detected lesion type is Hema. The lesion detection function 111 may also change the color of the detection frame displayed on the display 14 to any other color.

[0083] Furthermore, the lesion detection function 111 may display the detection frame along with the confidence level of the detected lesion type on the display 14, as shown in Figure 5B. The lesion detection function 111 may also display information regarding the update (the "threshold update status" in Figure 5B) on the display 14 when the detection threshold is updated.

[0084] As a result, the lesion detection device 1 displays information such as the detection frame of the detected lesion, the reliability of the lesion type, and information regarding the update of the detection threshold on the display 14. Therefore, the user can easily check the update status of the detection threshold and the detection status of lesions.

[0085] (Flowchart of the first embodiment) The following describes the flow of the lesion detection method according to the first embodiment (the flow of the lesion detection process according to the first embodiment) with reference to Figure 6. Figure 6 is a flowchart of the lesion detection method according to the first embodiment.

[0086] In step S101, the lesion detection device 1 acquires the captured image, and the lesion detection function 111 detects a lesion in each frame of the image. Next, the process proceeds to step S102.

[0087] In step S102, the frame result acquisition function 112 receives the detection result from the previous frame detected by the lesion detection function 111. The frame result acquisition function 112 also sends the detection result from the previous frame to the memory 12, where the detection result is stored in the memory 12 in the form of a queue. Next, the process proceeds to step S103.

[0088] In step S103, the threshold update function 113 calculates the degree of overlap IoU between two detection frames with the same lesion type in adjacent frames, based on the detection results stored in memory 12. Next, the process proceeds to step S104.

[0089] In step S104, the threshold update function 113 determines that the overlap degree IoU calculated in step S103 between two detection frames with the same lesion type in adjacent frames within the same group is set to a preset threshold T. IoU The detection results are grouped so that they exceed a certain threshold. Next, proceed to step S105.

[0090] In step S105, the threshold update function 113 updates the number of detection frames N(G) in each group that was grouped in step S104. k Count the number of )

[0091] Here, the processes of steps S106 to S113 described below are executed for each group. For example, the threshold update function 113 executes a process of determining whether there is an unselected group among one or more groups grouped in step S104 between the process of step S105 and the process of step S106. Then, when it is determined that there is an unselected group, the threshold update function 113 selects one group from the unselected groups. And the selected group becomes the target of the processes of steps S106 to S113. When it is determined that there is no unselected group, the threshold update function 113 proceeds to step S114.

[0092] In step S106, the threshold update function 113 compares the number N(G k ) of detection frames in the selected group with the threshold T N . When the number N(G k ) of detection frames in the selected group does not exceed the threshold T N , it proceeds to step S107. When the number N(G k ) of detection frames in the selected group exceeds the threshold T N , it proceeds to step S108.

[0093] <​​​​​​​​​​​​​​​​​​​​​​​​​​In step S110, the threshold update function 113 determines whether the number N conf of detection frames whose reliability in the selected group exceeds the threshold T conf (G k ) is greater than the threshold T Nconf . If the number N conf (G k ) of detection frames exceeds the threshold T Nconf , the process proceeds to step S111. If the number N conf (G k ) of detection frames does not exceed the threshold T Nconf , the processing after step S111 is not executed.

[0097] In step S111, the threshold update function 113 determines the lesion type corresponding to the selected group as the lesion type in the current frame. Next, the process proceeds to step S112.

[0098] In step S112, the threshold update function 113 updates the detection threshold according to the lesion type in the current frame determined in step S111. For example, the threshold update function 113 updates the detection threshold so as to lower the detection threshold corresponding to the lesion type in the current frame and raise the detection threshold corresponding to the lesion types other than the lesion type in the current frame. Next, the process proceeds to step S113.

[0099] In step S113, the lesion detection function 111 detects lesions in the current frame based on the detection threshold updated in step S112.

[0100] In step S114, the lesion detection function 111 causes the display 14 to display the lesions detected in step S113 for all the groups determined in step S110 to have the number N conf (G k ) of detection frames exceeding the threshold T Nconf . The display 14 may display, under the control of the lesion detection function 111, the detection frames of the detected lesions, the reliability of the lesion type, information regarding the update of the detection threshold, etc. Thereby, the flow shown in FIG. 6 ends.

[0101] (Comparative example) Referring to Figures 7A and 7B, a comparison will be made between the detection results obtained by the lesion detection device according to the comparative example and the detection results obtained by the lesion detection device 1 according to the first embodiment. Figure 7A is a schematic diagram showing the detection results obtained by the lesion detection device according to the comparative example. Figure 7B is a schematic diagram showing the detection results obtained by the lesion detection device 1 according to the first embodiment. Figure 7A shows the detection results obtained by the lesion detection device according to the comparative example, and Figure 7B shows the detection results obtained by the lesion detection device 1 according to the first embodiment.

[0102] In the following description, we will explain an example of detecting liver metastatic cancer (Meta) in a video using the lesion detection device according to the comparative example and the lesion detection device according to the first embodiment.

[0103] As shown in Figures 7A and 7B, the lesion areas of liver metastasis cancer in frames 42-45 of the video have high contrast and clear edges. Therefore, both the lesion detection device according to the comparative example and the lesion detection device 1 according to the first embodiment can detect highly reliable liver metastasis cancer (Meta) in frames 42-45 and display the detection frame.

[0104] However, the lesion area of ​​liver metastasis cancer in frames 46-49 of the video has poor contrast and blurred edges. For this reason, the lesion detection device according to the comparative example cannot detect liver metastasis cancer (Meta) and display a detection frame in frames 46-49. In contrast, the lesion detection device 1 according to the first embodiment can detect liver metastasis cancer (Meta) in frames 46-49 in the same way as in frames 42-45 and display a detection frame.

[0105] This is because, in the lesion detection device of the comparative example, the detection threshold is usually set to a fixed value; for example, the detection threshold corresponding to lesion type Meta is set to 0.076. In this case, in frames with high contrast (frames 42-45), a highly reliable detection result can be detected and the detection frame can be displayed, but in frames with low contrast (frames 46-49), a detection result higher than the set detection threshold cannot be detected, and no detection frame is displayed.

[0106] In contrast, in the lesion detection device 1 according to the first embodiment, the threshold update function 113 determines the lesion type in the current frame based on the detection result of the previous frame and updates the detection threshold according to the determined lesion type in the current frame. Therefore, the detection threshold can be appropriately updated in the current frame according to the lesion type.

[0107] As a result, the threshold update function 113 can, for example, update the detection threshold in frames 46-49 by determining the type of lesion in frames 46-49 from the detection results in the previous frame prior to frames 46-49, if the contrast in frames 46-49 is low.

[0108] For example, if the threshold update function 113 determines that the lesion type in frames 46-49 is metastatic liver cancer (Meta), it can update the detection threshold corresponding to the lesion type Meta by appropriately lowering it to 0.035. In this case, the threshold update function 113 can detect detection results that exceed the updated detection threshold in frames 46-49 and display the detection frame.

[0109] (Second embodiment) Referring to Figure 8, an example of the configuration of the lesion detection device 1A according to the second embodiment will be described. Figure 8 is a block diagram showing an example of the configuration of the lesion detection device 1A according to the second embodiment. Hereinafter, only the differences between the lesion detection device 1A according to the second embodiment and the lesion detection device 1 according to the first embodiment will be described, and the common points may be omitted from the explanation.

[0110] As shown in Figure 8, the processing circuit 11A of the lesion detection device 1A is further equipped with a learning function 115.

[0111] The learning function 115 is a function unit that trains the system using a pre-trained model. The pre-trained model is stored in memory 12.

[0112] The learning function 115 trains the model to set (output) a detection threshold in the current frame based on the detection result in the previous frame, thereby generating a trained model. The learning function 115 then stores the generated trained model in memory 12. The threshold update function 113 then updates the detection threshold in the current frame by inputting the detection result from the previous frame to the trained model stored in memory 12.

[0113] Specifically, the threshold update function 113 inputs, for example, the lesion type, location (detection frame coordinates), and confidence level corresponding to the detection frame in the previous frame into the trained model, and causes the trained model to output a detection threshold corresponding to each lesion type. Then, the threshold update function 113 updates the detection threshold corresponding to the corresponding lesion type by setting the detection threshold output from the trained model as the detection threshold corresponding to the corresponding lesion type.

[0114] Furthermore, during the training phase (learning phase) of a pre-trained model, manually labeled detection frames can be used as training data. The target detection threshold for the pre-trained model is to improve the detection rate (the percentage of manually labeled detection frames that match the detection frames output by the pre-trained model) and decrease the false positive rate (the percentage of manually labeled detection frames that do not match the detection frames output by the trained model).

[0115] As a result, the lesion detection device 1A according to the second embodiment updates the detection threshold for the current frame by inputting the detection result from the previous frame to the trained model. Therefore, the detection threshold can be appropriately updated, and the accuracy of lesion detection can be further improved.

[0116] (Third embodiment) Figure 9 is a block diagram showing an example configuration of an ultrasound diagnostic apparatus 1B according to the third embodiment. As illustrated in Figure 9, the ultrasound diagnostic apparatus 1B according to the first embodiment includes a main body 100, an ultrasound probe 101, an input device 102, and a display 103. The ultrasonic probe 101 has, for example, multiple elements (piezoelectric transducers, piezoelectric elements). These multiple elements generate ultrasound based on a drive signal supplied from the transmitting circuit 116 of the transmitting / receiving circuit 110 of the device body 100. Specifically, the multiple elements generate ultrasound with a waveform corresponding to the transmitting drive voltage when a voltage (transmitting drive voltage) is applied by the transmitting circuit 116. The waveform of the transmitting drive voltage indicated by the drive signal is the waveform of the voltage applied to the multiple elements. In other words, the ultrasonic probe 101 transmits ultrasound corresponding to the magnitude of the applied transmitting drive voltage. The ultrasonic probe 101 also receives reflected waves from the subject P, converts them into an electrical signal called a received signal (reflected wave signal), and outputs the received signal to the device body 100. The ultrasonic probe 101 also has, for example, a matching layer provided on the elements and a backing material to prevent the propagation of ultrasound backward from the elements. The ultrasonic probe 101 is detachably connected to the device body 100.

[0117] When ultrasound is transmitted from the ultrasound probe 101 to the subject P, the transmitted ultrasound is reflected one after another by discontinuities in acoustic impedance within the subject P's internal tissues, and the reflected waves are received by multiple elements of the ultrasound probe 101. The amplitude of the received reflected waves depends on the difference in acoustic impedance at the discontinuities where the ultrasound is reflected. When the transmitted ultrasound pulse is reflected by a moving blood flow or the surface of the heart wall, the reflected waves undergo a frequency shift due to the Doppler effect, depending on the velocity component of the moving object relative to the ultrasound transmission direction. The ultrasound probe 101 then outputs the received signal to the receiving circuit 117 of the transmitting / receiving circuit 110, which will be described later.

[0118] The ultrasonic probe 101 is detachably attached to the main unit 100 of the device. When scanning a two-dimensional area within the subject P (two-dimensional scanning), the operator connects a 1D array probe, for example, in which multiple elements are arranged in a row, to the main unit 100 as the ultrasonic probe 101. Examples of 1D array probes include linear ultrasonic probes, convex ultrasonic probes, and sector ultrasonic probes. When scanning a three-dimensional area within the subject P (three-dimensional scanning), the operator connects a mechanical 4D probe or a 2D array probe to the main unit 100 as the ultrasonic probe 101. A mechanical 4D probe can perform two-dimensional scanning using multiple elements arranged in a row, similar to a 1D array probe, and can also perform three-dimensional scanning by oscillating the multiple elements at a predetermined angle (oscillation angle). A 2D array probe can perform three-dimensional scanning using multiple elements arranged in a matrix, and can also perform two-dimensional scanning by focusing and transmitting ultrasound.

[0119] The input device 102 is implemented by input means such as a mouse, keyboard, buttons, panel switches, touch command screen, foot switch, trackball, or joystick. The input device 102 receives various setting requests from the operator of the ultrasound diagnostic device 1B and transmits the received setting requests to the main unit 100 of the device.

[0120] The display 103 may, for example, display a GUI (Graphical User Interface) for the operator of the ultrasound diagnostic device 1B to input various setting requests using the input device 102, or display ultrasound images based on ultrasound image data generated by the device body 100. The display 103 is implemented using an LCD monitor, a CRT (Cathode Ray Tube) monitor, or the like.

[0121] The device body 100 generates ultrasound image data based on the received signal transmitted from the ultrasound probe 101. Note that the ultrasound image data is just an example of image data. The device body 100 can generate two-dimensional ultrasound image data based on the received signal corresponding to the two-dimensional region of the subject P transmitted from the ultrasound probe 101. Furthermore, the device body 100 can generate three-dimensional ultrasound image data based on the received signal corresponding to the three-dimensional region of the subject P transmitted from the ultrasound probe 101. As shown in Figure 1, the device body 100 includes a transmit / receive circuit 110, a buffer memory 120, a signal processing circuit 130, an image generation circuit 140, an image memory 150, a storage circuit 160, a control circuit 170, and a processing circuit 180.

[0122] The transmitting / receiving circuit 110, under the control of the control circuit 170, causes the ultrasonic probe 101 to transmit ultrasonic waves and the ultrasonic probe 101 to receive reflected ultrasonic waves. In other words, the transmitting / receiving circuit 110 performs scanning via the ultrasonic probe 101. The transmitting / receiving circuit 110 is an example of a transmitting / receiving unit. The transmitting / receiving circuit 110 has a transmitting circuit 116 and a receiving circuit 117. The transmitting circuit 116 is an example of a transmitting unit, and the receiving circuit 117 is an example of a receiving unit.

[0123] The transmitting circuit 116, under the control of the control circuit 170, causes the ultrasonic probe 101 to transmit ultrasound. The transmitting circuit 116 includes a rate pulser generation circuit, a transmission delay circuit, and a transmitting pulser. The transmitting circuit 116 supplies a drive signal to the ultrasonic probe 101. When scanning a two-dimensional region within the subject P, the transmitting circuit 116 causes the ultrasonic probe 101 to transmit an ultrasonic beam for scanning the two-dimensional region. When scanning a three-dimensional region within the subject P, the transmitting circuit 116 causes the ultrasonic probe 101 to transmit an ultrasonic beam for scanning the three-dimensional region.

[0124] The rate pulser generation circuit, under the control of the control circuit 170, repeatedly generates rate pulses at a predetermined rate frequency (PRF: Pulse Repetition Frequency) to form a transmitted ultrasonic wave (transmitted beam). The rate pulses pass through the transmit delay circuit, applying voltages to the transmit pulser with different transmit delay times. For example, the transmit delay circuit provides each rate pulse generated by the rate pulser generation circuit with a transmit delay time for each element necessary to focus the ultrasonic waves generated from the ultrasonic probe 101 into a beam and determine the transmit directivity. The transmit pulser supplies a drive signal (drive pulse) to the ultrasonic probe 101 at a timing based on the rate pulse. That is, the transmit pulser applies a voltage (transmit drive voltage) of the waveform indicated by the drive signal to the ultrasonic probe 101 at a timing based on the rate pulse. The transmit delay circuit arbitrarily adjusts the transmission direction of the ultrasonic waves from the element surface by changing the transmit delay time applied to each rate pulse.

[0125] The drive pulse is transmitted from the transmitting pulser through the cable to the element in the ultrasonic probe 101, where it is converted from an electrical signal to a mechanical vibration. That is, when a voltage is applied to the element, it vibrates mechanically. The ultrasound generated by this mechanical vibration is transmitted into the living body (inside the subject P). Here, the ultrasound, which has a different transmission delay time for each element, is focused and propagates in a predetermined direction.

[0126] Furthermore, the transmitting circuit 116, under the control of the control circuit 170, has the function of instantaneously changing the transmitting frequency, transmitting drive voltage, etc., in order to execute a predetermined scanning sequence. In particular, the change in the transmitting drive voltage is achieved by a linear amplifier type oscillator circuit that can instantly switch the value of the transmitting drive voltage, or by a mechanism that electrically switches multiple power supply units.

[0127] The reflected ultrasonic waves transmitted by the ultrasonic probe 101 reach an element inside the ultrasonic probe 101, where they are converted from mechanical vibrations into electrical signals (received signals), and these received signals are input to the receiving circuit 117. The receiving circuit 117 includes a preamplifier, an A / D (Analog to Digital) converter, a quadrature detection circuit, a beamformer, etc., and performs various processing on the received signals transmitted from the ultrasonic probe 101 to generate reflected wave data (received data). The receiving circuit 117 then stores the generated reflected wave data in the buffer memory 120.

[0128] The preamplifier amplifies the received signal for each channel and performs gain adjustment (gain correction). The A / D converter converts the gain-corrected received signal into a digital signal by A / D conversion. The quadrature detection circuit converts the received signal, which has been converted into a digital signal, into a baseband in-phase signal (I signal, I: In-phase) and a quadrature signal (Q signal, Q: Quadrature-phase). The beamformer then performs beamforming on the IQ signal (I signal and Q signal) and stores the data obtained by beamforming as reflected wave data in the buffer memory 120.

[0129] The receiving circuit 117 generates two-dimensional reflected wave data from the two-dimensional received signal transmitted from the ultrasonic probe 101. The receiving circuit 117 also generates three-dimensional reflected wave data from the three-dimensional received signal transmitted from the ultrasonic probe 101.

[0130] The buffer memory 120 is a memory that temporarily stores reflected wave data generated by the transmitting and receiving circuit 110. For example, the buffer memory 120 is configured to store a predetermined number of frames of reflected wave data. When the buffer memory 120 has stored a predetermined number of frames of reflected wave data, and a new frame of reflected wave data is generated by the receiving circuit 117, the buffer memory 120, under the control of the receiving circuit 117, discards the oldest frame of reflected wave data and stores the newly generated frame of reflected wave data. For example, the buffer memory 120 can be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory.

[0131] The signal processing circuit 130 reads reflected wave data from the buffer memory 120, performs various signal processing on the read reflected wave data, and outputs the processed reflected wave data as B-mode data or Doppler data to the image generation circuit 140. The signal processing circuit 130 can be implemented, for example, by a processor. The signal processing circuit 130 is an example of a signal processing unit.

[0132] For example, whenever a new frame of reflected wave data is stored in the buffer memory 120, the signal processing circuit 130 reads out the newly stored frame of reflected wave data. The signal processing circuit 130 then applies various signal processing to the read frame of reflected wave data to generate a new frame of B-mode data or Doppler data. Each time the signal processing circuit 130 generates a frame of B-mode data or Doppler data, it outputs the newly generated frame of B-mode data or Doppler data to the image generation circuit 140. The following describes some examples of the various signal processing operations performed by the signal processing circuit 130.

[0133] For example, the signal processing circuit 130 performs quadrature detection on the reflected wave data read from the buffer memory 120, and then applies logarithmic amplification and envelope detection processing to generate B-mode data in which the signal intensity (amplitude intensity) for each sample point is expressed as brightness. For example, the signal processing circuit 130 outputs the generated B-mode data to the image generation circuit 140.

[0134] Furthermore, the signal processing circuit 130 performs frequency analysis on the reflected wave data read from the buffer memory 120 to extract motion information of moving objects (blood flow, tissue, contrast agent echo components, etc.) based on the Doppler effect from the reflected wave data, and generates Doppler data showing the extracted motion information. For example, the signal processing circuit 130 extracts average velocity, average variance, and average power values ​​as motion information of a moving object across multiple points, and generates Doppler data showing the extracted motion information of the moving object. The signal processing circuit 130 outputs the generated Doppler data to the image generation circuit 140.

[0135] Using the functions of the signal processing circuit 130 described above, the ultrasound diagnostic device 1B can perform color Doppler ultrasound, also known as color flow mapping (CFM). In color flow mapping, ultrasound is transmitted and received multiple times on multiple scan lines. In color flow mapping, an MTI (Moving Target Indicator) filter is applied to the data sequence at the same location to suppress signals originating from stationary or slow-moving tissue (clutter signals) and extract signals originating from blood flow (blood flow signals). Color flow mapping then estimates blood flow information such as blood flow velocity, blood flow dispersion, and blood flow power from this blood flow signal. The signal processing circuit 130 outputs color image data showing the blood flow information estimated by color flow mapping to the image generation circuit 140. The color image data is an example of Doppler data.

[0136] The signal processing circuit 130 is capable of processing both two-dimensional and three-dimensional reflected wave data.

[0137] The image generation circuit 140 generates ultrasound image data from B-mode data or Doppler data output from the signal processing circuit 130. The image generation circuit 140 is implemented by a processor.

[0138] For example, the image generation circuit 140 generates two-dimensional B-mode image data from the two-dimensional B-mode data generated by the signal processing circuit 130, representing the intensity of reflected waves in terms of brightness. The image generation circuit 140 also generates two-dimensional Doppler image data from the two-dimensional Doppler data generated by the signal processing circuit 130, which visualizes motion information or blood flow information. The two-dimensional Doppler image data that visualizes motion information may be velocity image data, dispersion image data, power image data, or a combination of these.

[0139] Here, the image generation circuit 140 generally converts the scan line signal sequence of the ultrasonic scan into a scan line signal sequence of a video format, such as that used in televisions (scan conversion), and generates ultrasonic image data for display. For example, the image generation circuit 140 generates ultrasonic image data for display by performing a coordinate transformation on the data output from the signal processing circuit 130 according to the scanning pattern of the ultrasonic probe 101. In addition to scan conversion, the image generation circuit 140 also performs various image processing tasks, such as image processing that regenerates an average brightness image using multiple image frames after scan conversion (smoothing process), and image processing that uses a differential filter within the image (edge ​​enhancement process). Furthermore, the image generation circuit 140 synthesizes various parameter text information, scales, body marks, etc., with the ultrasonic image data.

[0140] Furthermore, the image generation circuit 140 generates 3D B-mode image data by performing a coordinate transformation on the 3D B-mode data generated by the signal processing circuit 130. The image generation circuit 140 also generates 3D Doppler image data by performing a coordinate transformation on the 3D Doppler data generated by the signal processing circuit 130. In other words, the image generation circuit 140 generates "3D ultrasound image data (volume data)" from the "3D B-mode image data and 3D Doppler image data". Then, the image generation circuit 140 performs various rendering processes on the volume data to generate various 2D image data for display on the display 103.

[0141] The rendering process performed by the image generation circuit 140 includes, for example, generating MPR image data from volume data using the Multi-Planer Reconstruction (MPR) method. Another rendering process performed by the image generation circuit 140 is volume rendering (VR), which generates 2D image data that reflects 3D information. The image generation circuit 140 is an example of an image generation unit.

[0142] B-mode data and Doppler data are ultrasound image data before scan conversion processing, while the data generated by the image generation circuit 140 is ultrasound image data for display after scan conversion processing. B-mode data and Doppler data are also referred to as raw data.

[0143] The image memory 150 is a memory that stores various image data generated by the image generation circuit 140. The image memory 150 also stores data generated by the signal processing circuit 130. The B-mode data and Doppler data stored in the image memory 150 can be retrieved by the operator after a diagnosis, for example, and become ultrasound image data for display via the image generation circuit 140. For example, the image memory 150 can be implemented using semiconductor memory elements such as RAM (Random Access Memory) or flash memory, a hard disk, or an optical disk.

[0144] The memory circuit 160 stores control programs for scanning (transmission and reception of ultrasound), image processing, and display processing, as well as various data such as diagnostic information (e.g., patient ID, physician's findings, etc.), diagnostic protocols, and various body marks. The memory circuit 160 is also used, if necessary, to store data stored in the image memory 150. For example, the memory circuit 160 can be implemented using semiconductor memory elements such as flash memory, a hard disk, or an optical disk.

[0145] The control circuit 170 controls the entire processing of the ultrasound diagnostic device 1B. Specifically, the control circuit 170 controls the processing of the transmission circuit 116, reception circuit 117, signal processing circuit 130, and image generation circuit 140 based on various setting requests input from the operator via the input device 102, and various control programs and data read from the memory circuit 160. The control circuit 170 also controls the display 103 to display an ultrasound image based on ultrasound image data for display stored in the image memory 150. For example, the control circuit 170 controls the display 103 to display a B-mode image based on B-mode image data or a color image based on color image data. The control circuit 170 also controls the display 103 to display a color image superimposed on a B-mode image. The control circuit 170 is an example of a display control unit or a control unit. The control circuit 170 is implemented, for example, by a processor. An ultrasound image is an example of an image.

[0146] Furthermore, the control circuit 170 controls the ultrasonic scanning by controlling the ultrasonic probe 101 via the transmitting and receiving circuit 110.

[0147] The processing circuit 180 includes a lesion detection function 180a, a frame result acquisition function 180b, a threshold update function 180c, and a learning function 180d. For example, each of the processing functions of the processing circuit 180 shown in Figure 9—the lesion detection function 180a, the frame result acquisition function 180b, the threshold update function 180c, and the learning function 180d—is recorded in the memory circuit 160 in the form of a program that can be executed by a computer. The processing circuit 180 reads each program from the memory 160 and executes each read program to realize the function corresponding to each program. In other words, the processing circuit 180 in the state where each program has been read has the functions shown in the processing circuit 180 in Figure 9. The processing circuit 180 is implemented by, for example, a processor.

[0148] The processing circuit 180 performs the same processing as the processing circuit 11 of the first embodiment or the processing circuit 11A of the second embodiment on various images (videos) stored in the image memory 150, and displays the processing results on the display 103.

[0149] The lesion detection function 180a has the same function as the lesion detection function 111 according to the first or second embodiment. The frame result acquisition function 180b has the same function as the frame result acquisition function 112 according to the first or second embodiment. The threshold update function 180c has the same function as the threshold update function 113 according to the first or second embodiment. The learning function 180d has the same function as the learning function 115 according to the second embodiment. Therefore, the ultrasound diagnostic apparatus 1B according to the third embodiment provides the same effects as the first or second embodiment.

[0150] In this description, the term "processor" refers to circuits such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), or programmable logic device (e.g., Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), or Field Programmable Gate Array (FPGA)). The processor performs its functions by reading a program stored in the memory circuit 160 and executing the read program. Alternatively, instead of storing the program in the memory circuit 160, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor performs its functions by reading and executing the program incorporated into the circuitry. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor, and its functions may be performed by combining them. Furthermore, the multiple circuits in Figure 9 (for example, the signal processing circuit 130, the image generation circuit 140, the control circuit 170, and the processing circuit 180) may be integrated into a single processor to realize their functions. That is, the signal processing circuit 130, the image generation circuit 140, the control circuit 170, and the processing circuit 180 may be integrated into a single processing circuit realized by the processor.

[0151] The overall configuration of the ultrasound diagnostic apparatus 1B according to the third embodiment has been described above. The ultrasound diagnostic apparatus 1B according to the third embodiment provides the same effects as the first or second embodiment.

[0152] According to the lesion detection device, lesion detection method, and ultrasound diagnostic device according to each embodiment, the type of lesion in the current frame can be determined based on the detection result in the previous frame, and the detection threshold can be appropriately updated according to the type of lesion in the current frame, thereby improving the accuracy of lesion detection.

[0153] Furthermore, compared to setting a fixed detection threshold, it is possible to distinguish between lesion types in the current frame and lesion types other than those in the current frame and assign different detection thresholds. This reduces the possibility of detecting lesions that do not exist in the current frame, reduces the number of erroneous detection frames, and further improves the accuracy of lesion detection.

[0154] Furthermore, by reducing the number of detection frames for lesion types other than the current lesion type (false detection frames), the user's visual experience is improved and interference can be reduced.

[0155] Furthermore, by increasing the probability of detecting lesions present in the current frame while decreasing the probability of detecting lesions not present in the current frame, the accuracy of lesion detection can be improved without increasing the total number of detection frames.

[0156] Furthermore, the lesion detection device, lesion detection method, and ultrasound diagnostic device according to each embodiment can be applied to the detection of multiple types of lesions in various images using various existing detection models.

[0157] In the embodiments described above, the processing circuit is not limited to being implemented by a single processor, but may be composed of a combination of multiple independent processors, with each processor executing a program to realize each processing function. Furthermore, each processing function of the processing circuit may be implemented by appropriately distributing or integrating it across one or more processing circuits. In addition, each processing function of the processing circuit may be implemented by a mixture of hardware such as circuits and software. Moreover, although an example in which the programs corresponding to each processing function are stored in a single memory circuit has been described here, the embodiments are not limited to this. For example, the programs corresponding to each processing function may be distributed and stored in multiple memory circuits, and the processing circuit may read and execute each program from each memory circuit.

[0158] Furthermore, while the embodiments described above illustrate examples where each functional unit in this specification is implemented by a processing circuit, the embodiments are not limited to these. For example, each functional unit in this specification may be implemented not only by the processing circuits described in the embodiments, but also by hardware alone, software alone, or a combination of hardware and software.

[0159] Furthermore, in the embodiments described above, each component of each illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of distribution or integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed or integrated in any unit according to various loads and usage conditions. Moreover, each processing function performed by each device can be implemented, in whole or in any part, by a CPU and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

[0160] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0161] 1. Lesion detection device 111 Lesion detection function 112 Frame result acquisition function 113 Threshold update function

Claims

1. A lesion detection unit detects lesions in each of the multiple captured frames based on a detection threshold corresponding to the type of lesion, A frame result acquisition unit that acquires the detection results of the lesions in the plurality of frames, A threshold update unit determines the lesion type based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of frames, and updates the detection threshold according to the determined lesion type. Equipped with, The lesion detection unit is a lesion detection device that detects the lesion based on the updated detection threshold.

2. The frame result acquisition unit acquires the detection results of the lesion in a plurality of previous frames prior to the current frame, The threshold update unit determines the lesion type in the current frame based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of previous frames, and updates the detection threshold according to the determined lesion type in the current frame. The lesion detection device according to claim 1, wherein the lesion detection unit detects the lesion in the current frame based on the updated detection threshold.

3. The lesion detection device according to claim 2, wherein the threshold update unit further determines the lesion type in the current frame based on the reliability of the lesion type in the detection result of the previous frame.

4. The lesion detection device according to claim 3, wherein the threshold updating unit updates the detection threshold to lower the detection threshold when at least one of the continuity and frequency of occurrence and the reliability satisfy a preset condition.

5. The lesion detection device according to claim 4, wherein the threshold update unit sets a lower limit for the updated detection threshold.

6. The lesion detection device according to claim 3, wherein the threshold updating unit updates the detection threshold to increase the detection threshold when at least one of the continuity and frequency of occurrence and the reliability satisfy a preset condition.

7. The lesion detection device according to claim 6, wherein the threshold update unit sets an upper limit on the updated detection threshold.

8. The lesion detection device according to claim 1, wherein the lesion detection unit displays a detection frame indicating the detected lesion on a display when the lesion is detected based on the updated detection threshold.

9. The lesion detection device according to claim 8, wherein the lesion detection unit displays the detection frame on the display so as to change the color of the detection frame according to the type of lesion detected.

10. The lesion detection device according to claim 8, wherein the lesion detection unit displays the detection frame on the display along with the reliability of the detected lesion type.

11. The lesion detection device according to claim 8, wherein the lesion detection unit displays information regarding the update on the display when the detection threshold is updated.

12. The lesion detection device according to claim 2, wherein the threshold update unit updates the detection threshold in the current frame using the detection threshold output from the trained model by inputting the detection result in the previous frame to a trained model that has been trained to output the detection threshold in the current frame based on the detection result in the previous frame.

13. A lesion detection step in which a lesion is detected in each of the multiple captured frames based on a detection threshold for detecting a lesion corresponding to the type of lesion, A frame result acquisition step of acquiring the detection results of the lesion in the plurality of frames, A threshold update step which involves determining the lesion type based on at least one of the continuity and frequency of occurrence of the lesion type in the detection results of the plurality of frames, and updating the detection threshold according to the determined lesion type, Includes, A lesion detection method comprising detecting the lesion based on the updated detection threshold in the lesion detection step.

14. A lesion detection unit detects lesions in each of the multiple captured frames based on a detection threshold corresponding to the type of lesion, A frame result acquisition unit that acquires the detection results of the lesions in the plurality of frames, A threshold update unit determines the lesion type based on at least one of the continuity and frequency of appearance of the lesion type in the detection results of the plurality of frames, and updates the detection threshold according to the determined lesion type. Equipped with, The lesion detection unit is an ultrasound diagnostic device that detects the lesion based on the updated detection threshold.

Images