Ultrasonic diagnostic apparatus, model operation verification method, and storage medium

Abstract

The invention provides an ultrasonic diagnostic apparatus, a model operation verification method, and a storage medium. The present invention addresses the problem of supporting detailed verification of the operation of an image analysis model. A log management unit (36) creates a log in which the operation of the ultrasonic diagnostic device is recorded. A model operation management unit (40) stores, for each operation of an image analysis model (30), an image to be analyzed input to the image analysis model (30) and an analysis result of the image analysis model (30). When a specific model operation record in the log is selected, a report creation unit (42) creates a model operation report. The model operation report includes an image to be analyzed inputted to the image analysis model (30) and an analysis result of the image analysis model (30).

Description

Technical Field

[0001] The present invention relates to an ultrasonic diagnostic device and a model action verification method, and more particularly to a technology for verifying the action of an image analysis model as AI (Artificial Intelligence). Background Art

[0002] Ultrasonic diagnostic apparatuses are used in ultrasonic examinations of living bodies. Ultrasonic diagnostic apparatuses are medical devices that generate and display ultrasonic images based on reception signals obtained by transmitting and receiving ultrasonic waves. As a representative ultrasonic image, a tomographic image is known.

[0003] Recently, ultrasonic diagnostic devices having image analysis models generated by machine learning have become popular. The ultrasonic diagnostic devices described in Patent Documents 1 and 2 have image analysis models that automatically identify (or classify) tissue cross-sections based on tomographic images. Other image analysis models include models that detect lesions included in tomographic images and models that measure tissues on tomographic images.

[0004] Among ultrasonic diagnostic devices, there are devices with a log creation function. In such ultrasonic diagnostic devices, a log is created by sequentially recording multiple actions in chronological order (refer to patent document 3). By referring to the log, a series of actions in the past can be verified. Specifically, by referring to the log, it is possible to confirm whether the actions of each component are appropriate, or to determine the failure of any component. Generally speaking, a log is composed of multiple records corresponding to multiple actions (including event occurrence and execution of processing) that occur in chronological order. The information contained in each record is limited and generally very simple. Each record in the log is not associated with detailed information.

[0005] Patent Document 1: Japanese Patent No. 6423540

[0006] Patent Document 2: Japanese Patent Application Publication No. 2020-204970

[0007] Patent Document 3: Japanese Patent Application No. 2021-507325

[0008] The performance of the image analysis model has degraded due to various factors such as the degradation of the quality of the analyzed image and insufficient machine learning. It is expected that the performance or status of the image analysis model should be verified regularly. Previous ultrasonic diagnostic devices have not been able to fully meet such requirements. For example, in an ultrasonic diagnostic device with a log creation function, the actions of the image analysis model are recorded in a log. However, the content of each record in the log is limited, and the content of each record is generally very simple. It is difficult to verify the actions of the image analysis model in detail by referring to each record. Summary of the invention

[0009] An object of the present invention is to support verification of the operation of an image analysis model, or to support detailed verification of the operation of an image analysis model.

[0010] The ultrasonic diagnostic device involved in the present invention is characterized in that it includes: an analysis unit, which has a machine-learned image analysis model for analyzing ultrasonic images; a log creation unit, which creates a log composed of multiple action records by recording multiple actions including the actions of the image analysis model in chronological order; a storage unit, which stores detailed information for model action verification for each action of the image analysis model; and a report creation unit, which, when a specific model action record in the log is selected, creates a model action report containing detailed information associated with the specific model action record.

[0011] The model action verification method involved in the present invention is characterized in that it includes the following steps: creating a log consisting of multiple action records by recording multiple actions of an image analysis model that has been machine-learned to analyze ultrasonic images in chronological order; saving detailed information for model action verification for each action of the image analysis model; displaying the log; accepting selection of a specific model action record in the displayed log; determining detailed information to be associated with the specific model action record; creating a model action report containing the detailed information; and displaying the model action report.

[0012] Effects of the Invention

[0013] According to the present invention, it is possible to support verification of the operation of an image analysis model. Alternatively, according to the present invention, it is possible to support detailed verification of the operation of an image analysis model. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a block diagram showing a configuration example of an ultrasonic diagnostic apparatus according to an embodiment.

[0015] Figure 2 This is a block diagram showing a structural example of a model action management unit.

[0016] Figure 3 It is a diagram showing the detailed record column.

[0017] Figure 4 This is a diagram showing an example of a log.

[0018] Figure 5 This is a diagram showing the first example of the model action report.

[0019] Figure 6 This is a diagram showing the second example of the model action report.

[0020] Figure 7 This is a diagram showing the third example of the model action report.

[0021] Figure 8 This is a diagram showing the fourth example of the model action report.

[0022] Explanation of symbols

[0023] 28-image analysis unit, 30-image analysis model, 36-log management unit, 38-log analysis unit, 40-model action management unit, 42-report creation unit, 44-log storage unit, 46-detailed record storage unit, 48-analyzed image storage unit, 52-counter, 54-detailed record storage unit, 62-conversion unit, 64-image storage unit. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments will be described based on the drawings.

[0025] (1) Overview of Implementation

[0026] The ultrasonic diagnostic device involved in the embodiment includes an analysis unit, a log creation unit, a storage unit, and a report creation unit. The analysis unit has a machine-learned image analysis model that analyzes ultrasonic images. The log creation unit creates a log consisting of multiple action records by recording multiple actions including the actions of the image analysis model in chronological order. The storage unit saves detailed information for model action verification for each action of the image analysis model. When a specific model action record in the log is selected, the report creation unit creates a model action report containing detailed information associated with the specific model action record.

[0027] According to the above structure, the performance or status of the image analysis model can be verified in detail by referring to the detailed information contained in the model action report. The detailed information is more detailed information than the information contained in the log. For example, the model action record in the log contains information such as the execution time of the image analysis model and the identifier of the image analysis model. When the amount of information is defined as the first amount of information, the detailed information has a second amount of information that is larger than the first amount of information. The detailed information involved in the implementation method includes at least one of the analyzed image input to the image analysis model and the analysis result of the image analysis model. Usually, such information is discarded, but in an implementation method, such information is saved as detailed information for model action verification. In more detail, the detailed information is saved while establishing an association with the model action record in the log. Such an association is not recognized in general logs in the past.

[0028] In an embodiment, the storage unit stores the analyzed image as the above-mentioned detailed information for each action of the image analysis model. The analyzed image is an ultrasonic image input to the image analysis model or an image equivalent thereto. The model action report includes the analyzed image. According to this structure, it is possible to verify the action of the image analysis model while referring to the analyzed image input to the image analysis model. All ultrasonic images input to the image analysis model can be saved as analyzed images, and a part of all ultrasonic images input to the image analysis model can be saved as analyzed images.

[0029] The ultrasonic diagnostic apparatus according to the embodiment includes a conversion unit that converts an ultrasonic image input to an image analysis model into a low-resolution image. The image to be analyzed is a low-resolution image. According to this structure, the storage capacity required for storing the image to be analyzed can be reduced. It is also possible to set a storage period for each image to be analyzed, and automatically delete the image to be analyzed when the storage period has expired.

[0030] In an embodiment, the storage unit stores the analysis results of the image analysis model as the above-mentioned detailed information for each action of the image analysis model. The analysis results are included in the model action report. According to this structure, it is possible to verify the action of the image analysis model while referring to the analysis results of the image analysis model. All the analysis results of the image analysis model can be saved, and part of all the analysis results of the image analysis model can also be saved.

[0031] In an embodiment, the analysis result includes a plurality of scores corresponding to a plurality of classes. The report creation unit creates a graph based on the plurality of scores. The model operation report includes the graph. According to this structure, verification of the operation of the image analysis model becomes easy.

[0032] In an embodiment, the storage unit includes an image storage unit and a record storage unit. The image storage unit stores the analyzed image for each action of the image analysis model. The record storage unit stores a detailed record including the analysis result of the image analysis model for each action of the image analysis model. The analyzed image is a low-resolution image generated from the ultrasonic image input to the image analysis model. The detailed information in the model action report includes the analyzed image as input information of the image analysis model and the analysis result as output information of the image analysis model.

[0033] According to the above structure, the operation of the image analysis model can be verified while referring to both the input information and the output information of the image analysis model. Therefore, the performance and state of the image analysis model can be correctly determined, and in the case of certain failures, the cause can be determined. For example, if it is determined that the image analysis model is not functioning properly, the image analysis model can be relearned or replaced.

[0034] In an embodiment, each model action record included in the log includes a portion embedded with a hyperlink, and the hyperlink determines the location of detailed information associated with each model action record. The model action report is displayed by operating this portion. According to this structure, when a model action record of interest is found under the condition of referring to the log, the model action report can be displayed by a simple operation on the model action record. In this case, the model action report can be displayed in a pop-up. That is, the log and the model action report can be displayed at the same time.

[0035] The model action verification method involved in the implementation method includes a log creation process, a saving process, a log display process, a selection process, a determination process, a report creation process and a display process. In the log creation process, multiple actions including the actions of the image analysis model that has been machine-learned to analyze the ultrasonic image are recorded in chronological order, thereby creating a log consisting of multiple action records. In the saving process, detailed information for model action verification is saved for each action of the image analysis model. In the log display process, the log is displayed. In the selection process, a selection of a specific model action record in the displayed log is accepted. In the determination process, detailed information associated with the specific model action record is determined. In the report creation process, a model action report containing detailed information is created. In the display process, the model action report is displayed.

[0036] The program for executing the above-mentioned model action verification method is installed on the information processing device via a network or via a portable storage medium. The program is stored in a non-temporary storage medium in the information processing device. The information processing device is, for example, an ultrasonic diagnostic device.

[0037] (2) Details of implementation methods

[0038] Figure 1 An ultrasonic diagnostic apparatus according to an embodiment is shown in . This ultrasonic diagnostic apparatus is a medical apparatus installed in a medical institution such as a hospital and used for ultrasonic examination of a subject.

[0039] The ultrasonic probe 10 is a device for transmitting ultrasonic waves into a living body and receiving reflected waves from the living body. The ultrasonic probe 10 has a transducer array composed of a plurality of transducers. The transducer array forms an ultrasonic beam 12. A beam scanning plane 14 is formed by electronic scanning of the ultrasonic beam 12. The electronic scanning of the ultrasonic beam 12 is repeated according to the transmission frame rate, thereby repeatedly forming the beam scanning plane 14.

[0040] Electronic linear scanning and electronic sector scanning are known as electronic scanning methods for the ultrasonic beam 12. A two-dimensional transducer array may be provided as the transducer array. By two-dimensionally scanning the ultrasonic beam using the two-dimensional transducer array, volume data can be acquired from the three-dimensional space in the living body.

[0041] The transmission circuit 16 is a circuit that functions as a transmission beamformer, and outputs a plurality of transmission signals in parallel to the transducer array during transmission, thereby forming a transmission beam through the action of the transducer array.

[0042] The receiving circuit 18 is a circuit that functions as a receiving beam former. When receiving, it applies phasing and addition to a plurality of receiving signals output in parallel from the transducer array, thereby generating receiving beam data. As the electronic scanning is repeated, a receiving frame data column is output from the receiving circuit 18. Each receiving frame data is composed of a plurality of receiving beam data arranged in the electronic scanning direction. Each receiving beam data is composed of a plurality of echo data arranged in the depth direction.

[0043] The received frame data sequence is sent to the image forming unit 20 via a data processing unit (not shown). The data processing unit is a module that applies a plurality of processes to each received beam data. The plurality of processes include logarithmic transformation, filtering, and the like.

[0044] The image forming unit 20 has a digital scan converter (DSC). The DSC has a coordinate conversion function, a pixel interpolation function, etc. The DSC generates a display frame data column based on the received frame data column. The display frame data column is equivalent to a tomographic image as a dynamic image. Each display frame data constituting the display frame data column is equivalent to a tomographic image as a still image.

[0045] In the illustrated configuration example, after the display frame data sequence is temporarily stored in the cine memory 22, the display frame data sequence read therefrom is sent to the display 26 via the display processing unit 24. The tomographic image is displayed as a dynamic image or as a still image on the display 26. The display processing unit 24 has an image synthesis function, a color processing function, and the like.

[0046] The cine memory 22 is a temporary storage memory having a ring buffer structure. The cine memory 22 is composed of, for example, a semiconductor memory. The image forming unit 20 and the display processing unit 24 are each composed of, for example, a processor. The display 26 is composed of a liquid crystal display, an organic FL display device, etc. In the image forming unit 20, an ultrasonic image other than a tomographic image can also be formed. For example, a blood flow image, an elasticity image, etc. can also be formed.

[0047] The image analysis unit 28 has an image analysis model 30 as AI. The image analysis model 30 is a model that has been machine-learned, and is composed of, for example, a CNN (Convolutional Neural Network). The image analysis model 30 involved in the embodiment is a model for recognizing tissue cross-sections based on tomographic images. Specifically, it is a model for classifying classes. As the image analysis model 30, a model for measuring tissues based on tomographic images, a model for detecting lesions contained in tomographic images, etc. can be used.

[0048] In the embodiment, a display frame data column is transmitted from the cine memory 22 to the image analysis unit 28. The display frame data column is also transmitted to the information processing unit 34. The image analysis model 30 performs image analysis on each display frame data constituting the display frame data column (i.e., for each tomographic image), and outputs an analysis result for each display frame data. An analysis result column corresponding to the display frame data column is output from the image analysis unit 28 to the display processing unit 24 and the information processing unit 34. The analysis result column is sent to the display 26 via the display processing unit 24, and the analysis result column is displayed on the display 26. The image analysis unit 28 is composed of, for example, a processor. The received frame data may also be analyzed in the image analysis unit 28.

[0049] For example, the information processing unit 34 is composed of a CPU that executes a program. The information processing unit 34 functions as a control unit that controls the actions of the various elements that constitute the ultrasonic diagnostic apparatus. In the figure, multiple boxes are used to represent the multiple functions performed by the information processing unit 34. Specifically, the information processing unit 34 functions as a log management unit 36, a log analysis unit 38, and a model action management unit 40. This will be described in detail later. An operation panel 50 is connected to the information processing unit 34. The operation panel 50 is an input device having a trackball, multiple switches, multiple knobs, etc.

[0050] A log storage unit 44, a detailed record storage unit 46, and an analyzed image storage unit 48 are connected to the information processing unit 34. These storage units 44, 46, and 48 are composed of semiconductor memories, hard disks, and the like.

[0051] The log management unit 36 ​​functions as a log creation unit and a log reading unit. Specifically, a plurality of actions performed by the ultrasonic diagnostic device are recorded in chronological order, thereby creating a log consisting of a plurality of records. Typically, one record corresponds to one action. The plurality of actions performed by the ultrasonic diagnostic device include a plurality of actions of the image analysis model. The plurality of records constituting the log include a plurality of records corresponding to a plurality of model actions (a plurality of model action records). In practice, the log management unit 36 ​​registers the plurality of records in the log storage unit 44 in chronological order. Thus, a log is created on the log storage unit 44.

[0052] When referring to the log, the log management unit 36 ​​reads out a plurality of records registered in the log storage unit 44, and transmits these plurality of records to the display processing unit 24. Thus, these records are displayed on the display as logs. The display conditions are set by the user. For example, the display period is specified by the user. Alternatively, the display content is changed according to the user's scrolling operation.

[0053] The log analysis unit 38 analyzes the contents of the log. For example, the log analysis unit 38 has a statistical function, a search function, etc. The log analysis unit 38 can calculate the usage rate for each image analysis model, and can also determine the actions related to the actions of the image analysis model (such as approval or correction input). In addition, the log analysis unit 38 can also calculate the inspection time, processing speed, etc. The log analysis unit 38 involved in the embodiment has a report creation unit 42. This will be described later.

[0054] The model action management unit 40 functions as a detailed information storage unit. That is, the model action management unit 40 stores the ultrasonic image (specifically, a tomographic image) input to the image analysis model 30 as an analyzed image for each action of the image analysis model 30, and stores the analysis results (specifically, a plurality of scores) of the image analysis model 30. The analyzed image and the analysis results are basically information not included in the log, and are detailed information for model action verification.

[0055] More specifically, the model action management unit 40 generates a detailed record for each action of the image analysis model 30. The detailed record includes link information indicating the position of the analyzed image and a numerical value column as the analysis result.

[0056] A plurality of detailed records are stored in the detailed record storage unit 46. These constitute a detailed record column. When the log stored in the log storage unit 44 is represented as a main log or a general log, the detailed record column stored in the detailed record storage unit 46 can be called a sub-log or an image analysis model-specific log.

[0057] An analyzed image sequence consisting of a plurality of analyzed images is stored in the analyzed image storage unit 48. As described later, in the embodiment, each analyzed image is a low-resolution image generated from an ultrasonic image. The detailed record storage unit 46 and the analyzed image storage unit 48 may also be integrated. For example, the analyzed image may be included in each detailed record.

[0058] As described later, an ID number is issued for each action of the image analysis model to identify the action. Each model action record in the log contains an ID number, and each detailed record also contains an ID number. A specific detailed record is associated with each model action record by the ID number. In addition, the detailed record contains link information, and the analyzed image is associated with the detailed record by the link information.

[0059] When the user clicks a specific model action record in the displayed log, the report creation section 42 acquires a specific detailed record associated with the specific model action record, and acquires a specific analyzed image based on the link information included therein.

[0060] On this basis, the report creation unit 42 creates a model action report based on the information contained in the specific model action record, the information contained in the acquired detailed record, and the acquired analyzed image. The model action report is displayed on the display 26. As described later, the model action report includes the analyzed image as input information and the analysis result as output information. By referring to this information, the user evaluates the performance or status of the image analysis model.

[0061] Figure 2 FIG. 4 shows a configuration example of the model action management unit 40. Figure 2 In, with Figure 1 The same components as those shown in the figure are denoted by the same reference numerals. Figure 2 The content shown is equivalent to the algorithm of the model operation verification method involved in the embodiment.

[0062] The information processing unit 34 includes a log management unit 36, a log analysis unit 38, and a model action management unit 40. A log is constructed on the log storage unit 44. The model action management unit 40 has a counter 52, a detailed record storage unit 54, a conversion unit 62, and an image storage unit 64. The detailed record storage unit 54 and the image storage unit 64 function as a detailed information storage unit 25. The image analysis unit 28 has an image analysis model 30. When a tomographic image 55 is input to the image analysis model 30, the tomographic image is analyzed by the image analysis model 30, and an analysis result 56 is output from the image analysis model 30.

[0063] The counter 52 issues an AI processing request identifier (hereinafter referred to as an ID number) for each action of the image analysis model 30. When the image analysis model 30 performs repeated actions, the counter 52 performs an upward counting action, and the ID number increases sequentially. For example, the counter 52 can issue an ID number (reference symbol 58) according to a processing request from the main control unit, or the counter 52 can also issue an ID number (reference symbol 60) according to an input event of the tomographic image 55. The issued ID number is sent to the log management unit 36 ​​and the detailed record storage unit 54.

[0064] The log management unit 36 ​​includes the ID number in the model action record when creating the model action record. The detailed record storage unit 54 includes the ID number in the detailed record when creating the detailed record. The ID number issued by the counter 52 can be sent to the log management unit 36 ​​and the detailed record storage unit 54 via the image analysis unit 28. The analysis result of the image analysis model 30 is also sent to the detailed record storage unit 54.

[0065] The conversion unit 62 applies low-resolution conversion to the tomographic image 55, thereby generating a low-resolution image. As an example of a low-resolution image, a thumbnail image can be cited. Low-resolution conversion is a process of reducing the amount of information. As a reduction rate, 1 / 5, 1 / 10, or a ratio lower than 1 / 10 can be cited. Pixel thinning processing, compression processing, etc. can also be applied. From the perspective of verifying the image analysis model 30, the tomographic image 55 and the low-resolution image can both be referred to as analyzed images. Hereinafter, the low-resolution image will be referred to as the analyzed image.

[0066] The image storage unit 64 stores the analyzed image 66 in the analyzed image storage unit 48. The analyzed image storage unit 48 stores the analyzed images 66 in the number corresponding to the number of actions of the image analysis model 30. A storage period may be set for each analyzed image 66, and the analyzed image 66 may be deleted when the storage period has expired. Alternatively, all stored contents of the analyzed image 66 may be cleared in accordance with a clear instruction from the user. The image storage unit 64 notifies the detailed record storage unit 54 of information indicating the position of each analyzed image 66 as link information.

[0067] The detail record storage unit 54 creates a detail record including an ID number, link information, and analysis results, and stores it in the detail record storage unit 46. The detail record storage unit 46 stores a detail record column 47. The detail record column 47 is composed of a plurality of detail records 57.

[0068] When a specific model action record in the log is selected by the user, the report creation unit 42 in the log analysis unit 38 obtains a specific detailed record having the same ID number from the detailed record storage unit 46 based on the ID number included in the specific model action record, and obtains a specific analyzed image corresponding to the specific detailed record from the analyzed image storage unit 48 based on the link information included in the specific detailed record. In fact, the model action record contains a portion in which a hyperlink for specifying the location of the detailed record is embedded. By clicking on this portion, the specific detailed record is immediately obtained in the report creation unit 42.

[0069] As described above, the report creation unit 42 creates a model behavior report including information included in a specific model behavior record, information included in a specific detail record, and an analyzed image. The model behavior report is displayed on the display.

[0070] Figure 3 A detailed record column 47 is shown. The detailed record column 47 is composed of a plurality of detailed records 57 corresponding to a plurality of actions of the image analysis model. Each detailed record 57 includes an ID number 67, link information (URL) 68, an identifier 70 of the image analysis model, and an analysis result 72. The analysis result 72 is composed of a plurality of items 74 corresponding to a plurality of classes. Each item 74 is composed of a class identifier 76 and a score 78. The class identifier 76 indicates the type of tissue section to be identified. The score 78 is a confidence level. The content of the analysis result 72 may vary depending on the image analysis model used.

[0071] Figure 4 An example of a log is shown. For example, log 80 reflects the process of executing multiple steps that constitute the inspection protocol. Log 80 is composed of multiple records (rows) arranged in chronological order. Basically, one record is generated for one action (including events, state changes, etc.) of the ultrasonic diagnostic device. Log 80 contains multiple model action records.

[0072] For example, the model action record 82 includes execution time 84, ID number 86, image analysis model identifier 88, representative analysis result 90, information 92 indicating whether the analysis is successful or not, etc. In the part corresponding to the ID number (the part with underline and displayed in a specified color) 94, the URL of the detailed record is embedded as hyperlink information. By clicking on the part 94, the corresponding detailed record is instantly determined, and a model action report is created based on the detailed record, and the model action report is popped up and displayed. For example, the model action report is displayed overlapping on the log.

[0073] Among the displayed multiple records, each model action record can be easily identified by using an underscore or an ID number as a clue. In order to improve the recognition or visual recognition of each model action record, a special label can be given to each model action record.

[0074] Figure 5 The first example of the model operation report is shown. The model operation report 96 includes text information 98, analyzed images 100, and graphs 102. The text information 98 includes execution time 104, an identifier 106 of the image analysis model, a version 108 of the image analysis model, a subject ID 110, a type 124 of the tissue section to be observed, and a representative analysis result 126.

[0075] The analyzed image 100 is a low-resolution image. In the illustrated example, a still image is displayed, but a dynamic image may also be displayed. The chart 102 is created based on a plurality of scores (a plurality of confidences) constituting the analysis result. The plurality of scores are arranged in ascending order from left to right, and the size of each score is represented by the height of each bar. A plurality of classes (tissue cross-section categories) corresponding to the plurality of scores are displayed along the horizontal axis (reference symbol 128).

[0076] In the illustrated example, the category (class) of the tissue section to be observed is S8 (reference symbol 128a), and the corresponding score is relatively low (130a). The graph 102 indicates that the acquisition of the target tomographic image has failed (and has not been completed). As reasons for this, the quality of the image to be analyzed and the performance of the image analysis model can be cited. If the content of the tomographic image is inappropriate due to inappropriate position and posture of the probe, the score of the class of interest will naturally decrease. On the other hand, if the score of the class of interest is low when the content of the tomographic image is appropriate, it can be pointed out that the performance of the image analysis model may be insufficient.

[0077] In this way, by comparing the input information and output information contained in the model action report, the action of the image analysis model can be comprehensively verified. The content of the model action report can also be switched along the time axis. The model action report is displayed as needed. In other words, if you simply want to confirm a series of actions of the image analysis model, you can refer to the log itself.

[0078] Figure 6 The second example of the model action report is shown. For example, when using the image analysis model for detecting lesions, the display Figure 6 The model action report 132 is shown. The analyzed image 134 shows a frame 136 surrounding the detected lesion. The frame 136 is a graphic element displayed together with the tomographic image. By referring to the display coordinates and size of the frame 136, it is possible to verify whether the image analysis model has correctly detected the lesion. In this case, a dynamic image can also be displayed instead of a still image.

[0079] Figure 7 The third example of the model operation report is shown. For example, when using the image analysis model of the measured tissue, the display Figure 7 The model action report 138 is shown. The analyzed image 140 includes a tissue image 142 that is the object of measurement. Two focus positions in the tissue image 142 are automatically determined, and two markers 144 and 146 are displayed for the two focus positions. An auxiliary line 148 passing through them is also displayed. The distance between the two focus positions is automatically measured on the auxiliary line 148. The measurement result is displayed as a numerical value (reference symbol 150). For example, by comparing the positions of the tissue image 142 and the two markers 144 and 146, it is possible to verify whether the action of the image analysis model is appropriate.

[0080] Figure 8 The fourth example of the model action report is shown. The image analysis model is a class classification model. Seven scores corresponding to seven tissue sections from S2 to S8 are calculated. Each score represents the probability that the observed section corresponds to a specific tissue section.

[0081] The model action report 152 shown in the figure includes a plurality of information sets 154 to 160 (see t1 to t4) arranged in chronological order. That is, the model action report 152 corresponds to a plurality of model action records, in other words, it is created based on a plurality of detail records.

[0082] Each information set 154 to 160 is composed of an analyzed image 162 as input information and a chart 164 representing the analysis result as output information. Each chart 164 includes a plurality of bars representing a plurality of scores. The bars filled with dark colors are bars corresponding to the tissue sections (S8) to be observed. At t4, the analyzed image 162a is input to the image analysis model, and a good analysis result is obtained through its analysis. That is, in the chart 164a, a large value is calculated as the score corresponding to the tissue section (S8) to be observed. The height of the bar 168 representing the score is large, and the bar 168 is located in the front row. If such a model action report is displayed, the temporal changes in the action of the image analysis model can be intuitively and easily understood.

[0083] As described above, according to the above embodiment, the performance or status of the image analysis model can be verified in detail by referring to the detailed information (input information and output information) contained in the model action report. Ultrasonic images other than tomographic images can also be analyzed. The actions of multiple image analysis models that act in parallel or selectively can also be recorded.

Claims

1. An ultrasonic diagnostic device, characterized in that: include: An analysis unit having an image analysis model that has been machine-learned to analyze ultrasonic images; a log creation unit that creates a log consisting of a plurality of action records by recording a plurality of actions including the action of the image analysis model in chronological order; a storage unit that stores detailed information for model action verification for each action of the image analysis model; and The report creating unit creates a model operation report including detailed information associated with a specific model operation record when a specific model operation record in the log is selected.

2. The ultrasonic diagnostic apparatus according to claim 1, wherein: The storage unit stores the analyzed image as the detailed information for each operation of the image analysis model. The image to be analyzed is an ultrasonic image or an image equivalent thereto input to the image analysis model. The model action report includes the analyzed image.

3. The ultrasonic diagnostic apparatus according to claim 2, wherein: include: a conversion unit that converts the ultrasonic image input to the image analysis model into a low-resolution image, The analyzed image is the low-resolution image.

4. The ultrasonic diagnostic apparatus according to claim 1, wherein: The storage unit stores the analysis result of the image analysis model as the detailed information for each operation of the image analysis model. The model action report includes the analysis result.

5. The ultrasonic diagnostic apparatus according to claim 4, characterized in that: The analysis result includes multiple scores corresponding to multiple classes, the report creation unit creates a graph based on the plurality of scores, The model action report includes the chart.

6. The ultrasonic diagnostic apparatus according to claim 1, wherein: The storage unit includes: an image storage unit that stores the analyzed image for each action of the image analysis model; and a record storage unit that stores a detailed record including an analysis result of the image analysis model for each action of the image analysis model, The analyzed image is a low-resolution image generated by an ultrasonic image input to the image analysis model, The detailed information in the model action report includes the analyzed image as input information of the image analysis model and the analysis result as output information of the image analysis model.

7. The ultrasonic diagnostic apparatus according to claim 1, wherein: Each model action record included in the log includes a portion embedded with a hyperlink, the hyperlink identifying a location of the detailed information associated with each model action record, The model action report is displayed by selecting the section.

8. A model action verification method, characterized in that: Including the following processes: By recording in chronological order a plurality of actions including an action of an image analysis model that has been machine-learned to analyze an ultrasonic image, a log consisting of a plurality of action records is created; saving detailed information for model action verification for each action of the image analysis model; displaying the log; accepting a selection of a particular model action record in said displayed log; determining detailed information associated with the specific model action record; creating a model action report containing said detailed information; and Displays the model action report.

9. A storage medium storing a program for executing a model motion verification method in an ultrasonic diagnostic apparatus, characterized in that: The model action verification method comprises the following steps: By recording in chronological order a plurality of actions including an action of an image analysis model that has been machine-learned to analyze an ultrasonic image, a log consisting of a plurality of action records is created; saving detailed information for model action verification for each action of the image analysis model; displaying the log; accepting a selection of a particular model action record in said displayed log; determining detailed information associated with the specific model action record; creating a model action report containing said detailed information; and Displays the model action report.

Images