Display control program, display control method, and information processing device

Abstract

To improve the visibility of a scanning range from a gastric alveolus part to a cardiac base part when displaying a part detection map.SOLUTION: A display control program causes a computer to execute processing to: acquire a captured image in which multiple parts including a gastric alveolus part, a heart part, and a flow-out channel are scanned; detect the multiple parts at each scanning time; consider parts where a value of a detection rate exceeds a set value and which continue for a set number of frames or more as blocks of the target part and consider blocks that are closer than a certain number of frames to be one continuous block to detect blocks included in the captured image; determine a start position of the display area from a relative positional relation among a first block corresponding to the gastric alveolus part, a second block corresponding to the heart part, and a third block corresponding to the flow-out channel, so that the heart part is in the center of the display area; and display a part detection map associated with the time series of the scan according to the determined start position of the display area.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display control program, a display control method, and an information processing device. [Background technology]

[0002] Fetal cardiac screening is being carried out to enable prenatal diagnosis of congenital heart disease in fetuses. For example, abnormality detection technology using AI (Artificial Intelligence) has been proposed as a technology related to fetal cardiac screening.

[0003] Generally speaking, machine learning can only achieve recognition accuracy if sufficient normal and abnormal data is collected. However, the incidence of congenital heart disease in fetuses is low, making it difficult to collect sufficient abnormal data.

[0004] From this perspective, the anomaly detection technology described above uses object detection technology trained using normal data annotated with the names and locations of parts of the body, such as the heart and surrounding organs.

[0005] The abnormality detection technology compares the normal part of the heart that should be visible in the ultrasound image at the position where the probe is placed with the part of the heart detected in the ultrasound image using the object detection technology, and determines that there is an abnormality if there is any deviation from normal.

[0006] Another aspect is that viewing a moving image of a scanning cross section in an ultrasound examination requires time to play back, unlike a still image, which reduces the ease of viewing.

[0007] From this perspective, an inspection result display system has been proposed that displays a part detection map in which the detection results obtained by the above-mentioned object detection technology for each part, such as time-series data such as the presence or absence of a part and the confidence level, are associated.

[0008] This region detection map allows a list of the reliability of each region to be displayed, making it easier to check and explain the scanning cross sections that affect abnormality detection. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Patent Publication No. 2021-120820 [Patent Document 2] Japanese Patent Application Publication No. 2020-68797 [Patent Document 3] JP 2018-79000 A [Patent Document 4] Japanese Patent Publication No. 2020-28680 Summary of the Invention [Problem to be solved by the invention]

[0010] However, the above-mentioned site detection map includes a list of scanning sections other than the scanning range from the gastric alveolus to the cardiac base, which reduces the visibility of the scanning range from the gastric alveolus to the cardiac base.

[0011] In one aspect, the present invention aims to provide a display control program, a display control method, and an information processing device that can improve the visibility of the scanning range from the gastric alveolus to the cardiac base when displaying a region detection map. [Means for solving the problem]

[0012] A display control program according to one embodiment causes a computer to execute the following process: acquire an image in which multiple parts including the gastric alveoli, the heart, and the outflow tract are scanned; detect the multiple parts at each scanning time; consider parts where the detection rate exceeds a set value and which continue for a set number of frames or more as blocks of the target part; and consider blocks that are closer than a certain number of frames as one continuous block; detect blocks included in the image; determine the starting position of the display area based on the relative positional relationship between a first block corresponding to the gastric alveoli, a second block corresponding to the heart, and a third block corresponding to the outflow tract so that the heart is at the center of the display area; and display a part detection map associated with the time series of the scan according to the determined starting position of the display area. [Effects of the Invention]

[0013] According to one embodiment, the visibility of the scanning range from the gastric alveolus to the cardiac base can be improved when the region detection map is displayed. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram illustrating an example of the functional configuration of a server device. [Figure 2] FIG. 2 is a diagram showing an example of capturing an ultrasound image. [Figure 3] FIG. 3 is a diagram showing an example of generating a part detection map. [Figure 4] FIG. 4 is a diagram illustrating an example of the part detection map. [Figure 5] FIG. 5 is a diagram illustrating an example of the part detection map. [Figure 6] FIG. 6 is a schematic diagram showing an example of block detection. [Figure 7] FIG. 7 is a schematic diagram showing an example of detection of a heart block. [Figure 8] FIG. 8 is a schematic diagram showing an example of integration of the heart block. [Figure 9] FIG. 9 is a schematic diagram showing an example of detection of an outflow duct blockage. [Figure 10]FIG. 10 is a schematic diagram showing an example of excluding the heart block. [Figure 11] FIG. 11 is a schematic diagram showing an example of detection of the gastric alveolar block. [Figure 12] FIG. 12 is a diagram showing an example of association of gastric alveolar blocks. [Figure 13] FIG. 13 is a diagram showing an example of selection of display candidates. [Figure 14] FIG. 14 is a diagram illustrating an example of display control. [Figure 15] FIG. 15 is a diagram showing an example of a display on the client terminal. [Figure 16] FIG. 16 is a flowchart (1) showing the procedure of the display control process. [Figure 17] FIG. 17 is a flowchart (2) showing the procedure of the display control process. [Figure 18] FIG. 18 is a flowchart showing the procedure of the block detection process. [Figure 19] FIG. 19 illustrates an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the display control program, display control method, and information processing device according to the present application will be described with reference to the accompanying drawings. Each embodiment is merely an example, and the configuration, structure, functions, numerical values used to specify these, and usage scenarios are not limited by such examples. Each embodiment can be adaptively combined within the scope of not causing contradictions in the processing content.

[0016] Example 1 <Overall structure> Fig. 1 is a block diagram showing an example of the functional configuration of the server device 10. The server device 10 shown in Fig. 1 provides a display control function for displaying a part detection map in which the results of detection by the object detection technology described above for each part are associated with time-series data such as the presence or absence of the part and the confidence level.

[0017] The server device 10 is an example of an information processing device that provides the above-mentioned display control function. For example, the server device 10 can be realized as a SaaS (Software as a Service) type application to provide a display control service corresponding to the above-mentioned display control function as a cloud service. Alternatively, the server device 10 may be realized as a server that provides the above-mentioned display control function on-premise.

[0018] As shown in Fig. 1, the server device 10 can be communicatively connected to a client terminal 30 via a network NW. For example, the network NW may be any type of communication network, whether wired or wireless, such as the Internet or a local area network (LAN). Note that Fig. 1 shows an example in which one client terminal 30 is connected to one server device 10, but any number of client terminals 30 may be connected.

[0019] The client terminal 30 is a terminal device that receives the above-described display control function. This client terminal 30 can be used by examiners who perform fetal heart screening, such as obstetricians, ultrasound technicians, clinical laboratory technicians, diagnostic radiologists, nurses, and midwives. For example, the client terminal 30 may be realized by any computer, such as a personal computer, a smartphone, a tablet terminal, or a wearable terminal.

[0020] 1, the client terminal 30 may be connected to the inspection device 3. As just one example, the client terminal 30 and the inspection device 3 may be connected via HDMI (registered trademark) (High-Definition Multimedia Interface) or the like.

[0021] The inspection device 3 is a device that inspects the internal structure of an object to be inspected. As just one example, the inspection device 3 can be realized by an ultrasonic diagnostic device that applies a probe 3A that irradiates ultrasonic waves to the surface of the object to image the internal structure of the object based on ultrasonic waves reflected from the inside of the object.

[0022] Note that, although an example in which the client terminal 30 and the inspection device 3 are connected via an HDMI (registered trademark) terminal is given here, the present invention is not limited to this, and the server device 10 and the inspection device 3 may be connected so as to be able to communicate with each other. In this case, the server device 10 and the inspection device 3 may be connected by a communication standard such as DICOM (Digital Imaging and Communications in Medicine).

[0023] <Example of ultrasound image> In fetal echocardiography, ultrasound images are used that are generated in time series for each scanning cross section in accordance with the scanning direction in which the probe 3A scans the surface of the object to be examined.

[0024] FIG. 2 is a diagram showing an example of capturing an ultrasound image. FIG. 2 shows an example in which the heart of a fetus 20 inside a mother 2 is the subject of examination. As shown in FIG. 2, an ultrasound image is acquired by scanning the abdominal surface of the mother 2 with a probe 3A. Such scanning with the probe 3A is merely an example and is performed in accordance with fetal echocardiography guidelines. For example, the examiner continuously scans from near the stomach of the fetus 20 to above the heart. In this case, ultrasound images 210 to 230 are acquired as merely an example of ultrasound images.

[0025] For example, ultrasound image 210 includes ribs 211, spine 212, descending aorta 213, gastric vesicle 214, and umbilical vein 215. Ultrasound image 220 includes ribs 211, spine 212, descending aorta 213, and heart 221. Furthermore, ultrasound image 220 includes, as internal structural parts of heart 221, right ventricle 222, right atrium 223, left ventricle 224, left atrium 225, interventricular septum 226, and cardiac cross 227. Ultrasound image 230 includes ribs 211, spine 212, descending aorta 213, pulmonary artery 231, ascending aorta 232, and superior vena cava 233.

[0026] For ease of explanation, the ultrasound images 210 to 230 in FIG. 2 are drawn to improve the visibility of each part, but in actual ultrasound images, each part is not necessarily displayed as clearly as in the example shown in FIG.

[0027] <Example of generating a body part detection map> As just one example, the generation of the above-mentioned part detection map can be triggered by a specific operation being performed on the client terminal 30. In this case, the ultrasound image is uploaded to the server device 10 under the following operational logic of the client terminal 30 and the examination device 3.

[0028] As an example, the inspection device 3 transmits ultrasound images generated frame by frame in real time to the client terminal 30. Meanwhile, the client terminal 30 stores the ultrasound images transmitted from the inspection device 3, but deletes the ultrasound images after a specified time has elapsed since they were transmitted to the client terminal 30. In other words, the client terminal 30 stores ultrasound images from the most recent frame to the frame immediately preceding the specified time.

[0029] Under the operating logic of the client terminal 30 and the inspection device 3, the client terminal 30 accepts a specific operation, i.e., an operation to generate a part detection map, at the timing of starting scanning or the timing of ending scanning. For example, the specific operation may be the operation of a specific key included in a keyboard. Alternatively, it may be an eyes-free or hands-free operation. For example, it may be the operation of pressing a foot switch or the like to which the above-mentioned specific key is assigned.

[0030] Here, as an example, a case where an operation to generate a part detection map is accepted both at the timing of starting scanning and the timing of ending scanning will be taken as an example. In this case, the client terminal 30 uploads to the server device 10 ultrasound images for the operation section in which the two specific operations were accepted.

[0031] Fig. 3 is a diagram showing an example of generating a part detection map. As shown in Fig. 3, the server device 10 to which ultrasound images have been uploaded from the client terminal 30 inputs the ultrasound images for the operation section frame by frame into a machine learning model realized by the above-mentioned object detection technology. Hereinafter, the machine learning model realized by the above-mentioned object detection technology may be referred to as an "object detection model."

[0032] This object detection model takes an image as input and outputs the detection results for each part, which may include the class name of the part, the bounding box in which the part is detected on the image, and the confidence level.

[0033] As just one example, server device 10 can generate part detection map 40 by mapping the presence or absence of each part in chronological order. This part detection map 40 has the aspect of a two-dimensional map in which the presence or absence of detection of each part is mapped on a two-dimensional plane with multiple types of parts as the first axis and time series as the second axis. For example, in the example of Figure 3, for each bar corresponding to a part, hatching is drawn at the position of the frame where that part is detected, while a white background is drawn at the position of the frame where that part is not detected.

[0034] Such a part detection map 40 is sometimes called a barcode-like timeline, or simply a timeline. Note that, although Fig. 3 shows an example of part detection map 40 in which hatching is drawn depending on whether or not each part is present, it is also possible to draw by making the color of the bar darker as the certainty of each part increases.

[0035] The part detection map 40 generated in this manner is displayed on the client terminal 30 as reference information for anomaly detection, from the perspective of simplifying the confirmation and explanation of the scanning cross sections affected by anomalies detected using the above-mentioned anomaly detection technology.

[0036] <One aspect of the issue> As explained in the background art section above, in the examination result display system relating to the prior art, scanning sections other than the scanning range from the gastric alveolus to the cardiac base are included in the list display of the site detection map, which has the side effect of reducing the visibility of the scanning range from the gastric alveolus to the cardiac base.

[0037] Specifically, the fetal heart is small and has a high heart rate, making it difficult to capture images. Therefore, even if a specific key on the client terminal 30 or a foot switch or the like to which the corresponding key is assigned is pressed at the timing to start scanning, it is not necessarily possible to start capturing images of the gastric alveolus from the start position of the frame on the region detection map. Furthermore, even if a specific key on the client terminal 30 or a foot switch or the like to which the corresponding key is assigned is pressed at the timing to end scanning, it is not necessarily possible to finish capturing images of the cardiac base by the end position of the frame on the region detection map.

[0038] Fig. 4 is a diagram showing an example of an area detection map. Fig. 4 shows an area detection map 41 generated by the examination result display system according to the above-mentioned conventional technology. As shown in Fig. 4, area detection map 41 shows that the imaging of the observation planes proposed in fetal echocardiography guidelines, i.e., a series of observation planes of the gastric alveolus, heart, and outflow tract, begins from frame position L1. However, before frame position L1, area detection map 41 also includes frames in which the examiner is searching for the gastric alveolus.

[0039] As described above, since it is difficult to capture ultrasound images of the fetal heart, it is difficult to avoid a situation in which scanning sections other than the observation sections proposed in fetal echocardiography guidelines, i.e., the series of observation sections of the gastric alveolus, heart, and outflow tract, are included in the list display of the site detection map.

[0040] <One aspect of the problem-solving approach> Therefore, as part of the display control function of this embodiment, blocks of areas where the number of frames in which the detection rate of the area exceeds a threshold consecutively is detected from a series of images obtained by probe scanning, and the scanning range of the series of blocks of the gastric alveolus, heart, and outflow tract is displayed on a part detection map.

[0041] The "detection rate" here refers to the ratio of the parts detected by the object detection model to the total number of parts that make up the set of parts, which is taken as the population. The total number of parts in the population may be the total number per frame, or, for example, the total number per 10 frames including the frame before and after the frame in question, for purposes such as increasing robustness.

[0042] FIG. 5 is a diagram illustrating an example of a region detection map. FIG. 5 illustrates a region detection map 42 generated by the display control function according to this embodiment. As shown in FIG. 5, in the region detection map 42, a block in which the detection rates and the number of consecutive frames for the "gastric alveolus," "umbilical vein," and "inferior vena cava" exceed thresholds is detected as a gastric alveolus block B1. Furthermore, a block in which the detection rates and the number of consecutive frames for the "cardiac crossing," "ventricular septum," "right atrium," "tricuspid valve," "mitral valve," "left ventricle," "pulmonary vein," "right ventricle," and "left atrium" exceed thresholds is detected as a cardiac block B2. Furthermore, a block in which the detection rates and the number of consecutive frames for the "ascending aorta," "pulmonary artery," "ductus arteriosus," and "superior vena cava" exceed thresholds is detected as an outflow tract block B3. The scanning ranges of this series of gastric alveolus block B1, cardiac block B2, and outflow tract block B3 are displayed on the region detection map 42.

[0043] Therefore, it is possible to prevent the inclusion of scanning sections outside the scanning range from the gastric alveolus to the cardiac base, ie, the observation sections proposed in fetal echocardiography guidelines, in the list display of the region detection map 42.

[0044] Therefore, the display control function according to this embodiment can improve the visibility of the scanning range from the gastric alveolus to the cardiac base when the region detection map is displayed.

[0045] <Configuration of Server Device 10> Next, an example of the functional configuration of the server device 10 according to this embodiment will be described. Fig. 1 shows a block diagram related to the display control function of the server device 10. As shown in Fig. 1, the server device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that Fig. 1 only shows a selection of functional units related to the above-mentioned display control function, and the server device 10 may also be provided with functional units other than those shown.

[0046] The communication control unit 11 is a functional unit that controls communication with other devices such as the client terminal 30. As just one example, the communication control unit 11 can be realized by a network interface card such as a LAN card. In one aspect, the communication control unit 11 receives a request to generate a part detection map from the client terminal 30, or outputs the part detection map or the like to the client terminal 30.

[0047] The storage unit 13 is a functional unit that stores various types of data. As just one example, the storage unit 13 is realized by internal, external, or auxiliary storage of the server device 10. For example, the storage unit 13 stores a first part detection map 13A and a second part detection map 13B. The first part detection map 13A and the second part detection map 13B will be described together when reference, generation, or registration is performed.

[0048] The control unit 15 is a functional unit that performs overall control of the server device 10. For example, the control unit 15 can be realized by a hardware processor. As shown in FIG. 1, the control unit 15 has an acquisition unit 15A, a detection unit 15B, a detection unit 15C, a determination unit 15D, and a display control unit 15E. Note that the control unit 15 may also be realized by hardwired logic or the like.

[0049] The acquisition unit 15A is a processing unit that acquires images. As just one example, when a specific operation, i.e., an operation to generate a part detection map, is accepted by the client terminal 30, the acquisition unit 15A acquires a captured video including multiple frames of ultrasound images.

[0050] The following is merely an example of a case where an operation to generate a part detection map is accepted at the timing of the end of scanning. In this way, when client terminal 30 accepts an operation to generate a part detection map, such as pressing a specific key or a foot switch to which that key is assigned, it outputs a request to generate a part detection map to server device 10. This request to generate a part detection map includes ultrasound images of the scanning range for a specified time, for example, 15 seconds, going back from the time when client terminal 30 accepted the operation to generate the part detection map, and these images are uploaded to server device 10. The ultrasound images of the scanning range for the specified time uploaded in this way are acquired by acquisition unit 15A.

[0051] Here, the above-mentioned specified time is merely an example, and can be set to a period of time, for example 15 seconds, where there is a probability that gastric vesicles will be included that is greater than or equal to the specified value based on statistical values of the scanning speed of probe 3A, such as the mean value, median, mode, variance, etc.

[0052] By setting this specified time, the scanned cross sections of the gastric alveolus, cardiac region, and outflow tract can be included in the region detection map generation target. Furthermore, by accepting region detection map generation operations only at the timing of the end of scanning, the number of operations can be reduced.

[0053] The detector 15B is a processing unit that detects regions from images. As an example, the detector 15B inputs, for each frame, ultrasound images of the scanning range acquired by the acquirer 15A for a specified time into the object detection model. As a result, the object detection model outputs, for each region, a detection result including the region's class name, a bounding box in which the region is detected on the image, and a confidence level for each region, for each region. The detector 15B then generates a provisional region detection map as a first region detection map by mapping the presence or absence of each region in chronological order. This first region detection map is generated as a "provisional" region detection map because it allows scanning cross sections outside the scanning range from the gastric alveolus to the cardiac base to be included in the list display of the region detection map. The provisional region detection map generated in this manner is stored in the memory unit 13 as the first region detection map 13A.

[0054] Detector 15C is a processing unit that detects blocks. As just one example, detector 15C detects blocks of parts where the part detection rate exceeds threshold value Th1 and the number of consecutive frames in which the part detection rate exceeds threshold value Th1 exceeds threshold value Th2 from the first part detection map generated by detector 15B.

[0055] Hereinafter, the collection of "gastric alveoli," "umbilical vein," and "inferior vena cava" may be referred to as "gastric alveolar region." Furthermore, a block of the "gastric alveolar region" where the region detection rate exceeds threshold Th1 and the number of consecutive frames exceeds threshold Th2 may be referred to as "gastric alveolar region block."

[0056] The term "heart region" may refer to the collection of "cardiac crossing," "ventricular septum," "right atrium," "tricuspid valve," "mitral valve," "left ventricle," "pulmonary vein," "right ventricle," and "left atrium." Furthermore, the term "heart region block" may refer to a block of the "heart region" where the region detection rate exceeds threshold Th1 and the number of consecutive frames exceeds threshold Th2.

[0057] The term "outflow tract" may refer to the collection of the "ascending aorta," "pulmonary artery," "ductus arteriosus," and "superior vena cava." Furthermore, the term "outflow tract block" may refer to a block of the "outflow tract" where the detection rate of the area exceeds a threshold Th1 and the number of consecutive frames exceeds a threshold Th2.

[0058] Furthermore, among the judgment logics used for block detection, judgment logic in which the part detection rate exceeds threshold Th1 may be referred to as "judgment logic 1," and judgment logic in which the number of consecutive frames exceeds threshold Th2 may be referred to as "judgment logic 2."

[0059] The detection unit 15C can perform block detection to detect "gastric alveolar block," "cardiac block," and "outflow tract block" from the first region detection map generated by the detection unit 15B.

[0060] Fig. 6 is a schematic diagram showing an example of block detection. Fig. 6 shows a first part detection map 60 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each part, for example, the presence or absence of a part, are mapped in chronological order. Furthermore, Fig. 6 shows parts of first part detection map 60 that are considered to be blocks with thick solid lines, while parts that are not considered to be blocks are shown with thick dashed lines.

[0061] As shown in FIG. 6, alveolar region blocks B1-B3 are detected from the first region detection map 60, cardiac region blocks B4-B7 are detected, and an outflow tract block B8 is detected. For example, in the case of cardiac block B7, among the total number T of regions belonging to the "heart region," including "cardiac cross," "ventricular septum," "right atrium," "tricuspid valve," "mitral valve," "left ventricle," "pulmonary vein," "right ventricle," and "left atrium," the detection rate of the region detected by the object detection model exceeds a threshold value Th1, e.g., 70%. Furthermore, the number of consecutive frames F in which the region detection rate exceeds the threshold value exceeds a threshold value Th2, e.g., 10 frames. A region satisfying these determination logics 1 and 2 is considered a block, and is therefore detected as cardiac region block B7. Meanwhile, in the case of scanning range b1, the detection rate exceeds the threshold value Th1, e.g., 70%, but the number of consecutive frames in which the region detection rate exceeds the threshold value does not exceed a threshold value Th2, e.g., 10 frames. Thus, while determination logic 1 is satisfied, determination logic 2 is not satisfied, and therefore scanning range b1 is not considered a block.

[0062] Next, we will explain in more detail the block detection of "alveolar block," "heart block," and "outflow tract block." The detection of these "alveolar block," "heart block," and "outflow tract block" can be performed using common logic except for the threshold Th1 used for comparison with the detection rate of the region, so we will explain the detection of heart block as a representative example.

[0063] More specifically, detection unit 15C repeats the following process for the number K of frames included in the first part detection map, i.e., the number of frames included in the scanning range for the specified time. That is, detection unit 15C calculates the detection rate of the heart part in the kth frame. Next, detection unit 15C determines whether the detection rate of the heart part in the kth frame exceeds threshold value Th1.

[0064] Such threshold value Th1 can be changed individually because the number of regions differs between the "alveolar region block," "heart region block," and "outflow tract block." For example, the threshold value Th1 used to detect the "alveolar region block" can be set to "0%." In this case, if even one region is detected, the sensitivity can be set to satisfy judgment logic 1. Furthermore, the threshold value Th1 used to detect the "heart region block" can be set to "70%." Furthermore, the threshold value Th1 used to detect the "outflow tract block" can be set to "50%."

[0065] If the detection rate of the heart region in the k-th frame exceeds the threshold value Th1, the above-mentioned judgment logic 1 is satisfied. In this case, the detection unit 15C increments the number of consecutive frames f in which the detection rate exceeds the threshold value Th1. Note that if the detection rate of the heart region in the k-th frame does not exceed the threshold value Th1, the number of consecutive frames f is reset to an initial value, for example, "0."

[0066] Thereafter, the detection unit 15C determines whether the number of consecutive frames f exceeds the threshold value Th2. If the number of consecutive frames f exceeds the threshold value Th2, both the above-mentioned determination logic 1 and the above-mentioned determination logic 2 are satisfied. Furthermore, the detection unit 15C further determines whether the k-th frame is a new heart block, i.e., a leading frame that satisfies both the above-mentioned determination logic 1 and the above-mentioned determination logic 2.

[0067] If the kth frame is a new heart block, the detection unit 15C increments the block number i that identifies the heart block. The detection unit 15C then assigns the heart block number i to consecutive frames including the kth frame, i.e., frames whose detection rate exceeds the threshold value Th1 consecutively up to the kth frame. If the kth frame is not a new heart block, the detection unit 15C assigns the heart block number i to the kth frame without incrementing it.

[0068] Thereafter, the detection unit 15C determines whether or not the distance d, for example, the shortest distance, between the heart block with the i-th block number and the heart block with the (i-1)-th block number is less than a threshold value Th3.

[0069] Then, if the distance between the heart block of the i-th block number and the i-1-th block number is less than threshold Th3, detection unit 15C integrates the heart block of the i-th block number and the heart block of the i-1-th block number.

[0070] This series of processes is repeated over K frames, thereby making it possible to detect cardiac blocks from the first region detection map.

[0071] Here, the detection unit 15C does not necessarily have to detect the three types of blocks, "gastric alveolar block," "cardiac block," and "outflow tract block," from the start of the process.

[0072] As just one example, the detection unit 15C starts execution by detecting a "heart block." This is because, among the heart and outflow tract, which are highly important observation cross sections proposed in fetal echocardiography guidelines, etc., larger heart blocks can be detected more reliably, and if a heart block cannot be detected, there is a high possibility that other blocks will also not be detected. For this reason, by suspending subsequent processing when a "heart block" is not detected, it is possible to prevent resources from being allocated to unnecessary processing.

[0073] Fig. 7 is a schematic diagram showing an example of cardiac block detection. Fig. 8 is a schematic diagram showing an example of cardiac block integration. Figs. 7 and 8 show a first region detection map 60 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each region, for example, the presence or absence of a region, are mapped in chronological order. Furthermore, in Figs. 7 and 8, areas in first region detection map 60 that are considered to be cardiac blocks are indicated by thick solid lines, while areas that are not considered to be cardiac blocks are indicated by thick dashed lines.

[0074] As shown in FIG. 7, four heart blocks, heart blocks B11 to B14, are detected according to the first region detection map 60. Here, as shown in FIG. 8, the distance d1 between heart block B11 and heart block B12 is not less than threshold Th3. Therefore, heart block B11 and heart block B12 are not integrated. Next, the distance d2 between heart block B12 and heart block B13 is also not less than threshold Th3. Therefore, heart block B12 and heart block B13 are not integrated either. Then, the distance d3 between heart block B13 and heart block B14 is less than threshold Th3. Therefore, the two heart blocks, heart block B13 and heart block B14, are integrated into one heart block B13. This prevents the image from being divided into multiple blocks due to, for example, a frame in which the detection rate momentarily drops due to an acoustic imprint caused by ultrasound's spinal reflection.

[0075] When one or more cardiac blocks are detected in this manner, the detection unit 15C performs detection of an "outflow tract block." FIG. 9 is a schematic diagram illustrating an example of outflow tract block detection. FIG. 9 shows a first region detection map 60 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each region, such as the presence or absence of a region, are mapped in chronological order. Furthermore, in FIG. 9, areas in the first region detection map 60 that are considered to be cardiac blocks or outflow tract blocks are indicated by thick solid lines, while areas that are not considered to be cardiac blocks or outflow tract blocks are indicated by thick dashed lines. As shown in FIG. 9, an outflow tract block B21 is detected according to the first region detection map 60.

[0076] Returning to the explanation of Figure 1, determination unit 15D is a processing unit that determines the scanning range to be displayed on the second part detection map of the first part detection map. The "second part detection map" here may refer to the part detection map to be the final output, in contrast to the provisional first part detection map.

[0077] As an example, for each cardiac block detected by the detection unit 15C, the determination unit 15D associates the outflow tract block that follows the cardiac block. Furthermore, the determination unit 15D associates the cardiac block that has been successfully associated with the outflow tract block with the alveolar block that precedes the cardiac block. After that, if the determination unit 15D has successfully associated the series of blocks, including the outflow tract block, cardiac block, and alveolar block, it determines the scanning range of the series of blocks, including the outflow tract block, cardiac block, and alveolar block, as the display range of the second region map.

[0078] More specifically, the determination unit 15D repeats the following process for the number M of cardiac blocks detected by the detection unit 15C. Furthermore, the determination unit 15D repeats the following process for the m-th cardiac block for the number N of outflow tract blocks detected by the detection unit 15C.

[0079] That is, the determination unit 15D determines whether the position of the nth outflow tract block is behind the position of the mth cardiac block. If the position of the nth outflow tract block is behind the position of the mth cardiac block, it can be determined that the unidirectional slide scanning of the probe 3A has been successful. In this case, the determination unit 15D calculates the distance between the mth cardiac block and the nth outflow tract block, for example, the horizontal distance between the centers of gravity of the two types of blocks.

[0080] By repeating this calculation of distance, it is possible to calculate the distance between the outflow tract block subsequent to the m-th cardiac block among the N outflow tract blocks and the m-th cardiac block.

[0081] The determination unit 15D then associates the outflow tract block that is the shortest distance from the mth heart block among the outflow tract blocks subsequent to the mth heart block with the mth heart block.The determination unit 15D then extracts pairs in which the mth heart block and the outflow tract block with the shortest distance are associated.The association of the mth heart block and the outflow tract block with the shortest distance can also be narrowed down to cases in which the shortest distance is less than a threshold value Th4.

[0082] By repeating this association of cardiac blocks and outflow tract blocks, it is possible to associate each of the M cardiac blocks with its subsequent adjacent outflow tract block. At this time, cardiac blocks that do not have a subsequent adjacent outflow tract block among the M cardiac blocks can be identified as not including the outflow tract in the observation cross-sections proposed in fetal echocardiography guidelines, etc., and are therefore excluded from the display candidates for the second region detection map.

[0083] Fig. 10 is a schematic diagram showing an example of cardiac block exclusion. Fig. 10 shows a first region detection map 60 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each region, for example, the presence or absence of a region, are mapped in chronological order. Furthermore, Fig. 10 shows blocks in the first region detection map 60 for which association between cardiac blocks and outflow tract blocks was successful indicated by thick solid lines, while cardiac blocks or outflow tract blocks for which association between cardiac blocks and outflow tract blocks was unsuccessful indicated by thick dashed lines.

[0084] 10, according to the first part detection map 60, of the three heart blocks B11 to B13, heart block B13 is successfully associated with the adjacent outflow tract block B21, so the pair of heart block B13 and outflow tract block B21 is extracted as a display candidate for the second part detection map. On the other hand, heart block B11 and heart block B12 fail to be associated with the adjacent outflow tract block, so they are excluded from the display candidate for the second part detection map.

[0085] As a result, if there is at least one pair of a cardiac block and an outflow tract block that has been successfully associated, the determining unit 15D causes the detecting unit 15C to detect an "alveolar block."

[0086] Fig. 11 is a schematic diagram showing an example of detection of alveolar region blocks. Fig. 11 shows a first region detection map 60 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each region, for example, the presence or absence of a region, are mapped in chronological order. Furthermore, Fig. 11 shows thick solid lines in the first region detection map 60 for regions that are considered to be alveolar region blocks, cardiac blocks, or outflow tract blocks, while thick dashed lines show regions that are not considered to be alveolar region blocks, cardiac blocks, or outflow tract blocks. As shown in Fig. 11, three alveolar region blocks, B31 to B33, are detected according to the first region detection map 60.

[0087] Thereafter, the determination unit 15D repeats the following process for the number P of pairs of heart block and outflow tract block that have been successfully associated. Furthermore, the determination unit 15D repeats the following process for the number Q of gastric alveolar blocks detected by the detection unit 15C, targeting the p-th pair of heart block.

[0088] That is, the determination unit 15D determines whether the position of the qth alveolar portion block is before the position of the heart block of the pth pair. At this time, if the position of the qth alveolar portion block is before the position of the heart block of the pth pair, it can be determined that the unidirectional slide scanning of the probe 3A has been successful. In this case, the determination unit 15D calculates the distance between the heart block of the pth pair and the qth alveolar portion block, for example, the horizontal distance between the centers of gravity of the two types of blocks.

[0089] By repeating this calculation of distance, the distance between the alveolar block preceding the heart block of the pth pair among the Q alveolar blocks and the heart block of the pth pair can be calculated.

[0090] Then, the determination unit 15D associates the alveolar portion block that is the shortest distance from the heart block of the pth pair among the outflow tract blocks preceding the heart block of the pth pair with the heart block of the pth pair.The determination unit 15D then extracts three block combinations in which the pth pair and the alveolar portion block that is the shortest distance from the heart block of the pth pair are associated.Note that the association of the pth pair and the alveolar portion block with the shortest distance can also be narrowed down to when the shortest distance is less than a threshold value Th4.

[0091] By repeating this association of cardiac blocks and alveolar blocks, it is possible to associate the preceding adjacent alveolar block for each of the P pairs. At this time, it is possible to identify, among the P pairs, pairs that do not have a preceding adjacent alveolar block, as pairs that do not include the alveolar region in the observation cross-sections proposed in fetal echocardiography guidelines, etc., and therefore these pairs are excluded from the display candidates for the second region detection map.

[0092] Fig. 12 is a diagram showing an example of association of alveolar portion blocks. Fig. 12 shows a first region detection map 60 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each region, for example, the presence or absence of a region, are mapped in chronological order. Furthermore, Fig. 12 shows, in first region detection map 60, each block in which a pair of a heart region block and an outflow tract block has been successfully associated with an alveolar portion block, indicated by a thick solid line. On the other hand, Fig. 12 shows, in first region detection map 60, each block in which a pair of a heart region block and an outflow tract block has not been successfully associated with an alveolar portion block, indicated by a thick dashed line.

[0093] As shown in FIG. 12, according to the first region detection map 60, of the three alveolar region blocks B31-B33, the alveolar region block that precedes and is adjacent to the heart region block B13 is the alveolar region block B33. Therefore, the pair of the heart region block B13 and the outflow tract block B21 is successfully associated with the alveolar region block B33. As a result, the combination of the three blocks, the alveolar region block B33, the heart region block B13, and the outflow tract block B21, is extracted as a display candidate for the second region detection map. On the other hand, although the alveolar region blocks B31 and B32 precede the heart region block B13, they are not adjacent to it, so their association with the pair of the heart region block B13 and the outflow tract block B21 fails.

[0094] Then, when a plurality of combinations of three blocks are extracted as display target candidates for the second part detection map, the determining unit 15D accepts, via the client terminal 30, a selection of a display target candidate from among the plurality of display target candidates.

[0095] Fig. 13 is a diagram showing an example of selection of display candidates. Fig. 13 shows a first region detection map 61 in which detection results corresponding to ultrasound images of a scanning range for a specified time for each region, for example, the presence or absence of the region, are mapped in chronological order. As shown in Fig. 13, first region detection map 61 includes display candidate C1 and display candidate C2 as a combination of three blocks: an alveolar region block, a cardiac region block, and an outflow tract block. In this case, selection of one display candidate from display candidate C1 and display candidate C2 can be accepted.

[0096] Thereafter, determination unit 15D determines the display settings of the second part detection map based on the relative positions of the three blocks extracted as display target candidates. The "display settings" here may include the setting of the display range of the second part detection map, and the display positions and display sizes of each of the three blocks extracted as display target candidates.

[0097] As an example, determination unit 15D can match the scanning range of a series of blocks, including alveolar region blocks, cardiac region blocks, and outflow tract blocks, extracted as display candidate blocks with the display range of the second region detection map. In this case, if the number of frames corresponding to the scanning range of the display candidate block is smaller than the number of frames set as the display range of the second region detection map, the display size of each of the alveolar region blocks, cardiac region blocks, and outflow tract blocks is enlarged according to the ratio of the number of frames between the two.

[0098] FIG. 14 is a diagram illustrating an example of display control. As shown in FIG. 14, the determination unit 15D can set the display position of the first frame of the gastric alveolar region block extracted as a display candidate to the position of the frame where display begins in the second region detection map. Furthermore, the determination unit 15D can set the display position of the center of the heart region block extracted as a display candidate to the center position L13 of the second region detection map. In this case, the center position L15 of the heart region block is shifted leftward by the distance D1. Accordingly, the display position of the first frame of the heart region block is also shifted leftward by the distance D1 from the original position L14, and the display position of the first frame of the heart region block is set to position L12.

[0099] In this way, the display position and display size of each of the gastric alveolar block, cardiac block, and outflow tract block extracted as display target candidates are determined as display settings for the second region detection map. Then, determination unit 15D generates a second region detection map to be the final output in accordance with the display settings for the second region detection map. The second region detection map generated in this way may be stored in storage unit 13 as second region detection map 13B.

[0100] 14 shows an example in which the display position L12 of the first frame of the heart region block is to the right of a set value, for example, position L11, one-third of the way from the frame where display begins on the second region detection map, but if the display position L12 of the first frame of the heart region block is to the left of position L11, one-third of the way from the display begin position of the second region detection map, the shift of the first frame of the heart region block can also be limited to position L11, one-third of the way from the display begin position of the second region detection map. This makes it possible to secure a display area for the gastric alveolar region block.

[0101] Returning to the explanation of FIG. 1, display control unit 15E is a processing unit that executes display control for client terminal 30. As one aspect, display control unit 15E can cause client terminal 30 to display a second region detection map generated by determination unit 15D. FIG. 15 is a diagram showing an example of display on client terminal 30. As shown in FIG. 15, second region detection map 70 allows the list display to include the observation cross sections proposed in fetal echocardiography guidelines, i.e., the scanning ranges of the gastric alveolar block, cardiac block, and outflow tract block, and excludes other scanning cross sections from the list display.

[0102] Note that, although an example in which the second part detection map is displayed on the client terminal 30 has been given here, the output destination of the second part detection map is not limited to the client terminal 30. For example, the second part detection map may be output to an application or service that operates on the backend of the above-mentioned display control function.

[0103] <Processing flow> Next, a flow of processing by the server device 10 according to the present embodiment will be described. Here, (1) display control processing executed by the server device 10 will be described first, followed by (2) block detection processing.

[0104] (1) Display control processing 16 and 17 are flowcharts (1) and (2) showing the procedure of the display control process. This process can be started when a specific operation, i.e., an operation to generate a part detection map, is received by the client terminal 30, as an example.

[0105] As shown in FIG. 16, the acquisition unit 15A acquires ultrasound images of the scanning range for a specified time, for example, 15 seconds, going back from the time when the client terminal 30 accepted the operation to generate the part detection map (step S101).

[0106] Next, the detection unit 15B inputs the ultrasound images of the scanning range for a specified time acquired by the acquisition unit 15A into the object detection model for each frame, and generates a provisional part detection map as a first part detection map by mapping the presence or absence of each part in chronological order based on the detection results for each frame at each part (step S102).

[0107] Then, the detection unit 15C detects cardiac blocks from the first region detection map generated in step S102 (step S103). At this time, on the condition that one or more cardiac blocks are detected, the detection unit 15C detects outflow tract blocks from the first region detection map generated in step S102 (step S104).

[0108] Next, the determination unit 15D executes loop process 1, which repeats the processes from step S105 to step S108 described below, the number of times corresponding to M of the cardiac blocks detected in step S103. Furthermore, the determination unit 15D executes loop process 2, which repeats the processes from step S105 to step S106 described below, the number of times corresponding to N of the outflow tract blocks detected in step S104, for the m-th cardiac block.

[0109] That is, the determination unit 15D determines whether the position of the nth outflow tract block is behind the position of the mth cardiac block (step S105). At this time, if the position of the nth outflow tract block is behind the position of the mth cardiac block (step S105 Yes), it can be determined that the unidirectional slide scanning of the probe 3A has been successful. In this case, the determination unit 15D calculates the distance between the mth cardiac block and the nth outflow tract block, for example, the horizontal distance between the centers of gravity of the two types of blocks (step S106).

[0110] By repeating this loop process 2, it is possible to calculate the distance between the outflow tract block subsequent to the m-th cardiac block among the N outflow tract blocks and the m-th cardiac block.

[0111] Then, the determination unit 15D associates the outflow tract block that is the shortest distance from the m-th cardiac block among the outflow tract blocks subsequent to the m-th cardiac block with the m-th cardiac block (step S107).Then, the determination unit 15D extracts a pair in which the m-th cardiac block and the outflow tract block that is the shortest distance are associated with each other (step S108).

[0112] By repeating this loop process 1, it is possible to associate each of the M cardiac blocks with its subsequent adjacent outflow tract block. At this time, among the M cardiac blocks, cardiac blocks that do not have a subsequent adjacent outflow tract block can be identified as not including the outflow tract in the observation cross-sections proposed in fetal echocardiography guidelines, etc., and are therefore excluded from the display candidates for the second region detection map.

[0113] 17, if there is at least one pair of a cardiac block and an outflow tract block that has been successfully associated (step S201), the determination unit 15D executes the following process: That is, the determination unit 15D causes the detection unit 15C to detect an alveolar block from the first region detection map generated in step S102 (step S202).

[0114] Thereafter, the determination unit 15D executes loop process 3, which repeats the processes from step S203 to step S206 described below, the number of times corresponding to the number P of pairs of cardiac blocks and outflow tract blocks that have been successfully associated. Furthermore, the determination unit 15D executes loop process 4, which repeats the processes from step S203 to step S204 described below, the number of times corresponding to the number Q of alveolar blocks detected in step S202, for the cardiac blocks of the p-th pair.

[0115] That is, the determination unit 15D determines whether the position of the qth alveolar block is before the position of the heart block of the pth pair (step S203). At this time, if the position of the qth alveolar block is before the position of the heart block of the pth pair (step S203 Yes), it can be determined that the unidirectional slide scanning of the probe 3A has been successful. In this case, the determination unit 15D calculates the distance between the heart block of the pth pair and the qth alveolar block, for example, the horizontal distance between the centers of gravity of the two types of blocks (step S204).

[0116] By repeating this loop process 4, the distance between the alveolar block preceding the heart block of the pth pair among the Q alveolar blocks and the heart block of the pth pair can be calculated.

[0117] Then, the determination unit 15D associates the alveolar block that is the shortest distance from the heart block of the p-th pair among the outflow tract blocks preceding the heart block of the p-th pair with the heart block of the p-th pair (step S205).The determination unit 15D then extracts a combination of three blocks in which the p-th pair and the alveolar block that is the shortest distance from the heart block of the p-th pair are associated (step S206).

[0118] By repeating this loop process 3, it is possible to associate the preceding adjacent alveolar block with each of the P pairs. At this time, it is possible to identify, among the P pairs, pairs that do not have a preceding adjacent alveolar block, as pairs that do not include the alveolar region in the observation cross-sections proposed in fetal echocardiography guidelines, etc., and therefore these pairs are excluded from the display candidates for the second region detection map.

[0119] Then, when multiple combinations of three blocks are extracted as display target candidates for the second part detection map, the determination unit 15D accepts a selection of a display target candidate from among the multiple display target candidates via the client terminal 30 (step S207).

[0120] Thereafter, the determination unit 15D determines the display settings of the second part detection map based on the relative positions of the three blocks extracted as display target candidates (step S208).

[0121] Then, the display control unit 15E causes the client terminal 30 to display the second part detection map generated in accordance with the display settings determined in step S208 (step S209), and ends the process.

[0122] (2) Block detection process 18 is a flowchart showing the procedure for block detection processing. Here, the detection of "gastric alveolar block," "heart block," and "outflow tract block" can be performed using common logic except for threshold Th1, which is used for comparison with the detection rate of the region, so heart block detection will be explained as a representative example.

[0123] As shown in FIG. 18, the detection unit 15C executes loop process 1, which repeats the processes from step S301 to step S311 described below, for the number K of frames included in the first part detection map, i.e., the number of frames included in the scanning range for the specified time.

[0124] That is, the detection unit 15C calculates the detection rate of the heart region in the k-th frame (step S301), and then the detection unit 15C determines whether the detection rate of the heart region in the k-th frame exceeds a threshold value Th1 (step S302).

[0125] If the detection rate of the heart region in the k-th frame exceeds the threshold value Th1 (Yes in step S302), the above-described judgment logic 1 is satisfied. In this case, the detection unit 15C increments the number of consecutive frames f in which the detection rate exceeds the threshold value Th1 (step S303). Note that if the detection rate of the heart region in the k-th frame does not exceed the threshold value Th1 (No in step S302), the number of consecutive frames f is reset to an initial value, for example, "0" (step S304).

[0126] Thereafter, the detection unit 15C determines whether the number of consecutive frames f exceeds the threshold value Th2 (step S305). If the number of consecutive frames f exceeds the threshold value Th2 (step S305 Yes), both the above-mentioned determination logic 1 and the above-mentioned determination logic 2 are satisfied. Furthermore, the detection unit 15C further determines whether the k-th frame is a new heart block, that is, a leading frame that satisfies both the above-mentioned determination logic 1 and the above-mentioned determination logic 2 (step S306).

[0127] Then, if the k-th frame is a new heart block (Yes in step S306), the detection unit 15C increments the block number i that identifies the heart block (step S307).

[0128] Next, the detection unit 15C assigns a heart block number i to consecutive frames including the k-th frame, i.e., frames whose detection rate exceeds the threshold Th1 consecutively up to the k-th frame (step S308). Note that if the k-th frame is not a new heart block (step S306 No), the detection unit 15C assigns the heart block number i to the k-th frame without incrementing it (step S309).

[0129] Thereafter, the detection unit 15C determines whether or not the distance d, for example, the shortest distance, between the heart block with the i-th block number and the heart block with the (i-1)-th block number is less than a threshold value Th3 (step S310).

[0130] Then, if the distance between the heart block of the i-th block number and the i-1-th block number is less than the threshold value Th3 (Yes in step S310), the detection unit 15C merges the heart block of the i-th block number with the heart block of the i-1-th block number (step S311).

[0131] By repeating this loop process 1, cardiac blocks can be detected from the first region detection map.

[0132] <One aspect of the effect> As described above, the server device 10 according to this embodiment detects blocks of regions from a series of images obtained by probe scanning, where the number of frames in which the region detection rate continuously exceeds a threshold value exceeds a threshold value, and displays the scanning range of the series of blocks of the gastric alveolus, the heart, and the outflow tract on the region detection map. This prevents scanning planes other than the observation planes proposed in fetal echocardiography guidelines, i.e., the scanning range from the gastric alveolus to the cardiac base, from being included in the list display of the region detection map. Therefore, the display control function according to this embodiment improves the visibility of the scanning range from the gastric alveolus to the cardiac base when the region detection map is displayed.

[0133] <Example 2> Although the embodiments of the disclosed device have been described above, the present invention may be embodied in various different forms other than the above-described embodiments. Therefore, other embodiments included in the present invention will be described below.

[0134] <Distribution and Integration> Furthermore, the components of each device shown in the figure do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. For example, the acquisition unit 15A, the detection unit 15B, the detection unit 15C, the determination unit 15D, or the display control unit 15E may be connected to the server device 10 via a network as an external device. Alternatively, the acquisition unit 15A, the detection unit 15B, the detection unit 15C, the determination unit 15D, or the display control unit 15E may be included in a separate device, and the functions of the server device 10 may be realized by the separate device being connected to a network and operating in cooperation with the separate device.

[0135] <Hardware configuration> Furthermore, the various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. Therefore, an example of a computer that executes a display control program having the same functions as those in the first and second embodiments will be described below with reference to FIG. 19.

[0136] Fig. 19 is a diagram showing an example of a hardware configuration. As shown in Fig. 19, a computer 100 has an operation unit 110a, a speaker 110b, a camera 110c, a display 120, and a communication unit 130. The computer 100 also has a CPU 150, a ROM 160, an HDD 170, and a RAM 180. These units 110 to 180 are connected via a bus 140.

[0137] 19, the HDD 170 stores a display control program 170a that performs the same functions as the acquiring unit 15A, the detecting unit 15B, the detecting unit 15C, the determining unit 15D, and the display control unit 15E shown in the first embodiment. This display control program 170a may be integrated or separated, similar to the respective components of the acquiring unit 15A, the detecting unit 15B, the detecting unit 15C, the determining unit 15D, and the display control unit 15E shown in FIG. 1. In other words, the HDD 170 does not necessarily have to store all of the data shown in the first embodiment, as long as data used for processing is stored in the HDD 170.

[0138] Under such an environment, the CPU 150 reads the display control program 170a from the HDD 170 and loads it in the RAM 180. As a result, the display control program 170a functions as a display control process 180a, as shown in FIG. 19. The display control process 180a loads various data read from the HDD 170 in an area of the storage area of the RAM 180 allocated to the display control process 180a, and executes various processes using the loaded data. For example, examples of the processes executed by the display control process 180a may include the processes shown in FIGS. 16 to 18. Note that the CPU 150 does not necessarily need to operate all of the processing units shown in the first embodiment above; it is sufficient that the processing units corresponding to the processes to be executed are virtually implemented.

[0139] The display control program 170a does not necessarily have to be stored in the HDD 170 or the ROM 160 from the beginning. For example, the display control program 170a may be stored in a portable physical medium, such as a flexible disk inserted into the computer 100, such as a FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. The computer 100 may then acquire and execute the display control program 170a from such a portable physical medium. Alternatively, the display control program 170a may be stored in another computer or server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, or the like. The display control program 170a stored in this manner may be downloaded to the computer 100 and then executed.

[0140] The following additional notes are provided regarding the embodiments including the above examples.

[0141] (Appendix 1) Obtaining imaging images of multiple areas including the gastric vesicle, heart, and outflow tract, Detecting multiple parts at different times during scanning; The block included in the captured image is detected by regarding a portion where the detection rate value exceeds a set value and continues for a set number of frames or more as a block of the target portion, and regarding blocks that are adjacent to each other by a certain amount or more as one continuous block. determining a start position of the display area based on the relative positional relationship between a first block corresponding to the gastric alveolus, a second block corresponding to the heart, and a third block corresponding to the outflow tract so that the heart is at the center of the display area; displaying a region detection map associated with the time series of the scans according to the determined start position of the display region; A display control program that causes a computer to execute a process.

[0142] (Appendix 2) A display control program as described in Appendix 1, characterized in that the determining process includes a process of determining the display position of the first frame of the first block corresponding to the gastric alveolus portion as the start position of the part detection map.

[0143] (Appendix 3) The display control program described in Appendix 1, characterized in that the determining process includes a process of determining the display position of the center of the second block corresponding to the heart part to be the center position of the part detection map.

[0144] (Appendix 4) The display control program described in Appendix 1, characterized in that the determining process includes a process of limiting the shift of the display position of the first frame of the second block corresponding to the heart area to outside the predetermined range when the display position of the first frame of the second block corresponding to the heart area shifts within a predetermined range from the display start position of the part detection map.

[0145] (Appendix 5) The display control program described in Appendix 1, characterized in that the detection process includes a process that starts with detecting the second block corresponding to the heart part among a first block corresponding to the gastric alveolus part, a second block corresponding to the heart part, and a third block corresponding to the outflow tract.

[0146] (Appendix 6) The display control program described in Appendix 5, characterized in that the detection process includes a process of stopping detection of the first block corresponding to the gastric alveolus portion and the third block corresponding to the outflow tract if the second block corresponding to the heart portion is not detected.

[0147] (Appendix 7) The display control program described in Appendix 1, characterized in that the determining process includes a process of matching the scanning range of a first block corresponding to the gastric alveolus portion, a second block corresponding to the cardiac portion, and a third block corresponding to the outflow tract with the display range of the part detection map.

[0148] (Appendix 8) The display control program described in Appendix 7, characterized in that the scanning range includes each block in chronological order: a first block corresponding to the gastric alveolus portion, a second block corresponding to the cardiac portion, and a third block corresponding to the outflow tract.

[0149] (Appendix 9) Obtaining imaging images in which multiple areas including the gastric vesicle, the heart, and the outflow tract are scanned; Detecting multiple parts at different times during scanning; The block included in the captured image is detected by regarding a portion where the detection rate value exceeds a set value and continues for a set number of frames or more as a block of the target portion, and regarding blocks that are adjacent to each other by a certain amount or more as one continuous block. determining a start position of the display area based on the relative positional relationship between a first block corresponding to the gastric alveolus, a second block corresponding to the heart, and a third block corresponding to the outflow tract so that the heart is at the center of the display area; displaying a region detection map associated with the time series of the scans according to the determined start position of the display region; A display control method characterized in that processing is executed by a computer.

[0150] (Appendix 10) A display control method as described in Appendix 9, characterized in that the determining process includes a process of determining the display position of the first frame of the first block corresponding to the gastric alveolus portion as the start position of the part detection map.

[0151] (Appendix 11) The display control method described in Appendix 9, characterized in that the determining process includes a process of determining the display position of the center of the second block corresponding to the heart part to be the center position of the part detection map.

[0152] (Appendix 12) The display control method described in Appendix 9, characterized in that the determining process includes a process of limiting the shift of the display position of the first frame of the second block corresponding to the heart area to outside the predetermined range when the display position of the first frame of the second block corresponding to the heart area shifts within a predetermined range from the display start position of the part detection map.

[0153] (Appendix 13) The display control method described in Appendix 9, characterized in that the detection process includes a process starting with detection of the second block corresponding to the heart part among a first block corresponding to the gastric alveolus part, a second block corresponding to the heart part, and a third block corresponding to the outflow tract.

[0154] (Appendix 14) The display control method described in Appendix 13, characterized in that the detection process includes a process of stopping detection of the first block corresponding to the gastric alveolus portion and the third block corresponding to the outflow tract if the second block corresponding to the heart portion is not detected.

[0155] (Appendix 15) The display control method described in Appendix 9, characterized in that the determining process includes a process of matching the scanning range of a first block corresponding to the gastric alveolus portion, a second block corresponding to the cardiac portion, and a third block corresponding to the outflow tract with the display range of the part detection map.

[0156] (Appendix 16) The display control method described in Appendix 15, characterized in that the scanning range includes each block in chronological order: a first block corresponding to the gastric alveolus portion, a second block corresponding to the cardiac portion, and a third block corresponding to the outflow tract.

[0157] (Appendix 17) Obtaining imaging images in which multiple areas including the gastric vesicle, the heart, and the outflow tract are scanned; Detecting multiple parts at different times during scanning; The block included in the captured image is detected by regarding a portion where the detection rate value exceeds a set value and continues for a set number of frames or more as a block of the target portion, and regarding blocks that are adjacent to each other by a certain amount or more as one continuous block. determining a start position of the display area based on the relative positional relationship between a first block corresponding to the gastric alveolus, a second block corresponding to the heart, and a third block corresponding to the outflow tract so that the heart is at the center of the display area; displaying a region detection map associated with the time series of the scans according to the determined start position of the display region; An information processing device comprising: a control unit that executes processing.

[0158] (Appendix 18) The information processing device described in Appendix 17, characterized in that the determining process includes a process of determining the display position of the first frame of the first block corresponding to the gastric alveolus portion as the start position of the part detection map.

[0159] (Supplementary Note 19) The information processing device according to Supplementary Note 17, wherein the determining process includes a process of determining the display position of the center of the second block corresponding to the heart region to be the center position of the part detection map.

[0160] (Appendix 20) The information processing device described in Appendix 17, characterized in that the determining process includes a process of limiting the shift of the display position of the first frame of the second block corresponding to the heart area to outside the predetermined range when the display position of the first frame of the second block corresponding to the heart area shifts within a predetermined range from the display start position of the part detection map. [Explanation of symbols]

[0161] 3 Inspection equipment 3A probe 10 Server device 11 Communication control section 13 Storage section 13A First part detection map 13B Second part detection map 15 Control Unit 15A Acquisition Department 15B Detection unit 15C Detector 15D Decision section 15E Display control unit 30 client terminals

Claims

1. Acquire imaging images in which a plurality of regions including the gastric vesicle, the heart, and the outflow tract are scanned; Detecting multiple parts at different times during scanning; The block included in the captured image is detected by regarding a portion where the detection rate value exceeds a set value and continues for a set number of frames or more as a block of the target portion, and regarding blocks that are adjacent to each other by a certain amount or more as one continuous block. determining a start position of the display area based on a relative positional relationship between a first block corresponding to the gastric alveolus, a second block corresponding to the heart, and a third block corresponding to the outflow tract, so that the heart is at the center of the display area; displaying a region detection map associated with the time series of the scans according to the determined start position of the display region; A display control program that causes a computer to execute a process.

2. The display control program according to claim 1, characterized in that the determining process includes a process of determining the display position of the first frame of the first block corresponding to the gastric alveolus portion as the start position of the part detection map.

3. 2. The display control program according to claim 1, wherein the determining process includes determining a display position of a center of a second block corresponding to the heart to be a center position of the part detection map.

4. The display control program according to claim 1, characterized in that the determining process includes a process of limiting the shift of the display position of the first frame of the second block corresponding to the heart to outside the predetermined range when the display position of the first frame of the second block corresponding to the heart shifts within a predetermined range from the display start position of the part detection map.

5. The display control program of claim 1, characterized in that the detection process includes a process that starts with detecting the second block corresponding to the heart portion among a first block corresponding to the gastric alveolus portion, a second block corresponding to the heart portion, and a third block corresponding to the outflow tract.

6. The display control program of claim 5, characterized in that the detection process includes a process of stopping detection of the first block corresponding to the gastric alveolus portion and the third block corresponding to the outflow tract if the second block corresponding to the heart portion is not detected.

7. A display control program described in any one of claims 1 to 6, characterized in that the determination process includes a process of matching the scanning range of a first block corresponding to the gastric alveolus portion, a second block corresponding to the cardiac portion, and a third block corresponding to the outflow tract with the display range of the part detection map.

8. The display control program of claim 7, characterized in that the scanning range includes, in chronological order, a first block corresponding to the gastric alveolus portion, a second block corresponding to the cardiac portion, and a third block corresponding to the outflow tract.

9. Acquire imaging images in which a plurality of regions including the gastric vesicle, the heart, and the outflow tract are scanned; Detecting multiple parts at different times during scanning; The block included in the captured image is detected by regarding a portion where the detection rate value exceeds a set value and continues for a set number of frames or more as a block of the target portion, and regarding blocks that are adjacent to each other by a certain amount or more as one continuous block. determining a start position of the display area based on a relative positional relationship between a first block corresponding to the gastric alveolus, a second block corresponding to the heart, and a third block corresponding to the outflow tract, so that the heart is at the center of the display area; displaying a region detection map associated with the time series of the scans according to the determined start position of the display region; A display control method characterized in that processing is executed by a computer.

10. Acquire imaging images in which a plurality of regions including the gastric vesicle, the heart, and the outflow tract are scanned; Detecting multiple parts at different times during scanning; The block included in the captured image is detected by regarding a portion where the detection rate value exceeds a set value and continues for a set number of frames or more as a block of the target portion, and regarding blocks that are adjacent to each other by a certain amount or more as one continuous block. determining a start position of the display area based on a relative positional relationship between a first block corresponding to the gastric alveolus, a second block corresponding to the heart, and a third block corresponding to the outflow tract, so that the heart is at the center of the display area; displaying a region detection map associated with the time series of the scans according to the determined start position of the display region; An information processing device comprising: a control unit that executes processing.

Images