Heart ultrasonic image recognition system and ultrasonic diagnosis peripheral device

Abstract

The invention provides a cardiac ultrasonic image recognition system and an ultrasonic diagnosis peripheral device, and relates to the technical field of image recognition, and the system comprises a data collection module which is used for obtaining a to-be-recognized cardiac ultrasonic image; and the automatic identification module is used for extracting a to-be-detected area on the cardiac ultrasound image, counting the number of light spots with the area larger than a preset threshold in the to-be-detected area, and taking the number as an identification result of the cardiac ultrasound image. The method has the advantages that the light spots with the area larger than the preset threshold value contained in each frame of cardiac ultrasound image can be automatically recognized, the light spots are microbubbles, the number of the light spots can be counted to assist a doctor in grading the amount of the microbubbles, then whether the heart foramen ovale is not closed or not is prompted in an auxiliary mode, labor cost is effectively saved, and work efficiency is improved. The working efficiency and accuracy of doctors are improved; the device can be used as peripheral equipment of the cardiac ultrasonic detection equipment, software and hardware improvement on the cardiac ultrasonic detection equipment is not needed, and application and popularization cost is saved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a cardiac ultrasonic image recognition system and an ultrasonic diagnostic peripheral device. Background Art

[0002] In the field of ultrasound examination, ultrasound detection equipment usually transmits high-frequency sound waves (ultrasound) to the human body through an ultrasound probe. The sound waves penetrate human tissue and are reflected when encountering tissue interfaces of different densities. The ultrasound probe converts the received sound wave signals into electrical frequency signals, which are then processed and converted in a series of ways to form a two-dimensional image on the machine screen. Ultrasound doctors or professional technicians make corresponding diagnoses based on the different image manifestations.

[0003] The incidence of patent foramen ovale (PFO) is high, accounting for about 25% in adults. With the increasing understanding of the physiological and pathological significance of PFO, people have found that PFO is one of the causes of unexplained stroke, migraine, paradoxical embolism, etc. Transthoracic echocardiography is a commonly used examination method for non-invasive evaluation of cardiac structure and function. However, since the sound beam has to penetrate the body surface tissue to reach the heart, it will be attenuated. For adults, its acoustic window conditions are poor, so its detection rate of PFO is not high.

[0004] In the prior art, right heart acoustic angiography is usually used to check for patent foramen ovale. Right heart acoustic angiography is to inject contrast agent intravenously, and then observe whether there are microbubbles in the left atrium and ventricle to determine whether there is a patent foramen ovale. This examination has a high diagnostic value for PFO. The "Chinese Expert Consensus on Right Heart Acoustic Angiography for Patent Foramen Ovale" (2022) regards it as the gold standard imaging method for diagnosing PFO.

[0005] However, ultrasound diagnosis usually requires experienced professionals. When interpreting right heart acoustic angiography, each frame of the image needs to be replayed, and the appearance of microbubbles in the left atrium and ventricle, the time of microbubble appearance, the amount of microbubbles and the duration of microbubbles need to be carefully observed to comprehensively judge whether there is a patent foramen ovale or a pulmonary arteriovenous fistula or both. During the interpretation of right heart acoustic angiography, ultrasound doctors need to keep an eye on the screen at all times, collect dynamic images of at least 10-15 cardiac cycles (or 10-30s) after the right heart is developed, and manually replay them frame by frame to determine different cardiac cycles (including when microbubbles appear, when microbubbles reach their peak value and when they completely dissipate), the duration of microbubbles in cardiac cycles, and then manually count the microbubbles in the frame image where the ultrasound microbubbles reach their peak value to determine the microbubble classification. This method of naked eye observation and counting has low diagnostic efficiency. When the number of tests increases, the diagnostic report is often not issued in time, the patient's waiting time is prolonged, and even multiple trips to the hospital are required; it also affects the timely diagnosis and treatment of clinicians, and the medical efficiency and quality are reduced, which not only wastes the patient's time, but also increases the operating costs of medical institutions. In addition, there may be subjectivity in images between the diagnostic index cutoff points, which may affect the final result and easily cause visual fatigue to ultrasound doctors and affect their vision. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a cardiac ultrasound image recognition system, comprising:

[0007] A data acquisition module, used for acquiring a cardiac ultrasound image to be identified;

[0008] The automatic recognition module is connected to the data acquisition module and is used to extract the area to be detected on the cardiac ultrasound image and count the number of light spots contained in the area to be detected and whose area is greater than a preset threshold as the recognition result of the cardiac ultrasound image.

[0009] Preferably, the automatic identification module includes:

[0010] A human-computer interaction interface, used for displaying the acquired cardiac ultrasound image, so that the user can mark the extraction position of the area to be detected on the cardiac ultrasound image;

[0011] An extraction unit, connected to the human-computer interaction interface, and configured to extract the area to be detected on the cardiac ultrasound image according to the extraction position;

[0012] The image recognition unit is connected to the extraction unit and is used to perform image recognition on the area to be detected, and when it is recognized that the area to be detected contains the light spots with an area larger than the preset threshold, count the number of the light spots.

[0013] Preferably, the user marks the extraction position by setting a mask on the human-computer interaction interface, and sets the extraction position to white and other positions to black;

[0014] The extraction unit extracts the area to be detected using the mask.

[0015] Preferably, the image recognition unit comprises a preprocessing subunit for sharpening the area to be detected before performing image recognition on the area to be detected.

[0016] Preferably, the image recognition unit is also connected to the human-computer interaction interface, and includes a display processing subunit for marking the identified light spots with an area larger than the preset threshold on the cardiac ultrasound image and displaying them through the human-computer interaction interface.

[0017] Preferably, the image recognition unit uses Green's formula to calculate the area of ​​the light spot.

[0018] Preferably, the automatic identification module includes a prompt unit for giving a prompt that the foramen ovale is not closed when the number of the light spots is greater than a preset value.

[0019] The present invention also provides an ultrasonic diagnostic peripheral device, comprising:

[0020] A computer, wherein the automatic recognition module in the above cardiac ultrasound image recognition system is integrated into the computer;

[0021] The video acquisition card is inserted into the computer and connected to the cardiac ultrasound detection device as the data acquisition module in the cardiac ultrasound image recognition system.

[0022] Preferably, it also includes a mouse and a keyboard connected to the computer, and a user uses the mouse and the keyboard to mark the extraction position of the area to be detected on the cardiac ultrasound image displayed on the human-computer interaction interface of the computer;

[0023] When the image recognition algorithm carried by the computer is executed, the automatic recognition module extracts the area to be detected on the cardiac ultrasound image according to the extraction position, and then performs image recognition on the area to be detected, and when it is identified that the area to be detected contains the light spots with an area greater than the preset threshold, the number of the light spots is counted.

[0024] The above technical solution has the following advantages or beneficial effects:

[0025] 1) It can automatically identify the light spots in each frame of cardiac ultrasound image with an area larger than a preset threshold, which are microbubbles, and can count the number of light spots to assist doctors in grading the amount of microbubbles, and then assist in indicating whether there is a patent foramen ovale, effectively saving labor costs and improving doctors' work efficiency and accuracy;

[0026] 2) The present invention can be used as a peripheral device of a cardiac ultrasonic detection device, without the need to improve the software and hardware of the cardiac ultrasonic detection device itself, thus saving application and promotion costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the structure of a cardiac ultrasound image recognition system in a preferred embodiment of the present invention;

[0028] Figure 2 is a schematic diagram of a cardiac ultrasound image in a preferred embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the extraction position of the area to be detected in a preferred embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of an extracted area to be detected in a preferred embodiment of the present invention;

[0031] Figure 5 is a schematic diagram of a sharpened image in a preferred embodiment of the present invention;

[0032] Figure 6 The figure is a schematic structural diagram of an ultrasonic diagnostic peripheral device in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The present invention is not limited to this embodiment, and other embodiments may also fall within the scope of the present invention as long as they conform to the gist of the present invention.

[0034] In a preferred embodiment of the present invention, based on the above problems existing in the prior art, a cardiac ultrasound image recognition system is provided, comprising:

[0035] A data acquisition module 1 is used to obtain a cardiac ultrasound image to be identified;

[0036] The automatic recognition module 2 is connected to the data acquisition module 1 and is used to extract the area to be detected on the cardiac ultrasound image and count the number of light spots in the area to be detected whose area is larger than a preset threshold as the recognition result of the cardiac ultrasound image.

[0037] Specifically, in this embodiment, the cardiac ultrasound image is acquired by an external cardiac ultrasound detection device and then transmitted to the cardiac ultrasound image recognition system of the present invention for automatic recognition.

[0038] Furthermore, the automatic identification module 2 includes:

[0039] A human-computer interaction interface 21, used to display the acquired cardiac ultrasound image, so that the user can mark the extraction position of the area to be detected on the cardiac ultrasound image;

[0040] An extraction unit 22, connected to the human-computer interaction interface 21, is used to extract a region to be detected on the cardiac ultrasound image according to an extraction position;

[0041] The image recognition unit 23 is connected to the extraction unit 22 and is used to perform image recognition on the area to be detected, and count the number of light spots when it is recognized that the area to be detected contains light spots with an area larger than a preset threshold.

[0042] Specifically, in this embodiment, the acquired cardiac ultrasound image can be displayed in real time on the human-computer interaction interface 21, such as Figure 2 As shown, the user marks the extraction position by setting a mask on the human-computer interaction interface 21, and sets the extraction position to white and other parts to black, and obtains the following Figure 3 The white area shown; the extraction unit 22 uses the mask to extract the area to be detected. Among them, Figure 3 The white area in the middle defines the extraction position. After obtaining the extraction position, the mask is used to extract the original cardiac ultrasound image. Figure 4 The area to be detected is shown.

[0043] To facilitate subsequent spot recognition, in this embodiment, the image recognition unit 23 includes a pre-processing sub-unit 231, which is used to sharpen the area to be detected before performing image recognition on the area to be detected, so as to obtain Figure 5 The sharpened image shown in FIG. 1 is used as the basis for the image recognition unit 23 to perform light spot recognition based on the sharpened image. Figure 5 It can be seen that, without considering the size of the light spot area, the light spot in the sharpened image is a closed area surrounded by one or more smooth curves, and then the image recognition unit 23 can calculate the area of ​​the light spot using Green's formula.

[0044] Among them, the definition of Green's formula is as follows:

[0045] If the functions P(x,y), Q(x,y) are continuous in a closed region D enclosed by one or more smooth curves and have continuous first-order partial derivatives, then

[0046]

[0047] Wherein, L is the boundary curve of region D and takes the positive direction.

[0048] and Figure 5 The light spot in is a simple polygon with non-intersecting edges, which is a closed area surrounded by several line segments, so Green's formula can be used.

[0049] Furthermore, in the calculation, P = 0, Q = x, then the area S = ∫∫ D dσ=∮ L xdy

[0050] Let the i-th point P in the closed region be i The coordinates of (x i ,y i ), the i+1th point P i+1 The coordinates of (x i+1 ,y i+1 ), then line segment P i P i+1 The parameter formula is

[0051]

[0052] so

[0053]

[0054] So the area

[0055] When the calculated area of ​​the light spot is greater than the preset threshold, the light spot is considered to be a microbubble that can characterize the characteristics of the patent foramen ovale, and the corresponding light spots are counted to obtain the number of light spots characterizing the microbubbles in the area to be detected in each frame of the cardiac ultrasound image. Doctors can grade the microbubbles based on the number, such as grade 0, indicating that there are no microbubbles in the left heart cavity; grade I, indicating that there are <10 microbubbles / frame in the left heart cavity; grade II, indicating that there are 10-30 microbubbles / frame in the left heart cavity; grade III, indicating that >30 microbubbles / frame can be seen in the left heart cavity, or the left heart cavity is almost full of microbubbles.

[0056] In this embodiment, if the number of light spots with an area greater than a preset threshold in the area to be detected in the cardiac ultrasound image is greater than a preset value, where the preset value is preferably 30, it is considered that the foramen ovale is not closed. Based on this, the automatic identification module 2 includes a prompt unit 24, which is used to give a prompt that the foramen ovale is not closed when the number of light spots is greater than the preset value.

[0057] In a preferred embodiment of the present invention, the image recognition unit 23 is also connected to the human-computer interaction interface 21, and includes a display processing subunit 232, which is used to mark the identified light spots with an area greater than a preset threshold on the cardiac ultrasound image and display them through the human-computer interaction interface 21.

[0058] In a preferred embodiment of the present invention, the present invention also provides an ultrasonic diagnostic peripheral device, such as Figure 6 As shown, including:

[0059] Computer 100, the automatic recognition module 2 in the above cardiac ultrasound image recognition system is integrated into the computer 100;

[0060] The video acquisition card 200 is inserted into the computer 100 and connected to the cardiac ultrasound detection device 300 as the data acquisition module 1 in the above cardiac ultrasound image recognition system.

[0061] In a preferred embodiment of the present invention, a mouse and a keyboard 400 connected to the computer 100 are further included, and a user uses the mouse and the keyboard 400 to mark the extraction position of the area to be detected on the cardiac ultrasound image displayed on the human-computer interaction interface of the computer 100;

[0062] When the image recognition algorithm carried by the computer 100 is executed, the automatic recognition module 2 extracts the area to be detected on the cardiac ultrasound image according to the extraction position, and then performs image recognition on the area to be detected, and counts the number of light spots when it is identified that the area to be detected contains light spots with an area greater than a preset threshold.

[0063] Specifically, in this embodiment, the ultrasonic diagnostic peripheral device is composed of a video capture card 200, a computer 100 with a display screen, a mouse and a keyboard 400, and an image recognition algorithm running in the computer 100, wherein the video capture card 200 is inserted into the computer 100, and the computer 100 is externally connected to the mouse and the keyboard 400. When in use, the video external output port of the cardiac ultrasonic detection device 300 is connected to the video capture card 200, and the mouse and the keyboard 400 are used to set the area to be detected in the video, and then the image recognition algorithm is run to count the number of spots in the area to be detected that are greater than a preset threshold value, so that the doctor can view it.

[0064] As a preferred implementation, a case of right heart acoustic contrast examination is positive and diagnosed as PFO (large). The ultrasound examination equipment has an HDMI (high-definition) interface, which can transmit the image information of ultrasound detection through the HDMI interface. This embodiment uses the OK_HDMI40A-4E 4K HDMI image acquisition card of Jiaheng Image as the video acquisition card, and uses Python for software development. The steps are as follows:

[0065] 1. Call the video capture card driver to capture the cardiac ultrasound image transmitted by the ultrasound examination device, such as Figure 2 As shown;

[0066] 2. Use the mouse and keyboard to set the mask, set the position to be detected to white, and set other positions to black, such as Figure 3As shown;

[0067] 3. The program uses the mask to extract the required detection position and obtain the following Figure 4 The image shown;

[0068] 4. The program will extract the following Figure 4 The image shown is sharpened to obtain Figure 5 The sharpened image shown;

[0069] 6. The program uses an image recognition algorithm to identify spots larger than a certain area as microbubbles that characterize patent foramen ovale, and then counts the number of spots. In this embodiment, when the number of contrast agent spots in the detected area is greater than or equal to 30, it is considered to be patent foramen ovale.

[0070] 7. While counting the number of light spots, light spots larger than a certain area can also be marked on the cardiac ultrasound image for real-time viewing and confirmation by the doctor. Preferably, the light spots can be marked by marking a rectangular frame on the light spots, and the rectangular frame can be the smallest rectangle circumscribing the light spots, but it is not limited to this.

[0071] In summary, the ultrasonic inspection device has an HDMI (high definition) interface, and the image information of the ultrasonic detection can be transmitted to the ultrasonic diagnostic peripheral device of the present invention through the HDMI interface. The ultrasonic diagnostic peripheral device uses a computer algorithm to quickly give corresponding diagnostic parameters and make preliminary judgments on these parameters. The parameters here are the number of spots larger than a certain area. The ultrasonic diagnostic peripheral device is suitable for PFO diagnosis, saving labor costs and improving work efficiency and accuracy.

[0072] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.

Claims

1. A cardiac ultrasound image recognition system, characterized in that: include: A data acquisition module, used for acquiring a cardiac ultrasound image to be identified; The automatic recognition module is connected to the data acquisition module and is used to extract the area to be detected on the cardiac ultrasound image and count the number of light spots contained in the area to be detected and whose area is greater than a preset threshold as the recognition result of the cardiac ultrasound image.

2. The system according to claim 1, characterized in that The automatic identification module comprises: A human-computer interaction interface, used for displaying the acquired cardiac ultrasound image, so that the user can mark the extraction position of the area to be detected on the cardiac ultrasound image; An extraction unit, connected to the human-computer interaction interface, and configured to extract the area to be detected on the cardiac ultrasound image according to the extraction position; The image recognition unit is connected to the extraction unit and is used to perform image recognition on the area to be detected, and when it is recognized that the area to be detected contains the light spots with an area larger than the preset threshold, count the number of the light spots.

3. The system according to claim 2, characterized in that The user marks the extraction position by setting a mask on the human-computer interaction interface, setting the extraction position to white and other positions to black. The extraction unit extracts the area to be detected using the mask.

4. The system according to claim 2, characterized in that The image recognition unit includes a preprocessing subunit, which is used to sharpen the area to be detected before performing image recognition on the area to be detected.

5. The system according to claim 2, characterized in that The image recognition unit is also connected to the human-computer interaction interface, and includes a display processing subunit for marking the light spots whose areas are recognized to be larger than the preset threshold on the cardiac ultrasound image and displaying the light spots through the human-computer interaction interface.

6. The system according to claim 2, characterized in that The image recognition unit calculates the area of ​​the light spot using Green's formula.

7. The system according to claim 1, characterized in that The automatic identification module comprises a prompting unit, which is used to give a prompt that the foramen ovale is not closed when the number of the light spots is greater than a preset value.

8. An ultrasonic diagnostic peripheral device, characterized in that: include: A computer, wherein the automatic recognition module in the cardiac ultrasound image recognition system according to any one of claims 1 to 7 is integrated into the computer; The video acquisition card is inserted into the computer and connected to the cardiac ultrasound detection device as the data acquisition module in the cardiac ultrasound image recognition system as described in any one of claims 1 to 6.

9. The ultrasonic diagnostic peripheral device according to claim 8, characterized in that: It also includes a mouse and a keyboard connected to the computer, and a user uses the mouse and the keyboard to mark the extraction position of the area to be detected on the cardiac ultrasound image displayed on the human-computer interaction interface of the computer; When the image recognition algorithm carried by the computer is executed, the automatic recognition module extracts the area to be detected on the cardiac ultrasound image according to the extraction position, and then performs image recognition on the area to be detected, and when it is identified that the area to be detected contains the light spots with an area greater than the preset threshold, the number of the light spots is counted.

Images