Aortic plaque detection method, device and system based on multi-stage semantic segmentation

Abstract

The invention discloses an aortic plaque detection method, device and system based on multi-stage semantic segmentation, and the method comprises the steps: carrying out the region segmentation of an obtained image, and obtaining a region image which comprises an ascending aorta region image, a descending aorta region image and an aortic arch region image; and taking the regional images as input, inputting the regional images into a pre-trained first neural network model for processing, and obtaining plaque detection results corresponding to the regional images. The method based on deep learning and machine learning can efficiently and accurately position and calculate parameters such as the position and the volume of the calcified plaque in the aorta, so that repeated and complicated work of manual recognition and positioning can be effectively reduced, and the accuracy and the speed of aorta calcification evaluation can be greatly improved; and the detection efficiency is effectively improved. Meanwhile, by means of the method, accurate quantitative evaluation on aorta calcification can be achieved, and further medical research can be assisted.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an aortic plaque detection method, device and system based on multi-stage semantic segmentation. Background Art

[0002] Vascular calcification is the accumulation of calcium and phosphate products between and within the cells of the blood vessel wall. Accumulated calcified plaques often cause vessel wall stiffness and thrombosis, leading to narrowing of the vessel lumen and, in turn, reduced oxygen and blood supply to related organs. This ultimately manifests as chronic diseases such as atherosclerosis, hypertension, and various vascular lesions and injuries. Aortic calcification (AC) and coronary artery calcification (CAC) are significant contributors to the high morbidity and mortality rates of cardiovascular disease.

[0003] Aortic calcification is the accumulation of calcium salts and other components on the aortic wall, which, over time, can form aortic calcification. This calcified plaque typically adheres closely to the vessel wall and is essentially a special type of atherosclerosis.

[0004] The formation of aortic plaques is related to a variety of factors, including physiological factors (such as vascular aging caused by aging), lipid metabolism disorders, hypertension, hyperlipidemia, diabetes, and damage to blood vessel walls. These factors may cause lipid substances in the blood to accumulate on the inner wall of the aorta, forming plaques. Over time, these plaques may further calcify to form calcified plaques. The formation of calcified plaques is potentially harmful to vascular health. If the plaque is large, it may cause vascular stenosis. Although the aorta itself is relatively thick and the incidence of stenosis is relatively low, attention should still be paid to the possible risks. Calcified plaques may also cause the blood vessel wall to lose elasticity, increase the risk of blood vessel rupture, and then cause cardiovascular diseases such as coronary heart disease and myocardial infarction.

[0005] Currently, coronary artery calcification is primarily detected in computed tomography (CT) images. In clinical practice, doctors often use semi-automatic software to manually locate coronary artery calcification lesions in CT images based on spatial characteristics and CT intensity characteristics. Once the calcified lesions are located, the coronary artery calcification score (CACS) proposed by Agatston is primarily used to quantify coronary artery calcification and assess cardiovascular disease (CVD) risk. Furthermore, coronary artery calcification can often be further classified based on the different vascular branches from which it originates. The coronary artery calcification and corresponding calcification levels on different branches often have different reference significance for the prevention, diagnosis, and treatment of cardiovascular disease.

[0006] There is currently no unified standard for measuring aortic calcification. Studies have shown a positive correlation between the severity of calcified plaques in the thoracic aortic arch and acute stroke. Further research is needed to accurately measure aortic calcification, particularly in the thoracic aorta proximal to the carotid arteries. Existing methods for measuring aortic calcification are based on methods for detecting coronary artery calcification. However, due to differences between the two methods and the fact that coronary artery calcification still requires manual identification and location, the efficiency of aortic calcification measurement remains to be improved. Summary of the Invention

[0007] The embodiments of the present invention provide an aortic plaque detection method, device and system based on multi-stage semantic segmentation to solve the problem in the prior art that manual identification and positioning are still required, making the overall method repetitive, cumbersome and inefficient.

[0008] According to a first aspect of the present invention, a method for detecting aortic plaque based on multi-stage semantic segmentation is provided, comprising:

[0009] Performing regional segmentation on the acquired image to obtain regional images, wherein the regional images include an ascending aorta regional image, a descending aorta regional image, and an aortic arch regional image;

[0010] The regional image is used as input and input into the pre-trained first neural network model for processing to obtain the patch detection results corresponding to each regional image.

[0011] According to a second aspect of the present invention, a device for detecting aortic plaque based on multi-stage semantic segmentation is provided, comprising:

[0012] An image segmentation module is used to perform regional segmentation on the acquired image to obtain regional images, wherein the regional images include an ascending aorta regional image, a descending aorta regional image, and an aortic arch regional image;

[0013] The patch segmentation module is used to take the regional image as input and input it into a pre-trained first neural network model for processing to obtain the patch detection results corresponding to each regional image.

[0014] According to a third aspect of the present invention, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the aortic plaque detection method based on multi-stage semantic segmentation of the above-mentioned first aspect.

[0015] According to a fourth aspect of the present invention, there is provided a storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the aortic plaque detection method based on multi-stage semantic segmentation according to the first aspect is implemented.

[0016] The present invention's multi-stage semantic segmentation-based aortic plaque detection method, leveraging deep learning and machine learning, can efficiently and accurately locate and calculate parameters such as the location and volume of calcified plaques in the aorta. This effectively reduces the repetitive and complex manual work of identification and positioning, significantly improving the accuracy and speed of aortic calcification assessment and effectively enhancing detection efficiency. Furthermore, the present invention's method enables precise quantitative assessment of aortic calcification, assisting in further medical research. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a flowchart of an aortic plaque detection method based on multi-stage semantic segmentation according to an embodiment of the present invention;

[0019] Figure 2 This is a flowchart of step S11 in the aortic plaque detection method based on multi-stage semantic segmentation according to one embodiment of the present invention;

[0020] Figure 3This is a network architecture diagram of a first neural network model of an aortic plaque detection method based on multi-stage semantic segmentation according to an embodiment of the present invention;

[0021] Figure 4 This is a network architecture diagram of an aortic plaque detection method based on multi-stage semantic segmentation according to an embodiment of the present invention when the region category encoding is input to the residual module;

[0022] Figure 5 This is a principle block diagram of an aortic plaque detection device based on multi-stage semantic segmentation according to one embodiment of the present invention;

[0023] Figure 6 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] It should be noted that, unless there is any conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0026] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include" and "comprise" include not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, the elements defined by the phrase "include..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0027] The present invention will be further described in detail below with reference to the accompanying drawings.

[0028] Figure 1 The flowchart of the aortic plaque detection method based on multi-stage semantic segmentation according to one embodiment of the present invention is schematically shown. Figure 1 The aortic plaque detection method of the present invention comprises the following steps:

[0029] Step S11: performing regional segmentation on the acquired image to obtain regional images, wherein the regional images include an ascending aorta regional image, a descending aorta regional image, and an aortic arch regional image;

[0030] Step S12: using each region image as input and inputting it into a pre-trained first neural network model to obtain a patch detection result corresponding to each region image.

[0031] Reference Figure 1 As shown, in the present invention, the overall process is mainly divided into two steps: step S11 and step S12. In step S11, the acquired image is first processed to improve the accuracy of the result finally output when the acquired image is processed in the subsequent step S12. It can be understood that by dividing the acquired image into regions, useful parts (such as the aorta part) in the acquired image can be extracted, thereby reducing the influence of other parts of the image during the subsequent step S12 processing. Among them, the acquired image can be obtained by communicating with a CT device or a data storage system (server, etc.), or by user input, etc., and the present invention is not limited to this. According to the regional images obtained after processing in step S11, it includes an ascending aorta (ascending_aorta) regional image, a descending aorta (descending_aorta) regional image, and an aortic arch (aorta_arch) regional image. By segmenting the entire image into regions, a regional image that can accurately reflect whether there is plaque in the aorta is obtained, so that the corresponding regional image can be detected in step S12, thereby improving processing efficiency and accuracy.

[0032] Figure 2 The process steps of step S11 in the aortic plaque detection method based on multi-stage semantic segmentation according to one embodiment of the present invention are schematically shown. Figure 2 As shown, the specific steps of step S11 can be implemented as follows:

[0033] Step S21: performing a first region segmentation on the acquired image to obtain an aorta image;

[0034] Step S22: The aorta image is input into a pre-trained second neural network model to perform second region segmentation to obtain regional images. Each regional image obtained includes an image image and a binary image corresponding thereto.

[0035] Given that the original medical image has a large size and the calcification area is usually relatively small, if these large-sized original images are directly input into the aorta and plaque segmentation model for processing, the segmentation effect may be poor due to the loss of image details or the limitation of the model processing capability. Therefore, in step S21, the acquired image is first segmented into the first region to crop the area where the aorta is located in the image, thereby effectively reducing the image size and reducing the background area that is irrelevant to the subsequent task segmentation to obtain the aorta image. Specifically, in step S21, TotalSegmentor can be used for implementation. TotalSegmentor is a tool for segmenting most major anatomical structures (including the aorta) in any CT or MR image. It uses training from a variety of different CT and MR images (different scanners, institutions, protocols, etc.), so it can work well on most images. In step S21, TotalSegmentor is used to obtain the segmentation result of the aorta, and the original image is cropped according to the minimum circumscribed rectangle of the segmentation result to obtain the aorta image.

[0036] Then, step S22 is executed to perform a second regional segmentation on the aorta image using the pre-trained second neural network model, ultimately obtaining a regional image. The second neural network model used in step S22 is a semantic segmentation model, and its network architecture can be a U-Net-based network architecture, a V-Net-based network architecture, or other commonly used neural network model architecture for image regional segmentation, but the present invention is not limited thereto.

[0037] Specifically, when training the second neural network model, the second data set used includes the aortic image and the regional images corresponding to the aortic image. Furthermore, when training the second neural network model, the aortic image is used as input and the regional images corresponding to the aortic image are used as output. The second neural network model is trained, ultimately obtaining a second neural network model that can automatically segment the aortic image into regions and obtain regional images. The specific training process of the second neural network model can be implemented based on relevant content in the prior art and will not be described in detail here. After the second neural network model is trained, step S22 can be executed, where the aortic image is used as input and input into the pre-trained second neural network model for second regional segmentation to obtain regional images. The obtained regional images can be cropped to obtain image images and binary images (mask images) corresponding to the regional images. The binary image is an image of the same size as the image image, but with only two voxel values ​​corresponding to 0 and 1, where 0 represents the background and 1 represents the aorta. The final regional images include three groups, namely (image_ascending_aorta, mask_ascending_aorta) corresponding to the ascending aorta regional image, (image_descending_aorta, mask_descending_aorta) corresponding to the descending aorta regional image, and (image_aorta_arch, mask_aorta_arch) corresponding to the aortic arch regional image.

[0038] After obtaining the regional image according to step S21 and step S22, step S12 can be executed to obtain the plaque detection result corresponding to each regional image, thereby realizing a rapid assessment of aortic calcification. Specifically, in step S12, the pre-trained first neural network model used is a semantic segmentation model built on the basis of the U-Net architecture, which specifically includes an encoder and a decoder. Figure 3 As shown, Figure 3 The network architecture diagram of the first neural network model used in the aortic plaque detection method based on multi-stage semantic segmentation according to one embodiment of the present invention is schematically shown. Figure 3As shown, the encoder includes a first encoder and a second encoder. The first encoder takes an image as input, and the second encoder takes a binary image (mask image) as input. The input of the residual module at each level in the decoder is the result of the element-wise addition of the output of the residual module at the corresponding level in the first decoder and the output of the residual module at the corresponding level in the convolution of the second decoder. Since the position of the aorta can be represented by the voxel value 1 in the binary image, the binary image can guide the model to search for plaques within the current aorta without paying attention to plaques in other areas, thereby limiting which pixels in the image belong to this type of aortic area. It can be understood that the difference between the network architecture of the first neural network model adopted by the present invention and the general U-Net network architecture in the prior art is that the network architecture of the first neural network model of the present invention has two branches, namely the image input branch and the binary image (mask image) input branch. At the same time, in the binary image (mask image) input branch, it is not provided with a corresponding decoder separately, but the two input branches share a decoder.

[0039] Specifically, during training of the first neural network model, the first dataset used includes image images, binary images, and plaque detection results corresponding to each regional image. For example, taking the three sets of regional images described above as an example, the first dataset may include (image_ascending_aorta, mask_ascending_aorta, mask_ascending_aorta_plaque) corresponding to the ascending aorta regional image, (image_descending_aorta, mask_descending_aorta, mask_descending_aorta_plaque) corresponding to the descending aorta regional image, and (image_aorta_arch, mask_aorta_arch, mask_aorta_arch_plaque) corresponding to the aortic arch regional image. Among them, mask_ascending_aorta_plaque, mask_descending_aorta_plaque, and mask_aorta_arch_plaque are the plaque detection results corresponding to the ascending aorta regional image, the descending aorta regional image, and the aortic arch regional image, respectively. Furthermore, when training the first neural network model, the image image and the binary image corresponding to the regional image are used as input, and the plaque detection results corresponding to the corresponding regional image are used as output. The first neural network model is trained to ultimately obtain a first neural network model capable of obtaining the plaque detection results for the input regional image based on the input regional image. The obtained plaque detection results can include a mask image of the same size as the original image, containing values ​​corresponding to 10 different voxels, ranging from 0 to 9, where 0 represents background and 1 to 9 represent nine different types of plaque. By counting the number of values ​​corresponding to different voxels in the regional image, the types of plaque present in the regional image can be determined. Furthermore, the actual volume of each voxel can be calculated based on the properties of the CT image, thereby calculating the volume of the corresponding plaque. By further analyzing and processing these results, the final displayed plaque detection result can include one of the presence or absence of plaque, calcified plaque, non-calcified plaque, and mixed plaque, so that the plaque detection result output by the trained first neural network model after processing is correspondingly one of the absence of plaque, calcified plaque, non-calcified plaque, and mixed plaque. In other possible implementations, the plaque detection result can also include one of calcified plaque, non-calcified plaque, and mixed plaque, and can then be used only for images with plaques to quickly confirm the type of plaque in the current image.The specific training process of the first neural network model can be implemented based on relevant knowledge in the prior art and will not be further described here. After the first neural network model is trained, step S12 is executed, where the regional images are input into the pre-trained first neural network model for processing to obtain the plaque detection results corresponding to each regional image.

[0040] In the solution of the present invention, the regional images include three parts: the ascending aorta regional image, the descending aorta regional image, and the aortic arch regional image. These three parts have different regional image characteristics. If the first neural network model is used for plaque detection, they are likely to interfere with or affect each other during training and application, resulting in reduced detection accuracy. Therefore, in some possible embodiments, when training the first neural network model, the corresponding regional images can be marked to be included in the training of the first neural network model, so that the first neural network model can recognize the corresponding characteristics of different regional images during use, thereby improving recognition accuracy. Specifically, in step S11, the obtained regional images can be further divided and identified. For example, a regional category code can be designed for each of the ascending aorta regional image, the descending aorta regional image, and the aortic arch regional image. Furthermore, when training the first neural network model, the input of the first neural network model can also include the regional category identifier corresponding to each regional image. Specifically, during training and application, the regional category identifier corresponding to each regional image can be input into the residual module of each stage of the encoder and decoder of the first neural network model. Among them, since the inputs of the first encoder and the second encoder are superimposed, the region category identifier corresponding to each region image can be input only to the residual module of each level of the first encoder. Of course, the region category identifier corresponding to each region image can also be input to the first encoder and the second encoder at the same time. Figure 3 As shown, in Figure 3 In the network architecture diagram of the first neural network model shown in FIG, during training and application, the region category identifier corresponding to each region image is only input into the residual module of each level of the first encoder. When the region category identifier corresponding to the region image is input into the residual module, it can be referred to Figure 4 The network architecture shown in the figure is used to implement Figure 4In the illustrated network architecture, a fully connected layer first processes the region category identifiers and convolves the features in the corresponding residual modules. These two layers are then superimposed, and then the features in the residual modules before convolution are further superimposed. Finally, a final convolution is performed to produce the output. This design allows the first neural network model of the present invention to use the region category identifiers of the input region images to further identify the region to which the region image currently undergoing patch detection belongs, thereby enabling the use of a single neural network model to accurately detect and identify multiple different region images, improving recognition accuracy.

[0041] As for the region category codes corresponding to the above-mentioned region images, after step S11, that is, after the region segmentation is performed to obtain the region image, the region category codes of the region images obtained by segmentation can be marked to form region category codes corresponding to the type of the region image, so as to associate them with each region image. In this way, when the first neural network model is subsequently applied for analysis, the region category codes corresponding to each region image can be input into the first neural network model together with the region image.

[0042] The aortic plaque detection method based on multi-stage semantic segmentation of the present invention is based on deep learning and machine learning methods and can efficiently and accurately locate and calculate parameters such as the position and volume of calcified plaques in the aorta, thereby effectively reducing the repetitive, complex and tedious work of manual identification and positioning, and thus can greatly improve the accuracy and speed of aortic calcification assessment, effectively improving detection efficiency. At the same time, the method of the present invention can also provide an accurate quantitative assessment of aortic calcification to assist further medical research. Moreover, the aortic plaque detection method based on multi-stage semantic segmentation of the present invention is a multi-stage semantic segmentation method for the aorta and plaques, which first performs a first region segmentation on the acquired image to obtain an aortic image, then performs a second region segmentation on the aortic image to obtain images of each region of the aorta, and finally performs plaque segmentation and detection on each region image to solve the problem of large image size and small target. The aortic plaque detection method based on multi-stage semantic segmentation of the present invention is a segmentation method that uses aortic region binary maps and region category identifiers for conditional injection, thereby enabling the model to know the aortic region in which the currently detected part is located and which pixels belong to the aorta, thereby effectively improving the overall detection accuracy.

[0043] Figure 5 The present invention schematically illustrates an aortic plaque detection device based on multi-stage semantic segmentation according to an embodiment of the present invention. Figure 5 As shown, the aortic plaque detection device based on multi-stage semantic segmentation of the present invention includes the following modules:

[0044] An image segmentation module 1 is configured to segment the acquired image into regions to obtain regional images, wherein the regional images include an ascending aorta regional image, a descending aorta regional image, and an aortic arch regional image;

[0045] The patch segmentation module 2 is used to take the regional image as input and input it into a pre-trained first neural network model for processing to obtain the patch detection results corresponding to each regional image.

[0046] Among them, the image segmentation module 1 may include a positioning unit and an aortic segmentation unit. The positioning unit is used to perform a first region segmentation on the acquired image to obtain an aortic image. The aortic segmentation unit is used to perform a second region segmentation on the aortic image to obtain various regional images. These regional images all include image images and binary images corresponding to each regional image.

[0047] It should be noted that the implementation process and principles of the multi-stage semantic segmentation-based aortic plaque detection device according to the present invention can be found in the corresponding description of the method embodiment above, such as the corresponding description of image segmentation, plaque segmentation, and other related contents in the method embodiment, and will not be repeated here. For example, the multi-stage semantic segmentation-based aortic plaque detection device according to the present invention can be any intelligent device with a processor, including but not limited to computers, smartphones, personal computers, robots, cloud servers, etc.

[0048] In some embodiments, an embodiment of the present invention provides an aortic plaque detection system based on multi-stage semantic segmentation, which includes an aortic plaque detection device based on multi-stage semantic segmentation according to any one of the above embodiments of the present invention and a tomography device for CT scanning, so that the aortic plaque detection system based on multi-stage semantic segmentation can be used in medical institutions to quickly detect whether there is plaque in the aorta in the image image of the CT scan result, and quickly detect and determine the type of plaque present in the aorta in the image image of the CT scan result.

[0049] In some embodiments, an embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the aortic plaque detection method based on multi-stage semantic segmentation of any of the above embodiments of the present invention.

[0050] In some embodiments, an embodiment of the present invention further provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the aortic plaque detection method based on multi-stage semantic segmentation of any one of the above embodiments.

[0051] In some embodiments, an embodiment of the present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the aortic plaque detection method based on multi-stage semantic segmentation of any of the above embodiments.

[0052] In some embodiments, an embodiment of the present invention further provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the aortic plaque detection method based on multi-stage semantic segmentation of any of the above embodiments is implemented.

[0053] Figure 6 is a hardware structure diagram of an electronic device for performing an aortic plaque detection method based on multi-stage semantic segmentation, provided in another embodiment of the present application. Figure 6 As shown, the device includes:

[0054] One or more processors 610 and memory 620, Figure 6 A processor 610 is taken as an example.

[0055] The device for performing the aortic plaque detection method based on multi-stage semantic segmentation may further include: an input device 630 and an output device 640 .

[0056] The processor 610, the memory 620, the input device 630 and the output device 640 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0057] Memory 620, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the aortic plaque detection method based on multi-stage semantic segmentation in the embodiments of this application. Processor 610 executes the non-volatile software programs, instructions, and modules stored in memory 620 to execute various server functions and data processing, thereby implementing the aortic plaque detection method based on multi-stage semantic segmentation in the aforementioned method embodiments.

[0058] Memory 620 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store data generated based on the aortic plaque detection method based on multi-stage semantic segmentation. Furthermore, memory 620 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, memory 620 may optionally include memory remotely located from processor 610, and such remote memory may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0059] The input device 630 may receive input digital or character information and generate signals related to user settings and function control of the image processing device. The output device 640 may include a display device such as a display screen.

[0060] The one or more modules are stored in the memory 620 , and when executed by the one or more processors 610 , perform the aortic plaque detection method based on multi-stage semantic segmentation in any of the above method embodiments.

[0061] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0062] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0063] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0064] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0065] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0066] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0067] (5) Other electronic devices with data interaction functions.

[0068] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0069] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting aortic plaque based on multi-stage semantic segmentation, characterized in that: include: Performing regional segmentation on the acquired image to obtain regional images, wherein the regional images include an ascending aorta regional image, a descending aorta regional image, and an aortic arch regional image; The regional image is used as input and input into the pre-trained first neural network model for processing to obtain the patch detection results corresponding to each regional image.

2. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 1, characterized in that: The step of performing region segmentation on the acquired image to obtain a region image includes: Performing a first region segmentation on the acquired image to obtain an aorta image; The aorta image is used as input and input into a pre-trained second neural network model for second region segmentation to obtain regional images, wherein each regional image includes an image image and a binary image corresponding to each regional image.

3. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 2, characterized in that: The first region segmentation is implemented using TotalSegmentor.

4. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 2, characterized in that: The second neural network model is a semantic segmentation model, and the second data set used in its training includes an aortic image and regional images corresponding to the aortic image. When training the second neural network model, the aortic image is used as input and the regional images corresponding to the aortic image are used as output to train the second neural network model.

5. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 1, characterized in that: The regional images include image images and binary images corresponding to each regional image; The first neural network model is a semantic segmentation model built on the basis of the U-Net architecture, including an encoder and a decoder. The encoder includes a first encoder that takes an image as input and a second encoder that takes a binary image as input. The input of each residual module in the decoder is the result of the element-wise addition of the output of the residual module at the corresponding level in the first decoder and the output of the residual module at the corresponding level in the convolutional second decoder. The image of each region is input into a pre-trained first neural network model to obtain the plaque detection result corresponding to each region image, including: The image image and binary image corresponding to each regional image are used as input and input into the pre-trained first neural network model to obtain the plaque detection results corresponding to each regional image.

6. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 5, characterized in that: The first data set used in training the first neural network model includes image images, binary images and plaque detection results corresponding to each regional image. When training the first neural network model, the image images and binary images corresponding to each regional image are used as input, and the plaque detection results corresponding to the corresponding regional images are used as output to train the first neural network model.

7. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 5, characterized in that: The regional images also include regional category identifiers corresponding to the regions of each regional image; When the first neural network model is trained, the input of the first neural network model also includes a region category identifier corresponding to each region image; When each regional image is used as input and input into a pre-trained first neural network model to obtain a patch detection result corresponding to each regional image, the input of the first neural network model also includes a regional category identifier corresponding to each regional image.

8. The aortic plaque detection method based on multi-stage semantic segmentation according to claim 7, characterized in that: When training, the first neural network model further includes: Inputting the region category identifier corresponding to the region of each region image into the residual module of each level of the encoder and decoder of the first neural network model to train the first neural network model; When the images of each region are used as input to the pre-trained first neural network model, the method further includes: The region category identifier corresponding to each region image is input into the residual module of each level of the encoder and decoder in the first neural network model.

9. An aortic plaque detection device based on multi-stage semantic segmentation, characterized in that: include: An image segmentation module is used to perform regional segmentation on the acquired image to obtain regional images, wherein the regional images include an ascending aorta regional image, a descending aorta regional image, and an aortic arch regional image; The patch segmentation module is used to take the regional image as input and input it into a pre-trained first neural network model for processing to obtain the patch detection results corresponding to each regional image.

10. An aortic plaque detection system based on multi-stage semantic segmentation, characterized in that: include: The aortic plaque detection device based on multi-stage semantic segmentation according to claim 9, A tomography device is communicatively connected to the aortic plaque detection device based on multi-stage semantic segmentation to send image images to the aortic plaque detection device based on multi-stage semantic segmentation.

Images