Dsa image foreground feature enhancement method and module, and stenosis lesion detection device

Abstract

The application provides a DSA image foreground feature enhancement method and module and a stenosis lesion detection device. The method comprises the following steps: receiving an initial feature map of a DSA image, wherein the initial feature map is extracted from the DSA image by a Faster-RCNN network in advance; and obtaining a foreground feature enhancement map corresponding to the initial feature map by using a channel attention mechanism, so that the Faster-RCNN network locates a blood vessel stenosis position of the DSA image based on the foreground feature enhancement map. The application can effectively enhance the foreground features of the DSA image used for blood vessel stenosis detection, thereby improving the automation and intelligence degrees of locating the blood vessel stenosis position of the DSA image, and effectively improving the accuracy and effectiveness of locating the blood vessel stenosis position of the DSA image by using the Faster-RCNN algorithm.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and module for enhancing the foreground features of DSA images and a device for detecting stenotic lesions. Background Technology

[0002] The success of convolutional neural networks in the field of natural image processing is already well-known, and the methods and devices based on convolutional neural networks have also been applied in the field of medical imaging, such as Faster-RCNN (a target detection algorithm based on convolutional neural networks), whose superior performance is trusted by doctors.

[0003] Digital subtraction angiography (DSA) images, due to their excellent real-time performance, high resolution, and intuitiveness, enable the localization and measurement of vascular stenosis, thus providing essential conditions for various interventional treatments. Currently, to achieve automated target detection in the field of medical imaging, convolutional neural networks (CNNs), which have already achieved success in natural image processing, are widely used. For example, the Faster-RCNN target detection algorithm based on CNNs is trusted by physicians for its superior performance.

[0004] However, since there is currently no Faster-RCNN algorithm capable of detecting targets such as vascular stenosis in DSA images, the performance of DSA images in target detection has not been explored or reported. DSA images are created by injecting contrast agents into blood vessels under X-ray irradiation, allowing the contrast agents to flow with the blood to reflect narrowing or occlusion. Due to the uneven variation in blood vessel thickness, large fluctuations in image quality, and differences in surgeon experience, quickly and automatically locating the location of vascular stenosis lesions in DSA images remains a challenging problem. Simply applying DSA images directly to the Faster-RCNN algorithm will result in insufficient emphasis on the target region, shallow feature extraction of the target region, and treating it the same as the image background, thus sacrificing performance. Therefore, the accuracy and effectiveness of vascular stenosis detection results based on DSA images cannot be guaranteed. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and module for enhancing the foreground features of DSA images and a device for detecting stenotic lesions, so as to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of this application provides a method for enhancing the foreground features of a DSA image, comprising:

[0007] Receive an initial feature map of the DSA image, which is pre-extracted from the DSA image by the Faster-RCNN network;

[0008] A channel attention mechanism is used to obtain a foreground feature enhancement map corresponding to the initial feature map, so that the Faster-RCNN network can locate the vascular stenosis in the DSA image based on the foreground feature enhancement map.

[0009] In some embodiments of this application, the step of using a channel attention mechanism to perform foreground feature enhancement processing on the initial feature map to obtain corresponding enhanced image features includes:

[0010] The initial feature map is compressed and its response distribution is adjusted based on the channel attention mechanism. A corresponding foreground feature enhancement map is then generated based on the response distribution adjustment result and the initial feature map.

[0011] In some embodiments of this application, the step of performing image compression and response distribution adjustment on the initial feature map based on the channel attention mechanism, and generating a corresponding foreground feature enhancement map based on the corresponding response distribution adjustment result and the initial feature map, includes:

[0012] The initial feature map is input into a preset global average pooling layer, so that the initial feature map is compressed into the corresponding target vector based on the channel attention mechanism in the global average pooling layer;

[0013] The target vector is input into a preset fully connected layer, and the response distribution of the target vector is adjusted in the fully connected layer to obtain a re-response operator as the result of the response distribution adjustment.

[0014] A corresponding foreground feature enhancement map is generated based on the re-response operator and the initial feature map.

[0015] The second aspect of this application provides a channel attention module for performing the DSA image foreground feature enhancement method provided in the first aspect above, the channel attention module comprising:

[0016] The feature receiving unit is used to receive an initial feature map of the DSA image, which is extracted in advance from the DSA image by the Faster-RCNN network;

[0017] The feature enhancement unit is used to obtain a foreground feature enhancement map corresponding to the initial feature map using a channel attention mechanism, so that the Faster-RCNN network can locate the vascular stenosis location in the DSA image based on the foreground feature enhancement map.

[0018] A third aspect of this application provides a device for detecting vascular stenosis lesions in DSA images, comprising:

[0019] The image receiving module is used to receive the target DSA image;

[0020] The model prediction module is used to input the target DSA image into a Faster-RCNN network for automatic detection of vascular stenosis lesions, so that after the Faster-RCNN network for automatic detection of vascular stenosis lesions extracts the initial feature map from the target DSA image, it uses the DSA image foreground feature enhancement method provided in the first aspect to obtain the corresponding foreground feature enhancement map, and then determines the location of vascular stenosis and the detection result of vascular stenosis lesions in the target DSA image based on the foreground feature enhancement map.

[0021] In some embodiments of this application, it also includes:

[0022] The training set generation module is used to generate a training set based on each DSA historical image and the correspondence between the corresponding vascular stenosis location identifier and the lesion detection result identifier for each DSA historical image. The lesion detection result identifier includes: a first identifier for indicating that a lesion has occurred at the vascular stenosis location and a second identifier for indicating that no lesion has occurred at the vascular stenosis location.

[0023] The model training module is used to train a preset deep learning model based on the training set using the Faster-RCNN algorithm to obtain the corresponding Faster-RCNN network for automatic detection of vascular stenosis lesions.

[0024] In some embodiments of this application, the Faster-RCNN network for automatic detection of vascular stenosis lesions includes:

[0025] The convolutional layer skeleton is used to extract features from the input DSA image to obtain the corresponding initial feature map;

[0026] The channel attention module is used to execute the DSA image foreground feature enhancement method provided in the first aspect above, and to obtain the foreground feature enhancement map corresponding to the DSA image;

[0027] An RPN network is used to extract candidate regions from the foreground feature enhancement map and obtain the corresponding candidate region location data.

[0028] A Region of Interest Pooling Network (RPN) is used to extract the foreground feature map corresponding to the candidate region based on the foreground feature enhancement map output by the channel attention module and the candidate region location data output by the RPN network.

[0029] A classifier is used to determine the location of vascular stenosis and the detection result of vascular stenosis lesions in the target DSA image based on the foreground feature map, using a preset fully connected layer and candidate box regression, wherein the detection result of vascular stenosis lesions is used to indicate whether a lesion has occurred at the location of vascular stenosis.

[0030] In some embodiments of this application, the convolutional layer skeleton includes a ResNet50 network.

[0031] A fourth aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the function of the vascular stenosis detection device in the DSA image provided in the third aspect above.

[0032] The fifth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the function of the vascular stenosis detection device in the DSA image provided in the third aspect above.

[0033] The DSA image foreground feature enhancement method provided in this application receives an initial feature map of a DSA image, which is pre-extracted from the DSA image by a Faster-RCNN network. A channel attention mechanism is used to obtain a foreground feature enhancement map corresponding to the initial feature map, enabling the Faster-RCNN network to locate the vascular stenosis position in the DSA image based on this foreground feature enhancement map. This method improves the automation and intelligence of vascular stenosis location in DSA images, saving time and manpower costs. It effectively enhances the foreground features of DSA images used for vascular stenosis detection, thereby improving the efficiency, accuracy, and effectiveness of using the Faster-RCNN algorithm for vascular stenosis location in DSA images, while also enhancing the automation and intelligence of vascular stenosis location using DSA images.

[0034] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0035] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings:

[0037] Figure 1 This is a schematic diagram of the first process of a DSA image foreground feature enhancement method according to an embodiment of this application.

[0038] Figure 2 This is a schematic diagram of a second process of the DSA image foreground feature enhancement method in one embodiment of this application.

[0039] Figure 3 This is a schematic diagram of the channel attention module in another embodiment of this application.

[0040] Figure 4 This is a schematic diagram of the first structure of a device for detecting vascular stenosis lesions in DSA images according to another embodiment of this application.

[0041] Figure 5 This is a second structural schematic diagram of a device for detecting vascular stenosis lesions in DSA images according to another embodiment of this application.

[0042] Figure 6 This is a schematic diagram of the architecture of the Faster-RCNN network for automatic detection of vascular stenosis lesions provided in this application.

[0043] Figure 7 This is a schematic diagram of the channel attention module provided in the application example of this application.

[0044] Figure 8 This is a schematic diagram of the ResNet50 framework provided in the application example of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0046] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0047] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0048] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0049] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0050] To automate vascular stenosis detection in DSA images and improve detection accuracy, this application provides a DSA image foreground feature enhancement method that can be implemented using a channel attention module. (See [link to relevant documentation]). Figure 1 The DSA image foreground feature enhancement method specifically includes the following:

[0051] Step 100: Receive the initial feature map of the DSA image, which is extracted in advance from the DSA image by the Faster-RCNN network.

[0052] Understandably, the Faster-RCNN network can also be referred to as the Faster-RCNN network for detecting blood vessel stenosis. Specifically, the Faster-RCNN network is a two-stage object detection method that aims to find the coordinates and category of a target object in a given image.

[0053] In step 100, the channel attention module can be pre-installed in the Faster-RCNN network. The input of the channel attention module can be connected to the module in the Faster-RCNN network used to output the initial feature map of the DSA image, thereby receiving the initial feature map of the DSA image. This allows for automated vascular stenosis detection of DSA images and improves detection accuracy through structural optimization of the Faster-RCNN network. Alternatively, the channel attention module can be set up independently of the Faster-RCNN network. The input of this independently set channel attention module can also be connected to the model in the Faster-RCNN network used to output the initial feature map of the DSA image, receiving the initial feature map of the DSA image. This approach does not require changing the structure of the Faster-RCNN network and can be applied to more types of deep learning models capable of vascular stenosis detection, thus improving the applicability of the channel attention module.

[0054] In one or more embodiments of this application, the initial feature map refers to the image features extracted from the DSA image and displayed as image data, for example, it can be extracted using convolutional layers based on the Faster-RCNN algorithm.

[0055] Step 200: Use a channel attention mechanism to obtain a foreground feature enhancement map corresponding to the initial feature map, so that the Faster-RCNN network can locate the vascular stenosis in the DSA image based on the foreground feature enhancement map.

[0056] In step 200, the channel attention module can be pre-installed in the Faster-RCNN network. The output of the channel attention module can be connected to the module in the Faster-RCNN network used to locate the vascular stenosis in the DSA image based on the foreground feature enhancement map, so as to receive the foreground feature enhancement map output by the channel attention module. This allows for further optimization of the Faster-RCNN network structure to achieve automated vascular stenosis detection in DSA images and improve detection accuracy. Alternatively, the channel attention module can be set up independently of the Faster-RCNN network. The output of this independently set channel attention module can also be connected to the module in the Faster-RCNN network used to locate the vascular stenosis in the DSA image based on the foreground feature enhancement map, so as to receive the foreground feature enhancement map output by the channel attention module. This approach does not require changing the structure of the Faster-RCNN network and can be applied to more types of deep learning models capable of vascular stenosis detection, thereby improving the applicability of the channel attention module.

[0057] In step 200, locating the vascular stenosis in the DSA image specifically refers to the Faster-RCNN network outputting vascular stenosis detection results for the DSA image. These vascular stenosis detection results specifically include the location coordinate data of at least one target region, which can be marked with a colored box in the DSA image. The vascular stenosis detection results also include an indicator of whether the location coordinate data of each target region corresponds to vascular stenosis. For example, different indicators can be set in each box in the DSA image to indicate whether vascular stenosis occurs within the box. For example, Y can indicate that vascular stenosis occurs within the box, and N can indicate that vascular stenosis does not occur within the box.

[0058] In one or more embodiments of this application, steps 100 and 200 described above are applicable to the online application stage of the Faster-RCNN network and also to the offline training stage of the Faster-RCNN network. That is, the DSA image mentioned in the DSA image foreground feature enhancement method provided in the embodiments of this application includes: the target DSA image to be detected in the target region during the model application stage, and the historical DSA image used to train the Faster-RCNN network.

[0059] As can be seen from the above description, the DSA image foreground feature enhancement method provided in this application embodiment can effectively enhance the foreground features of DSA images used for vascular stenosis detection. In this way, it can effectively improve the accuracy and effectiveness of using the Faster-RCNN algorithm to locate vascular stenosis in DSA images, while improving the automation and intelligence of vascular stenosis location using DSA images.

[0060] To further improve the reliability and effectiveness of foreground feature enhancement, in a DSA image foreground feature enhancement method provided in this application embodiment, step 200 of the DSA image foreground feature enhancement method specifically includes the following:

[0061] Step 210: Perform image compression and response distribution adjustment on the initial feature map based on the channel attention mechanism, and generate a corresponding foreground feature enhancement map based on the corresponding response distribution adjustment result and the initial feature map.

[0062] It is understandable that image compression and response distribution adjustment of the initial feature map based on the channel attention mechanism means: compressing the original initial feature map F of size CxHxW into a vector of size Cx1x1, where C represents the number of channels, H is the height in the image size, and W is the width in the image size; and then readjusting the response distribution of this vector to generate the corresponding response distribution adjustment result.

[0063] To further improve the reliability and effectiveness of foreground feature enhancement, in a DSA image foreground feature enhancement method provided in this application embodiment, see [link to relevant documentation]. Figure 2 Step 210 in the DSA image foreground feature enhancement method specifically includes the following:

[0064] Step 211: Input the initial feature map into a preset global average pooling layer, so that the initial feature map is compressed into the corresponding target vector based on the channel attention mechanism in the global average pooling layer.

[0065] Step 212: Input the target vector into a preset fully connected layer, and adjust the response distribution of the target vector in the fully connected layer to obtain a re-response operator as the result of the response distribution adjustment.

[0066] Step 213: Generate a corresponding foreground feature enhancement map based on the re-response operator and the initial feature map.

[0067] In step 213, the re-response operator V and the initial feature map F can be multiplied together to obtain the foreground feature enhancement map O after re-response.

[0068] From a software perspective, this application also provides a channel attention module for performing all or part of the aforementioned DSA image foreground feature enhancement method; this module can also be called a channel attention device. See [link to relevant documentation]. Figure 3 The channel attention module specifically includes the following components:

[0069] The feature receiving unit 10 is used to receive an initial feature map of the DSA image, which is extracted in advance from the DSA image by the Faster-RCNN network;

[0070] The feature enhancement unit 20 is used to obtain a foreground feature enhancement map corresponding to the initial feature map using a channel attention mechanism, so that the Faster-RCNN network can locate the vascular stenosis location in the DSA image based on the foreground feature enhancement map.

[0071] The channel attention module provided in this application can be used to execute the processing flow of the DSA image foreground feature enhancement method in the above embodiments. Its function will not be repeated here, but can be referred to the detailed description of the above DSA image foreground feature enhancement method embodiments.

[0072] As can be seen from the above description, the channel attention module provided in this application embodiment can effectively enhance the foreground features of DSA images used for vascular stenosis detection, thereby improving the automation and intelligence of vascular stenosis location using DSA images, and effectively improving the accuracy and effectiveness of vascular stenosis location using the Faster-RCNN algorithm.

[0073] In recent years, with the improvement of people's living standards and the increase in work pressure, the number of stroke patients has been rising year by year. Stroke, also known as cerebrovascular accident, has the world's highest disability rate and the world's second highest mortality rate, placing a heavy burden on the medical system and causing great distress to patients' families. Transvascular interventional surgery is currently a common treatment for stroke, and DSA imaging, due to its excellent real-time performance, high resolution, and intuitiveness, has become the gold standard in the treatment process.

[0074] Faster R-CNN is a convolutional neural network-based object detection algorithm. After training with a certain number of image samples, it can quickly and automatically locate objects in similar images. Faster R-CNN is a two-stage object detection algorithm. The first stage extracts feature maps from the input image through a series of stacked layers. The second stage feeds the obtained feature maps into an RPN network to generate candidate regions for the target. Finally, based on these candidate regions and feature maps, a ROI Pooling layer extracts refined recommended target regions. The final classification layer uses the recommended target regions to predict the category of the target region and uses a candidate box regression algorithm to obtain the location of the target region in the image. Although Faster R-CNN has achieved good performance on natural images, its performance on DSA images has not been explored or reported. Furthermore, the feature maps extracted in the first stage are fed directly to the RPN network without processing, resulting in insufficient emphasis on target regions and shallow feature extraction of target regions, treating them the same as the background, thus sacrificing performance.

[0075] Therefore, based on the DSA image foreground feature enhancement method and / or DSA image foreground feature enhancement device provided in the foregoing embodiments, this application also provides an embodiment of a device for detecting vascular stenosis lesions in DSA images. First, an automatic vascular stenosis lesion detection algorithm is proposed for DSA images. At the same time, improvements are made based on the Faster-RCNN algorithm by adding a channel attention mechanism, so that the Faster-RCNN algorithm can better mine foreground target features, thereby better detecting stenosis lesions.

[0076] The device for detecting vascular stenosis in DSA images provided in this application primarily focuses on the automatic detection of stenosis lesions in DSA images. It mainly relies on a target detection algorithm based on convolutional neural networks. The success of convolutional neural networks in natural image processing is well-established, and methods based on convolutional neural networks have also been applied in the field of medical imaging, where their superior performance is trusted by doctors. However, research has revealed that due to the scarcity of DSA images, variations in the thickness of blood vessels, fluctuations in image quality, and differences in surgeon experience, a solution for quickly and automatically locating vascular stenosis lesions in DSA images has not yet been proposed. This application represents the first attempt in this regard.

[0077] Based on this, in order to further realize the automatic detection of vascular stenosis lesions in DSA images, in the embodiment of this application, a device for detecting vascular stenosis lesions in DSA images is provided, see... Figure 4 The device for detecting vascular stenosis lesions in DSA images specifically includes the following components:

[0078] Image receiving module 1 is used to receive target DSA images.

[0079] Model prediction module 2 is used to input the target DSA image into a Faster-RCNN network for automatic detection of vascular stenosis lesions, so that after the Faster-RCNN network for automatic detection of vascular stenosis lesions extracts the initial feature map from the target DSA image, it uses the DSA image foreground feature enhancement method mentioned in the previous embodiment to obtain the corresponding foreground feature enhancement map, and then determines the location of vascular stenosis and the detection result of vascular stenosis lesions in the target DSA image based on the foreground feature enhancement map.

[0080] As can be seen from the above description, the vascular stenosis lesion detection device in DSA images provided in this application embodiment can effectively enhance the foreground features of DSA images used for vascular stenosis lesion detection, thereby improving the accuracy and effectiveness of using the Faster-RCNN algorithm to detect vascular stenosis lesions in DSA images, while improving the automation and intelligence of vascular stenosis lesion detection using DSA images.

[0081] To further improve the accuracy and effectiveness of using the Faster-RCNN algorithm to detect vascular stenosis lesions in DSA images, this application provides a device for detecting vascular stenosis lesions in DSA images, see [link to relevant documentation]. Figure 5 The device for detecting vascular stenosis lesions in DSA images also specifically includes the following components:

[0082] Training set generation module 3 is used to generate a training set based on the correspondence between each DSA historical image and the corresponding vascular stenosis location identifier and lesion detection result identifier for each DSA historical image. The lesion detection result identifier includes a first identifier indicating that a lesion has occurred at the vascular stenosis location and a second identifier indicating that no lesion has occurred at the vascular stenosis location.

[0083] It is understood that the first identifier can be the value 1, and the second identifier can be the value 0.

[0084] Model training module 4 is used to train a preset deep learning model based on the training set using the Faster-RCNN algorithm to obtain a corresponding Faster-RCNN network for automatic detection of vascular stenosis lesions.

[0085] Specifically, when training the Faster-RCNN algorithm in the framework, the input is each DSA historical image I and the location coordinates L of the vascular stenosis in each DSA historical image. The lesion category is 1, which represents a lesion, and 0, which represents normal.

[0086] To further improve the accuracy and effectiveness of using the Faster-RCNN algorithm to detect vascular stenosis lesions in DSA images, embodiments of the DSA image foreground feature enhancement method and the device for detecting vascular stenosis lesions in DSA images, etc., are provided in this application. Figure 6 The Faster-RCNN network used for automatic detection of vascular stenosis specifically includes the following components:

[0087] The convolutional layer skeleton is used to extract features from the input DSA image to obtain the corresponding initial feature map.

[0088] It is understood that the convolutional layer skeleton can be a ResNet50 network.

[0089] The channel attention module is used to execute the DSA image foreground feature enhancement method to obtain the foreground feature enhancement map corresponding to the DSA image.

[0090] An RPN network is used to extract candidate regions from the foreground feature enhancement map to obtain the corresponding candidate region location data.

[0091] A Region of Interest Pooling Network (RPN) is used to extract the foreground feature map corresponding to the candidate region based on the foreground feature enhancement map output by the channel attention module and the candidate region location data output by the RPN network.

[0092] A classifier is used to determine the location of vascular stenosis and the detection result of vascular stenosis lesions in the target DSA image based on the foreground feature map, using a preset fully connected layer and candidate box regression, wherein the detection result of vascular stenosis lesions is used to indicate whether a lesion has occurred at the location of vascular stenosis.

[0093] The embodiments of the DSA image vascular stenosis detection device provided in this application can be used to execute the processing flow of the embodiments of the DSA image vascular stenosis detection method described above. Its functions will not be repeated here, but can be referred to the detailed description of the embodiments of the DSA image vascular stenosis detection method described above.

[0094] The portion of the DSA image vascular stenosis detection device that detects vascular stenosis in DSA images can be executed on a server. Alternatively, in another practical application, all operations can be completed on the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed on the client device, the client device may further include a processor for specific processing of foreground feature enhancement in the DSA image.

[0095] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0096] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0097] As can be seen from the above description, the vascular stenosis lesion detection device in DSA images provided in this application embodiment can effectively enhance the foreground features of DSA images used for vascular stenosis detection. In this way, it can effectively improve the accuracy and effectiveness of using the Faster-RCNN algorithm to locate vascular stenosis in DSA images, while improving the automation and intelligence of vascular stenosis location using DSA images.

[0098] To further illustrate this solution, this application also provides a specific application example of a method for detecting vascular stenosis in DSA images executed by a device for detecting vascular stenosis in DSA images. This method can also be described as a method for detecting vascular stenosis in DSA images based on convolutional neural networks. This application example proposes an algorithm for automatic vascular stenosis detection in DSA images using Faster-RCNN. Unlike traditional manual inspection of DSA images to find and determine the location of vascular stenosis, this method uses the Faster-RCNN algorithm to extract image features and automatically locate the stenosis. The specific technical solution is as follows: Faster-RCNN first extracts features from the DSA image, then feeds these features into a channel attention module to obtain a foreground feature enhancement map with enhanced foreground target weights. This map is then fed into a Region Proposal Network (RPN) to extract candidate target region locations. After obtaining the target region location, it is combined with the foreground feature enhancement map. Figure 1 The foreground feature map of the target region is extracted by the Region of Interest Pooling (ROI Pooling) network. Finally, the foreground feature map is fed into the classifier, and the narrowness and narrow position are determined by the fully connected layer and the candidate box regression.

[0099] The specific details of the automatic detection framework for vascular stenosis lesions in DSA images are as follows:

[0100] 1) Input and output of automatic detection of vascular stenosis lesions in DSA images

[0101] Input: When training the Faster-RCNN algorithm in the framework, the input is the DSA image I and the location coordinates L of the stenosis in the DSA image. The lesion category is 1, which represents a lesion, and 0, which represents normal.

[0102] 2) Network structure for automatic detection of vascular stenosis lesions in DSA images

[0103] This application uses Faster-RCNN as the baseline method and incorporates a Channel Attention (CA) module to readjust the response of the feature map. The structure of the Channel Attention module is as follows: Figure 7 As shown. After obtaining the feature map extracted by the convolutional layer backbone, the original CxHxW feature is compressed into a 1x1xC vector by global average pooling. Then, it is fed into a fully connected layer to readjust the response distribution to generate the re-response operator V. Finally, the re-response operator V and F are multiplied together to obtain the re-response feature O.

[0104] While other convolutional neural network detection methods can achieve the same detection purpose by replacing the Faster-RCNN network, these other methods do not have data on cerebral vascular stenosis lesions in DSA images, so they cannot be directly used in the tasks related to this application.

[0105] The convolutional layer backbone structure is as follows: Figure 8 As shown, the structure used is ResNet50, and the activation function throughout the method is the ReLU activation function, which maintains non-zero gradients during training and avoids gradient vanishing. After the candidate target regions are obtained by the region of interest pooling network, the candidate target regions are flattened. On the one hand, classification is performed through fully connected layers, and the sigmoid function is chosen as the activation function because this application only distinguishes between lesion locations and non-lesion locations. On the other hand, non-maximum suppression and candidate box regression are used to regress the coordinates of the lesion location.

[0106] exist Figure 8In the diagram, "DSA1×512×512" indicates the input DSA image, which has 1 channel and dimensions of 512*512 pixels; "CONV_7×7_64_S2" indicates the convolutional layer with a kernel size of 7, 64 output channels, and a stride of 2; "BN_ReLu" indicates the input DSA image, which has 1 channel and dimensions of 512*512 pixels. "MaxPooling_3×3_S2" represents a module consisting of batch regularization, linear rectified function, max pooling, kernel size 3, and stride 2; "RESBLOCK_IN64-OUT256" represents a residual convolution module with 64 input channels and 256 output channels; "RESBLOCK_IN128-OUT512" represents a residual convolution module with 128 input channels and 512 output channels; "RESBLOCK_IN256-OUT1024" represents a residual convolution module with 256 input channels and 1024 output channels; "RESBLOCK_IN512-OUT2048" represents a residual convolution module with 512 input channels and 2048 output channels; "RESBLOCK" represents a residual convolution module; "[W, H, IN]" represents the width, height, and number of channels of the feature map or image; "IN==OUT?" indicates... This indicates whether the input channels are equal to the output channels; "COV_1×1_OUT" indicates a convolution kernel of 1 and an output channel count of OUT; "BN" indicates batch regularization; "COV_1×1_IN" indicates a convolution kernel of 1 and an input channel count of IN; "BN_ReLu" indicates a module consisting of batch regularization and a linear rectified function; "COV_3×3_IN(s2)" indicates a convolutional layer with a kernel size of 3, an output channel count equal to the input channel count of IN, and a stride of 2; "COV_3×3_IN" indicates a convolutional layer with a kernel size of 3 and an output channel count of IN; "[int[(W-1) / 2+1],int[(H-1) / 2+1]],OUT" indicates the calculation method of the width, height, and output channel count of the feature map or image based on the width and height of the input feature map or image; "[W,H,OUT]" indicates a feature map with a width of W, a height of H, and an output channel count of OUT.

[0107] 3) Network training process for automatic detection of vascular stenosis lesions in DSA images

[0108] This application conducts experiments on real DSA images. The initial learning rate is set to 3e-4, and cosine annealing is used for decay. The maximum decay period is set to 100 periods, and the model is trained using the Adam optimizer with a batch size of 16. The image size is also resampled from the original 800×800 to 512×512. Common data augmentation operations, such as random flipping, random cropping, random masking, and random resizing, are also used to augment the data and enhance the robustness of the model. The loss function is L(p,u,t). uThe result is as follows:

[0109] L(p,u,t u ,v)=L cls (p,u)+λ[u≥1]L loc (t u ,v)

[0110] L cls (p,u)=-logp u

[0111]

[0112]

[0113] The box loss is calculated only when the candidate region is a non-background category (positive sample). cls For class loss, L loc The bounding box loss is defined as u ≥ 1, indicating that the candidate region belongs to a class greater than or equal to 1, while 0 represents the background. λ is a balancing factor, defaulting to 1. To prevent the L2 loss of candidate region bounding box regression from amplifying noise (abnormal loss) and affecting training, the L2 loss is modified to smooth. L1 The loss is calculated using L1 loss when there are significant differences in box size, to suppress the contribution of outliers to the gradient. The variable V is then calculated using the following formula:

[0114] t x =(G x -P x ) / P w

[0115] t y =(G y -P y ) / P h

[0116] t w =log(G w / P w )

[0117] t h =log(G h / P h )

[0118] In the above formula, P = (P x ,P y ,P w ,P h P represents the predicted box coordinates. x ,P y P represents the predicted center coordinates. w,P h This represents the predicted width and height of the box; G = (G x G y G w G h ), representing the coordinates of the true bounding box, with the subscript meaning consistent with the predicted bounding box, and T = (T x ,T y ,T w ,T h The subscript ) represents the target box for regression, and its meaning is consistent with that of the predicted box. This application proposes an automated framework for detecting vascular stenosis lesions in DSA images based on convolutional neural networks to assist surgeons in quickly locating lesions in DSA images, facilitating further planning of surgical procedures and treatment plans. It also establishes a dataset and a deep learning neural network model for detecting vascular stenosis in DSA images. This model can extract features from DSA images, and after training, it can locate lesions in newly collected DSA images or during actual surgical procedures.

[0119] This application also provides an electronic device (i.e., a computer device), which may include a processor, a memory, a receiver, and a transmitter. The processor is used to perform the functions of the DSA image vascular stenosis lesion detection device mentioned in the above embodiments. The processor and memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and memory via wired or wireless means.

[0120] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0121] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the DSA image vascular stenosis lesion detection device in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby realizing the function of the DSA image vascular stenosis lesion detection device in the above method embodiments.

[0122] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] The one or more modules are stored in the memory and, when executed by the processor, perform the function of the vascular stenosis lesion detection device in the DSA image of the embodiment.

[0124] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0125] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0126] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0127] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the function of the aforementioned DSA image vascular stenosis lesion detection device. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0128] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0129] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0130] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0131] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method of DSA image foreground feature enhancement, characterized in that, include: Receive an initial feature map of the DSA image, which is pre-extracted from the DSA image by the Faster-RCNN network; The initial feature map is input into a preset global average pooling layer, where the initial feature map is compressed into a corresponding target vector based on a channel attention mechanism. The target vector is then input into a preset fully connected layer, where the response distribution of the target vector is adjusted to obtain a re-response operator as the result of the response distribution adjustment. Based on the re-response operator and the initial feature map, a corresponding foreground feature enhancement map is generated, so that the Faster-RCNN network can locate the vascular stenosis in the DSA image based on the foreground feature enhancement map.

2. A channel attention module for DSA images, characterized in that, include: The feature receiving unit is used to receive an initial feature map of the DSA image, which is extracted in advance from the DSA image by the Faster-RCNN network; The feature enhancement unit is used to input the initial feature map into a preset global average pooling layer, so as to compress the initial feature map into a corresponding target vector based on the channel attention mechanism in the global average pooling layer; and to input the target vector into a preset fully connected layer, so as to perform response distribution adjustment on the target vector in the fully connected layer to obtain a re-response operator as the result of response distribution adjustment. Based on the re-response operator and the initial feature map, a corresponding foreground feature enhancement map is generated, so that the Faster-RCNN network can locate the vascular stenosis in the DSA image based on the foreground feature enhancement map.

3. A device for detecting vascular stenosis lesions in DSA images, characterized in that, include: The image receiving module is used to receive the target DSA image; The model prediction module is used to input the target DSA image into a Faster-RCNN network for automatic detection of vascular stenosis lesions, so that after the Faster-RCNN network for automatic detection of vascular stenosis lesions extracts the initial feature map from the target DSA image, it uses the DSA image foreground feature enhancement method as described in claim 1 to obtain the corresponding foreground feature enhancement map, and then determines the location of vascular stenosis and the detection result of vascular stenosis lesions in the target DSA image based on the foreground feature enhancement map.

4. The device for detecting vascular stenosis lesions in DSA images according to claim 3, characterized in that, Also includes: The training set generation module is used to generate a training set based on each DSA historical image and the correspondence between the corresponding vascular stenosis location identifier and the lesion detection result identifier for each DSA historical image. The lesion detection result identifier includes: a first identifier for indicating that a lesion has occurred at the vascular stenosis location and a second identifier for indicating that no lesion has occurred at the vascular stenosis location. The model training module is used to train a preset deep learning model based on the training set using the Faster-RCNN algorithm to obtain the corresponding Faster-RCNN network for automatic detection of vascular stenosis lesions.

5. The device for detecting vascular stenosis lesions in DSA images according to claim 3, characterized in that, The Faster-RCNN network used for automatic detection of vascular stenosis includes: The convolutional layer skeleton is used to extract features from the input DSA image to obtain the corresponding initial feature map; The channel attention module is used to execute the DSA image foreground feature enhancement method to obtain the foreground feature enhancement map corresponding to the DSA image. An RPN network is used to extract candidate regions from the foreground feature enhancement map and obtain the corresponding candidate region location data. A Region of Interest Pooling Network (RPN) is used to extract the foreground feature map corresponding to the candidate region based on the foreground feature enhancement map output by the channel attention module and the candidate region location data output by the RPN network. A classifier is used to determine the location of vascular stenosis and the detection result of vascular stenosis lesions in the target DSA image based on the foreground feature map, using a preset fully connected layer and candidate box regression, wherein the detection result of vascular stenosis lesions is used to indicate whether a lesion has occurred at the location of vascular stenosis.

6. The device for detecting vascular stenosis lesions in DSA images according to claim 5, characterized in that, The convolutional layer skeleton includes a ResNet50 network.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it performs the function of the device for detecting vascular stenosis lesions in DSA images as described in any one of claims 3 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program performs the function of the device for detecting vascular stenosis lesions in DSA images as described in any one of claims 3 to 6.

Images