Image processing method and system for cleaning secretions from a visual double-lumen bronchial catheter.

Abstract

This invention discloses an image processing method and system for cleaning secretions using a visual double-lumen bronchial catheter, relating to the field of deep learning model feature extraction technology. The method includes: acquiring raw feature data of the patient's bronchus in real time through a visual double-lumen bronchial catheter; preprocessing the data to obtain initial feature data; inputting the initial feature data into a multi-target feature decomposition model based on a cross-attention gating mechanism for multi-target feature decomposition to obtain cuff features and secretion features; synthesizing synthetic feature data of the patient's bronchus based on the cuff features and the secretion features, and determining a secretion cleaning strategy. By performing multi-target feature decomposition on the multi-target feature decomposition model based on a cross-attention gating mechanism, high-precision target features are obtained, and then synthetic feature data that is highly consistent with the real environment inside the bronchus is obtained, thereby ensuring the reliability of the secretion cleaning strategy generated based on the synthetic feature data.

Description

Technical Field

[0001] This invention discloses a method and system for image processing of secretions for cleaning a visual dual-lumen bronchial duct, specifically relating to the field of feature extraction technology for deep learning models. Background Technology

[0002] Lung isolation is routinely required during anesthesia for lung-related diagnostic and treatment surgeries to physically isolate the tracheal pathway between the operated lung and the unoperated lung (healthy lung). This allows for the prolonged coexistence of two different states: the operated lung remaining in a state of respiratory arrest while the healthy lung is under normal artificial ventilation. It also prevents the direct dissemination of tumorous or infectious fluids or blood from the operated lung to the healthy lung via the airway during the surgical procedure. Visual double-lumen endotracheal tubes (DBLs) are the most common tool for managing lung isolation airways and are widely used in clinical surgical anesthesia to achieve lung isolation and one-lung ventilation. DBLs are available in left and right lateral versions, physically separating the ventilation pathways of the two lungs at the level of the left / right main bronchus openings. This allows for flexible switching between bilateral lung ventilation, left-sided unilateral lung ventilation, or right-sided unilateral lung ventilation as needed for the surgical anesthesia.

[0003] Normally, when changes occur in the images acquired by a visual double-lumen bronchial catheter, the attending physician or anesthesiologist needs to assess the real-time images to determine if there has been a problem with the catheter insertion. This is to prevent displacement of the catheter, which has been inserted into the left or right main bronchus and is well-aligned, leading to poor mechanical ventilation of the unaffected lung or incomplete closure of the operated lung, causing respiratory problems. However, when the patient has bronchial inflammation or is stimulated by the visual double-lumen bronchial catheter, the bronchi are prone to secretions. This not only easily leads to catheter displacement but also obscures the catheter's position, making it difficult to effectively identify changes in its position, thus affecting the assessment of abnormal catheter placement. Therefore, effectively identifying and cleaning secretions within the field of view of the visual double-lumen bronchial catheter has become an urgent problem to be solved. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an image processing method and system for cleaning secretions in a visual dual-lumen bronchial catheter. This method can obtain high-precision target features by performing multi-target feature decomposition on a multi-target feature decomposition model based on a cross-attention gating mechanism, and then synthesize synthetic feature data that is highly consistent with the real environment inside the bronchus, thereby ensuring the reliability of the secretion cleaning strategy generated based on the synthetic feature data.

[0005] In a first aspect, embodiments of the present invention provide an image processing method for cleaning secretions from a visual double-lumen bronchial catheter, comprising:

[0006] Real-time acquisition of raw feature data within the patient's bronchi using a visual double-lumen bronchial catheter;

[0007] The original feature data is preprocessed to obtain preprocessed initial feature data;

[0008] The initial feature data is input into a multi-target feature splitting model based on cross-attention gating mechanism to split the multi-target features and obtain cuff features and secretion features. The cuff features are the extraction features of the second inflatable cuff in the visual double-lumen bronchial tube.

[0009] Synthetic feature data of the patient's bronchus is synthesized based on the cuff features and the secretion features, in order to determine a secretion cleaning strategy based on the synthetic feature data.

[0010] In some embodiments, the multi-target feature splitting model includes a feature extraction branch corresponding to each target feature. The step of inputting the initial feature data into the multi-target feature splitting model based on a cross-attention gating mechanism to perform multi-target feature splitting to obtain sac features and secretion features includes:

[0011] The initial feature data is input into the cuff feature extraction branch and the secretion feature extraction branch respectively to obtain the cuff features and the secretion features;

[0012] The cuff feature extraction branch is used to extract features from the initial feature data based on the secretion diffusion weight determined layer by layer by the secretion feature extraction branch; the secretion feature extraction branch is used to extract features from the initial feature data based on the contact stress weight determined layer by layer by the cuff feature extraction branch; the contact stress weight is used to characterize the contact stress distribution between the second inflatable cuff and the bronchial wall.

[0013] In some embodiments, the step of extracting features from the initial feature data based on the secretion diffusion weights determined layer by layer by the secretion feature extraction branch includes:

[0014] The secretion feature extraction branch obtains the first intermediate feature map extracted from the i-th feature extraction layer;

[0015] Based on the first intermediate feature map, generate the secretion diffusion weights corresponding to the i-th feature extraction layer;

[0016] Based on the secretion diffusion weight, determine the attention mask for the secretion region;

[0017] The cuff feature extraction branch obtains the attention mask of the secretion region and performs feature extraction on the second intermediate feature map based on the attention mask of the secretion region. The second intermediate feature map is the feature map obtained by the cuff feature extraction branch from the initial feature data in the i-th feature extraction layer.

[0018] In some embodiments, the step of extracting features from the initial feature data based on the contact stress weights determined layer by layer by the feature extraction branch of the sheath includes:

[0019] The feature extraction branch of the bladder obtains the second intermediate feature map extracted in the i-th feature extraction layer;

[0020] Based on the second intermediate feature map, the contact stress weights corresponding to the i-th feature extraction layer are generated;

[0021] Determine the contact stress attention mask based on the contact stress weight;

[0022] The secretion feature extraction branch acquires the contact stress attention mask and performs feature extraction on the first intermediate feature map based on the contact stress attention mask. The first intermediate feature map is the feature map obtained by the secretion feature extraction branch from the initial feature data in the i-th feature extraction layer.

[0023] In some embodiments, the synthesis of synthetic feature data of the patient's bronchus based on the cuff features and the secretion features includes:

[0024] The composite feature data is obtained by synthesizing the features of the sac and the secretion based on the multi-objective consistency loss function.

[0025] In some embodiments, the multi-objective consistency loss function can be:

[0026]

[0027] in, For multi-objective consistency loss, The attention mask for the secretion region output by the secretion feature extraction branch in the last feature extraction layer is used to... The initial feature data, The synthesized feature data.

[0028] In some embodiments, determining the secretion cleansing strategy based on the synthetic feature data includes:

[0029] When there are multiple secretion blocks, the priority score of each secretion block is determined based on the minimum characteristic distance between each secretion block and the edge of the second inflatable bladder, the area of ​​the secretion block, and the average contact stress weight of the corresponding region of the secretion block.

[0030] Cleaning strategies are generated for each of the secretion blocks in the order of priority scoring.

[0031] In some embodiments, the priority of each of the secretion blocks is calculated using the following formula:

[0032]

[0033] in, For secretion area Priority rating, For secretion area The minimum feature distance from the edge of the second inflatable bladder. For secretion area area, For secretion area The average contact stress weight of the corresponding region , , These are the weighting coefficients. This is the distance scaling parameter.

[0034] In some embodiments, preprocessing the original feature data to obtain preprocessed initial feature data includes:

[0035] The original feature data is subjected to adaptive histogram equalization to obtain process-optimized feature data;

[0036] The process optimization feature data is subjected to nonlocal mean denoising processing to obtain the initial feature data.

[0037] In a second aspect, embodiments of the present invention provide an image processing system for cleaning secretions from a visual dual-lumen bronchial catheter, comprising:

[0038] The acquisition module is used to acquire raw feature data of the patient's bronchi in real time through a visual double-lumen bronchial catheter;

[0039] The preprocessing module is used to preprocess the original feature data to obtain preprocessed initial feature data;

[0040] The feature extraction module is used to input the initial feature data into a multi-target feature splitting model based on cross-attention gating mechanism to split the multi-target features and obtain cuff features and secretion features. The cuff features are the extraction features of the second inflatable cuff in the visual double-lumen bronchial tube.

[0041] A synthesis module is used to synthesize synthetic feature data of the patient's bronchus based on the cuff features and the secretion features, so as to determine a secretion cleaning strategy based on the synthetic feature data.

[0042] Thirdly, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the embodiments of the present invention.

[0043] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the embodiments of the present invention.

[0044] Fifthly, embodiments of the present invention provide a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the method described in the embodiments of the present invention.

[0045] This invention proposes an image processing method and system for cleaning secretions using a visual double-lumen bronchial catheter. The method involves real-time acquisition of raw feature data from within the patient's bronchi via the visual double-lumen bronchial catheter; preprocessing the raw feature data to obtain preprocessed initial feature data; inputting the initial feature data into a multi-target feature decomposition model based on a cross-attention gating mechanism to perform multi-target feature decomposition, obtaining cuff features and secretion features, where the cuff features are the extraction features of the second inflatable cuff in the visual double-lumen bronchial catheter; synthesizing synthetic feature data of the patient's bronchi based on the cuff features and the secretion features to determine a secretion cleaning strategy. The initial feature data is decomposed using a multi-target feature decomposition model based on a cross-attention gating mechanism to obtain high-precision cuff features and secretion features, and then a synthesis operation is performed to obtain synthetic feature data that is highly consistent with the actual environment within the bronchi, thereby ensuring the reliability of the secretion cleaning strategy generated based on the synthetic feature data.

[0046] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0047] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0048] Figure 1a This is an image of a problem that occurs during the placement of a dual-lumen endotracheal tube, as described in the prior art.

[0049] Figure 1b This is an image showing a problem that occurred during the insertion of a dual-lumen endotracheal tube using another existing technology.

[0050] Figure 2 This diagram illustrates the implementation environment architecture of the image processing method for cleaning secretions from a visual dual-lumen bronchial catheter provided in an embodiment of the present invention.

[0051] Figure 3a This is a schematic diagram of the structure of a visual double-lumen bronchial tube provided in an embodiment of the present invention;

[0052] Figure 3b This is a front view of a visual double-lumen bronchial tube provided in an embodiment of the present invention;

[0053] Figure 3c for Figure 3b Sectional view along the middle II;

[0054] Figure 3d This is a bottom view of a visual double-lumen bronchial tube provided in an embodiment of the present invention;

[0055] Figure 3e for Figure 3d Sectional view along HH;

[0056] Figure 3f This is a magnified view of the distal end of a visual double-lumen bronchial tube provided in an embodiment of the present invention;

[0057] In the picture:

[0058] 10-Double-lumen bronchial tube, 101-First inflatable cuff, 102-Second inflatable cuff, 103-First lumen, 104-Second lumen, 105-One-way valve, 106-Camera, 107-Suction hole, 108-Nozzle;

[0059] Figure 4 A schematic flowchart of an image processing method for cleaning secretions from a visual double-lumen bronchial catheter, according to an embodiment of the present invention, is shown.

[0060] Figure 5 This diagram illustrates the technical principle of a multi-target feature segmentation model based on a cross-attention gating mechanism provided in an embodiment of the present invention for multi-target feature segmentation.

[0061] Figure 6This diagram illustrates the structure of an image processing system for cleaning secretions from a visual dual-lumen bronchial catheter, according to an embodiment of the present invention.

[0062] Figure 7 A schematic diagram of a computer system suitable for implementing embodiments of the present invention is shown. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0064] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0065] In related technologies, to facilitate the alignment of double-lumen bronchial tubes during placement, visual double-lumen bronchial tubes have been applied in clinical anesthesia. For example, CN202682499U discloses a visual double-lumen bronchial tube, which includes at least two lumens of different lengths relative to the tracheal carina. The lumens selectively communicate with the patient's main trachea or left / right main bronchi at at least two locations within the trachea for ventilation. The tube includes: a first lumen having a distal opening associated with a first inflatable cuff near the carina within the trachea; a second lumen having a distal opening extending distally through the carina and associated with a second inflatable cuff within one of the left and right bronchial branches; and a dedicated image sensor lumen spanning the length of the first lumen and including an image sensor and an illumination source disposed distally near the first lumen, configured to provide images of the tracheal bifurcation point of the tracheal carina, the opening of the left bronchial branch, and the opening of the right bronchial branch. The image signals collected by the image sensor are transmitted to the display screen on the near side of the doctor via wired or wireless means, so that the doctor can observe the image in front of the image sensor in real time.

[0066] Furthermore, such as Figure 1aAs shown, nearly half of the distal second inflatable cuff is positioned above the carina. At this point, the visual double-lumen bronchial tube is inserted too superficially, and there is viscous secretion between the second inflatable cuff and the bronchial wall on the side of the intubation at the carina level. If the traction during surgery increases, even a slight displacement of the visual double-lumen bronchial tube could cause the second inflatable cuff to slip out of the bronchus on that side, resulting in displacement of the visual double-lumen bronchial tube. This could lead to closure of the operated lung, preventing respiratory movement and affecting the surgical procedure. Furthermore, the opening of the main bronchus on the non-operated lung could be blocked or partially blocked by the second inflatable cuff, affecting normal ventilation of the healthy lung. In severe cases, blood oxygen saturation may not be maintained at normal levels. Figure 1b As shown, secretions were present in both the trachea and the main bronchus, making it impossible to obtain a clear image of the distal second inflatable cuff. Therefore, the camera could not determine the insertion depth of the double-lumen endotracheal tube in the bronchus on that side. If this problem is not detected in time by the anesthesiologist, it is highly likely that the patient will experience poor artificial ventilation or decreased blood oxygen saturation, and in severe cases, an accident that endangers the patient's life.

[0067] It should be understood that during surgery, complications such as... Figure 1a or Figure 1b The phenomenon of secretions obscuring the view shown in the image highlights the importance of accurately identifying and cleaning the secretion areas. This is crucial for ensuring the proper identification of abnormal positions of the visual double-lumen endotracheal tube during surgery.

[0068] Based on this, the present invention proposes an image processing method and system for cleaning secretions from a visual dual-lumen bronchial catheter. By performing a series of image processing operations on the original feature data, synthetic feature data that is highly consistent with the real environment inside the bronchus is obtained, thereby ensuring the reliability of the cleaning strategy generated based on the synthetic feature data.

[0069] For the specific implementation environment of the image processing method for cleaning secretions from a visual double-lumen bronchial catheter proposed in this invention, please refer to [link / reference needed]. Figure 2 . Figure 2 This diagram illustrates the implementation environment architecture of the image processing method for cleaning secretions from a visual dual-lumen bronchial catheter provided in an embodiment of the present invention.

[0070] like Figure 2 As shown, the implementation environment architecture includes: a visual dual-lumen bronchial tube 10 and a server 20.

[0071] like Figures 3a-3fThe diagram shows various views, cross-sectional views, and partial enlarged views of the visual dual-lumen bronchial catheter 10 of the present invention. The visual dual-lumen bronchial catheter 10 includes a first lumen 103, a second lumen 104, a first inflatable cuff 101, a second inflatable cuff 102, and a camera 106. The first lumen 103 has a first opening distal end near the tracheal carina and associated with the first inflatable cuff 101. The second lumen 104 has a second opening distal end extending through the carina and associated with the second inflatable cuff 102 within one of the left and right bronchial branches. It also includes an image sensor lumen spanning the length of the first lumen 103. The camera 106 is located distal to the first opening and is used to capture images of the front end of the first opening, transmitting the image signal to a display at the proximal end via a cable or wireless module.

[0072] More preferably, a one-way valve 105 that can communicate with the outside atmosphere is provided between the interface at the proximal end of the first lumen 103 and the ventilator (not shown in the figure). The one-way valve 105 is configured to allow airflow from the affected lung to be discharged to the atmosphere only, and not allow the atmosphere to enter the first lumen 103. Preferably, the one-way valve 105 is a thin film cavity or a latex cavity, and has a slit at the top of the cavity.

[0073] More preferably, at least one adsorption hole 108 is provided near the proximal end of the second inflatable cuff 102. Preferably, a plurality of adsorption holes 108 are evenly spaced along the outer periphery of the catheter. The adsorption holes 108 are connected to an adsorption device (not shown in the figure) at the proximal end through a dedicated adsorption lumen, for adsorbing sputum, blood and other adhering substances on the outer periphery of the catheter.

[0074] Further preferably, the front end of the camera 106 is provided with multiple nozzles 107. The nozzles 107 are connected to a near-end cleaning liquid source (not shown in the figure) through a dedicated cleaning tube. At least one nozzle 107 is designed to spray cleaning agent toward the lens of the camera 106. Further preferably, at least one nozzle 107 is designed to spray cleaning agent toward the far-end suction hole 108, so that the cleaning agent, along with sputum, blood, and other adhering substances, can be promptly and cleanly removed. As a result, the camera 106 can capture clear images from the far end, especially clear images of the second inflatable cuff 102 and its vicinity.

[0075] Server 20 is communicatively connected to camera 106 and adsorption device of visual dual-lumen bronchial tube 10, respectively. Server 20 is used to receive images acquired by image sensor of visual dual-lumen bronchial tube 10, and execute the image processing method for cleaning secretions of visual dual-lumen bronchial tube proposed in this embodiment of the invention to generate a secretion cleaning strategy, and control adsorption device to clean secretions according to the cleaning strategy. For example, the adsorption device is controlled to provide negative pressure to absorb secretions.

[0076] Server 20 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0077] Server 20 and the visual dual-lumen bronchial tube 10 are connected directly or indirectly via wired or wireless communication. Optionally, the aforementioned wireless or wired network uses standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to any combination of Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless network, private network, or virtual private network.

[0078] The image processing method for cleaning secretions from a visual double-lumen bronchial tube proposed in this invention can be implemented by an image processing system for cleaning secretions from a visual double-lumen bronchial tube, which can be installed on a terminal device or a server.

[0079] To further illustrate the technical solutions provided by the embodiments of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of the present invention provide method operation instruction steps as shown in the following embodiments or drawings, the method may include more or fewer operation instruction steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided by the embodiments of the present invention. In actual processing or when the device executes the method, it may be executed sequentially or in parallel according to the method shown in the embodiments or drawings.

[0080] It should be noted that the acquisition or use of data in this embodiment of the invention requires user consent. Data can only be obtained after user authorization, and the acquisition or use of data complies with relevant laws and regulations. Users include, but are not limited to, patients and doctors.

[0081] Please refer to Figure 4 , Figure 4 A schematic flowchart of an image processing method for cleaning secretions from a visual double-lumen bronchial catheter, according to an embodiment of the present invention, is shown. Figure 2 As shown, the method includes:

[0082] Step 401: Real-time acquisition of raw feature data from the patient's bronchi using a visual double-lumen bronchial catheter.

[0083] It should be noted that the original feature data refers to the feature data collected by the visual double-lumen bronchial catheter, including but not limited to color images and infrared images acquired in real time by the camera attached to the visual double-lumen bronchial catheter, which are obtained through convolution processing. In existing technologies, attending physicians or anesthesiologists analyze the original images corresponding to the original feature data to adjust the position of the visual double-lumen bronchial catheter, or to control the adsorption device inside the visual double-lumen bronchial catheter to clean secretions.

[0084] Step 402: Preprocess the original feature data to obtain preprocessed initial feature data.

[0085] It should be noted that the original images used to acquire the raw feature data are obtained from point light sources in a visible double-lumen bronchial tube. This can easily lead to uneven lighting, with the center bright and the edges dark. Furthermore, when secretions are present in the bronchi, these secretions, typically transparent or semi-transparent mucus, can cause significant reflections of the light from the point light source, increasing noise interference in the image and affecting the accuracy of subsequent identification of the second inflatable cuff location based on the raw feature data. Therefore, this invention preprocesses the raw feature data to improve its quality.

[0086] In some embodiments, preprocessing the original feature data to obtain preprocessed initial feature data includes: performing adaptive histogram equalization on the original feature data to obtain process-optimized feature data, and performing nonlocal mean denoising on the process-optimized feature data to obtain initial feature data.

[0087] It should be noted that adaptive histogram equalization divides the original feature data into small blocks, and then performs histogram equalization on each feature block. By eliminating the boundaries between blocks through the difference, it can enhance local contrast while suppressing the amplification of overall background noise.

[0088] For example, the original feature data can be subjected to adaptive histogram equalization using the following formula:

[0089]

[0090] in, To optimize feature data for the process, The original feature data, For low-frequency light components, This is the contrast scaling factor. To limit the contrast-adaptive histogram equalization operation.

[0091] Furthermore, the low-frequency illumination component can be obtained using the following formula. :

[0092]

[0093]

[0094]

[0095] in, For the frequency domain representation of the image, These are the pixel coordinates of the original image. The transfer function is the homomorphic filter. Let (u,v) be the distance from the frequency point (u,v) to the frequency center. The cutoff frequency, To control the filter steepness parameter, For low-frequency gain, This is for high-frequency gain.

[0096] In other words, low-frequency illumination components can be obtained through frequency domain transformation and corresponding analysis before the original feature data is converted from the original image.

[0097] Furthermore, the present invention performs nonlocal mean denoising processing on the process optimization feature data to obtain initial feature data.

[0098] For example, the following formula is used to perform nonlocal mean denoising on the process optimization feature data:

[0099]

[0100]

[0101] in, For initial feature data, To optimize feature data for the process, The search window is centered on pixel i. For filtering weights, and For the neighborhood blocks of pixel i and pixel j, To calculate the squared Euclidean distance between two neighborhood blocks. This represents the filter strength coefficient.

[0102] Therefore, this embodiment of the invention enhances the contrast of the original feature data through adaptive histogram equalization, making the details of the second inflatable bladder and secretions more prominent, while avoiding excessive amplification of noise. Then, a nonlocal mean denoising algorithm is used to denoise the nonlocal information of all pixels in the image, effectively smoothing the noise while preserving important edge and texture information.

[0103] Step 403: Input the initial feature data into the multi-target feature splitting model based on the cross-attention gating mechanism to split the multi-target features and obtain the cuff features and secretion features. The cuff features are the extracted features of the second inflatable cuff in the visible double-lumen bronchial tube.

[0104] In some embodiments, the multi-target feature splitting model includes a feature extraction branch corresponding to each target feature. The initial feature data is input into the multi-target feature splitting model based on the cross-attention gating mechanism to split the multi-target features and obtain the sac features and secretion features. This includes: inputting the initial feature data into the sac feature extraction branch and the secretion feature extraction branch respectively to obtain the sac features and secretion features.

[0105] It should be noted that the multi-objective feature segmentation model based on the cross-attention gating mechanism essentially performs multi-objective feature segmentation on the initial feature data. When extracting features from the initial feature data using the cuff feature extraction branch and the secretion extraction branch respectively, the cuff feature extraction branch considers the influence of secretion diffusion during cuff feature extraction. That is, it uses secretion diffusion weights to suppress the focus on the secretion area during cuff feature extraction, thereby avoiding misidentification of secretions as part of the cuff, making the cuff region in the subsequent synthesized feature data more accurate. Similarly, the secretion extraction branch considers the contact stress between the cuff and the bronchus during secretion feature extraction, thereby effectively avoiding misidentification of bronchial mucosa deformed by cuff compression as secretions, making the extracted secretion features more accurate.

[0106] The cuff feature extraction branch is used to extract features from the initial feature data based on the secretion diffusion weights determined layer by layer by the secretion feature extraction branch. The secretion feature extraction branch is also used to extract features from the initial feature data based on the contact stress weights determined layer by layer by the cuff feature extraction branch. The contact stress weights are used to characterize the contact stress distribution between the second inflatable cuff and the bronchial wall.

[0107] In some embodiments, feature extraction is performed on the initial feature data according to the secretion diffusion weights determined layer by layer by the secretion feature extraction branch, including: the secretion feature extraction branch obtains the first intermediate feature map extracted at the i-th feature extraction layer, generates the secretion diffusion weights corresponding to the i-th feature extraction layer based on the first intermediate feature map, determines the secretion region attention mask according to the secretion diffusion weights, the cyst feature extraction branch obtains the secretion region attention mask, and performs feature extraction on the second intermediate feature map based on the secretion region attention mask.

[0108] Among them, the second intermediate feature map is the feature map obtained by the bladder feature extraction branch in the i-th feature extraction layer from the initial feature data.

[0109] In other embodiments, feature extraction is performed on the initial feature data based on the contact stress weights determined layer by layer by the cuff feature extraction branch. This includes: the cuff feature extraction branch obtaining a second intermediate feature map extracted from the i-th feature extraction layer; generating contact stress weights corresponding to the i-th feature extraction layer based on the second intermediate feature map; determining a contact stress attention mask based on the contact stress weights; the secretion feature extraction branch obtaining the contact stress attention mask; and performing feature extraction on the first intermediate feature map based on the contact stress attention mask. The first intermediate feature map is the feature map obtained by the secretion feature extraction branch from the initial feature data at the i-th feature extraction layer.

[0110] In one specific embodiment, such as Figure 5 As shown, the initial feature data is input to the cuff feature extraction branch and the secretion feature extraction branch, respectively. The cuff feature extraction branch extracts the second intermediate feature map in the first feature extraction layer, and generates the contact stress weight corresponding to the first feature extraction layer based on the second intermediate feature map. It then determines the contact stress attention mask based on the contact stress weight and sends the contact stress attention mask to the secretion extraction branch. The secretion extraction branch obtains the contact stress attention mask and performs feature extraction on the first intermediate feature map extracted in the first feature extraction layer in the second feature extraction layer based on the contact stress attention mask.

[0111] Accordingly, the secretion extraction branch extracts a first intermediate feature map in the first feature extraction layer, generates a secretion diffusion weight corresponding to the first feature extraction layer based on the first intermediate feature map, determines the secretion region attention mask based on the secretion diffusion weight, and then sends the secretion region attention mask to the capsule extraction branch. The capsule extraction branch obtains the secretion region attention mask and performs feature extraction on the second intermediate feature map extracted by the first feature extraction layer based on the secretion region attention mask in the second feature extraction layer.

[0112] Similarly, the sac extraction branch and the secretion extraction branch perform the above steps based on the first intermediate feature map and the second intermediate feature map extracted by the second feature extraction layer until the sac extraction branch and the secretion extraction branch end.

[0113] It should be noted that in the embodiments of the present invention, the number of feature extraction layers in the sac extraction branch and the secretion extraction branch is the same, and the convolution structure of the feature extraction layer can be the same or different. The present invention does not make any specific limitation.

[0114] For example, the expression for the capsule extraction branch at the (i+1)th feature extraction layer can be:

[0115]

[0116] in, The second intermediate feature map obtained by the feature extraction branch of the capsule at the (i+1)th feature extraction layer. The second intermediate feature map obtained by the feature extraction branch of the capsule at the i-th feature extraction layer. The first intermediate feature map obtained by the secretion feature extraction branch at the i-th feature extraction layer. This is the secretion diffusion weighting transformation function.

[0117] Therefore, this embodiment of the invention can utilize a cuff feature extraction branch and a secretion feature extraction branch to extract cuff and secretion features separately. This allows the feature extraction branch to highly focus on the feature types it is interested in, improving the accuracy of cuff and secretion feature extraction and reducing error interference from multi-target extraction. Simultaneously, this invention also utilizes a cross-attention gating mechanism to indirectly use the analysis results of another feature extraction branch to correct the current feature extraction branch during the extraction process. This ensures reasonable attention to other targets within a single feature extraction branch, further improving the accuracy of feature extraction and providing reliable feature data for the subsequent generation of synthetic feature data.

[0118] Step 404: Synthetic feature data of the patient's bronchus is synthesized based on the cuff features and secretion features to determine a secretion cleaning strategy based on the synthetic feature data.

[0119] It should be noted that synthesizing feature data using feature data is an existing technology, and this invention does not impose any specific limitations on it.

[0120] Preferably, in the process of synthesizing feature data, a multi-objective consistency loss function is added, that is, the features of the scabbard and the secretion are synthesized based on the multi-objective consistency loss function to obtain the synthesized feature data.

[0121] For example, the multi-objective consistency loss function can be expressed as follows:

[0122]

[0123] in, For multi-objective consistency loss, The attention mask for the secretion region output by the secretion feature extraction branch in the last feature extraction layer is used to... The initial feature data, The synthesized feature data.

[0124] Therefore, the embodiments of the present invention utilize the above-mentioned series of image processing processes to obtain synthetic feature data that closely approximates the real environment based on the cuff features and secretion features in the initial feature data, effectively improving the reliability of the images generated for the secretion cleaning strategy. At the same time, it effectively reduces the impact of noise interference from the original feature data or initial feature data on the abnormal identification of the second inflatable cuff position.

[0125] Furthermore, in a feasible embodiment, determining the secretion cleaning strategy based on the synthetic feature data includes: when there are multiple secretion blocks, determining the priority score of each secretion block based on the minimum feature distance between each secretion block and the edge of the second inflatable cuff, the area of ​​the secretion block, and the average contact stress weight of the area corresponding to the secretion block; and generating a corresponding cleaning strategy for each secretion block in order of priority score.

[0126] The secretion cleaning strategy may include, but is not limited to, the magnitude and duration of suction force generated on the adsorbent, which can be determined according to the type of adsorption device set in the visible dual-lumen bronchial tube. This invention does not impose specific limitations.

[0127] In one specific embodiment, the priority of each secretion block can be calculated using the following formula:

[0128]

[0129] in, For the k-th secretion block Priority rating, For secretion area The minimum feature distance from the edge of the second inflatable cuff. For secretion area area, For secretion area The average contact stress weight of the corresponding region , , These are the weighting coefficients. This is the distance scaling parameter.

[0130] It should be understood that by adding a contact stress weight-related item to the priority score, the present invention can optimize the scoring of secretion blocks according to the location of the secretions, thereby significantly improving the priority score of the high contact stress area (the contact position between the second inflatable cuff and the bronchial wall), thereby improving the cleaning order of some secretions in the contact stress area and reducing the sliding of the second inflatable cuff in the patient's bronchus due to the influence of secretions in the high contact stress area.

[0131] It should be noted that although the operation of the method of the present invention is described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed in order to achieve the desired result.

[0132] Figure 6 A schematic diagram of the structure of an image processing system for cleaning secretions from a visual dual-lumen bronchial catheter, according to an embodiment of the present invention, is shown.

[0133] like Figure 6 As shown, the image processing system 10 for cleaning secretions from a visual dual-lumen bronchial catheter includes:

[0134] The acquisition module 11 is used to acquire raw feature data of the patient's bronchus in real time through a visual double-lumen bronchial catheter;

[0135] Preprocessing module 12 is used to preprocess the original feature data to obtain preprocessed initial feature data;

[0136] Feature extraction module 13 is used to input the initial feature data into a multi-target feature splitting model based on cross-attention gating mechanism to split the multi-target features and obtain cuff features and secretion features. The cuff features are the extraction features of the second inflatable cuff in the visual double-lumen bronchial tube.

[0137] Synthesis module 14 is used to synthesize synthetic feature data of the patient's bronchus based on the cuff features and the secretion features, so as to determine the secretion cleaning strategy according to the synthetic feature data.

[0138] It should be understood that the modules or modules described in the image processing system 10 for cleaning secretions from a visual double-lumen bronchial catheter are similar to those in the reference. Figure 4The steps in the described method correspond accordingly. Therefore, the operations and features described above for the method are also applicable to the image processing system 10 for cleaning secretions from a visual dual-lumen bronchial tube and its included modules, and will not be repeated here. The image processing system 10 for cleaning secretions from a visual dual-lumen bronchial tube can be pre-implanted in the browser or other secure applications of an electronic device, or can be loaded into the browser or its secure applications of an electronic device by downloading. The corresponding modules in the image processing system 10 for cleaning secretions from a visual dual-lumen bronchial tube can cooperate with modules in the electronic device to implement the solutions of the embodiments of the present invention.

[0139] The division of modules or units mentioned in the detailed description above is not mandatory. In fact, according to the embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0140] The following is for reference. Figure 7 , Figure 7 A schematic diagram of a computer system suitable for implementing embodiments of the present invention is shown.

[0141] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. RAM 703 also stores various programs and data required for the system's operating instructions. The CPU 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0142] The following components are connected to I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 710 as needed so that computer programs read from it can be installed into storage section 708 as needed.

[0143] In particular, according to embodiments of the present invention, the above-described flowchart is referenced. Figure 2 The described process can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program contains program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined in the system of the present invention.

[0144] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operational instructions of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two connected blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operational instructions, or using a combination of dedicated hardware and computer instructions.

[0146] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not assembled into the electronic device. The computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the image processing method for cleaning secretions from a visual dual-lumen bronchial catheter described in the present invention.

[0147] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. An image processing method for secretion cleaning of a visual double-lumen bronchial tube, characterized by, The method comprises: Real-time acquisition of original feature data in the patient's bronchus through a visual double-lumen bronchial catheter; Preprocessing the original feature data to obtain preprocessed initial feature data; Inputting the initial feature data into a multi-target feature splitting model based on a cross-attention gate mechanism for multi-target feature splitting to obtain cuff features and secretion features, the cuff features being extracted features of a second inflatable cuff in the visual double-lumen bronchial catheter; Synthesizing the cuff features and the secretion features to obtain synthesized feature data in the patient's bronchus, so as to determine a secretion cleaning strategy according to the synthesized feature data; The multi-target feature splitting model comprises a feature extraction branch corresponding to each target feature, and the multi-target feature splitting comprises: Inputting the initial feature data into the cuff feature extraction branch and the secretion feature extraction branch respectively to obtain the cuff features and the secretion features; The cuff feature extraction branch is used for feature extraction on the initial feature data according to the secretion diffusion weight determined by the secretion feature extraction branch layer by layer; the secretion feature extraction branch is used for feature extraction on the initial feature data according to the contact stress weight determined by the cuff feature extraction branch layer by layer; the contact stress weight is used for representing the contact stress distribution between the second inflatable cuff and the bronchial wall; and The cuff features and the secretion features are synthesized based on a multi-target consistency loss function to obtain the synthesized feature data; The multi-target consistency loss function can be: wherein, is a multi-target consistency loss, is a secretion feature extraction branch outputting a secretion region attention mask at the last feature extraction layer, is the initial feature data, is the synthesized feature data.

2. The image processing method for secretions cleaning of a visual dual-lumen bronchial tube according to claim 1, characterized in that, The feature extraction on the initial feature data according to the secretion diffusion weight determined by the secretion feature extraction branch layer by layer comprises: The secretion feature extraction branch obtains a first intermediate feature map extracted at an i-th feature extraction layer; Based on the first intermediate feature map, a secretion diffusion weight corresponding to the i-th feature extraction layer is generated; According to the secretion diffusion weight, a secretion region attention mask is determined; The cuff feature extraction branch obtains the secretion region attention mask and performs feature extraction on a second intermediate feature map based on the secretion region attention mask, the second intermediate feature map being a feature map obtained by performing feature extraction on the initial feature data at the i-th feature extraction layer of the cuff feature extraction branch.

3. The image processing method for secretions cleaning of a visual dual-lumen bronchial tube catheter according to claim 1, characterized in that, The feature extraction on the initial feature data according to the contact stress weight determined by the cuff feature extraction branch layer by layer comprises: The cuff feature extraction branch obtains a second intermediate feature map extracted at an i-th feature extraction layer; Based on the second intermediate feature map, a contact stress weight corresponding to the i-th feature extraction layer is generated; According to the contact stress weight, a contact stress attention mask is determined; The secretion feature extraction branch acquires the contact stress attention mask, and performs feature extraction on a first intermediate feature map based on the contact stress attention mask. The first intermediate feature map is a feature map obtained by performing feature extraction on the initial feature data by the secretion feature extraction branch at an i-th feature extraction layer.

4. The image processing method for secretions cleaning of a visual dual-lumen bronchial tube catheter according to claim 1, characterized in that, The secretion cleaning strategy is determined according to the synthesized feature data, including: When the secretion blocks are multiple, a priority score of each secretion block is determined according to a minimum feature distance between each secretion block and an edge of the second inflatable cuff, an area of the secretion block, and an average contact stress weight of a corresponding area of the secretion block; A corresponding cleaning strategy is generated for each secretion block in order of priority score.

5. The image processing method for secretions cleaning of a visual dual-lumen bronchial tube catheter according to claim 1, characterized in that, The preprocessing of the original feature data to obtain the preprocessed initial feature data includes: The original feature data is subjected to adaptive histogram equalization processing to obtain process optimization feature data; The process optimization feature data is subjected to non-local mean denoising processing to obtain the initial feature data.

6. An image processing system for visualizing secretion cleaning of a double-lumen bronchial tube, characterized by It includes: The acquisition module is configured to acquire original feature data in a patient's bronchus in real time through a visual double-lumen bronchial catheter; The preprocessing module is configured to preprocess the original feature data to obtain preprocessed initial feature data; The feature extraction module is configured to input the initial feature data into a multi-target feature splitting model based on a cross-attention gate mechanism to perform multi-target feature splitting, thereby obtaining a cuff feature and a secretion feature. The cuff feature is an extracted feature of a second inflatable cuff in the visual double-lumen bronchial catheter; The synthesis module is configured to synthesize synthesized feature data in the patient's bronchus based on the cuff feature and the secretion feature, and to determine a secretion cleaning strategy according to the synthesized feature data; The feature extraction module is specifically configured to input the initial feature data into a cuff feature extraction branch and a secretion feature extraction branch, respectively, to obtain the cuff feature and the secretion feature. The cuff feature extraction branch is configured to perform feature extraction on the initial feature data according to secretion diffusion weights determined by the secretion feature extraction branch layer by layer; the secretion feature extraction branch is configured to perform feature extraction on the initial feature data according to contact stress weights determined by the cuff feature extraction branch layer by layer; the contact stress weight is used to represent the contact stress distribution between the second inflatable cuff and the bronchial wall; and The synthesis module is specifically configured to synthesize the cuff feature and the secretion feature based on a multi-target consistency loss function to obtain the synthesized feature data. The multi-target consistency loss function can be: wherein, is a multi-object consistency loss, is a secretion feature extraction branch outputting a secretion region attention mask at the last feature extraction layer, is the initial feature data, is the synthetic feature data.

Images