Bronchial image data automatic acquisition and storage method and system based on voice interaction
By automatically acquiring and storing bronchoscopic images through voice interaction technology, the problem of reliance on manual operation for image acquisition in existing technologies has been solved, achieving efficient and low-cost image processing and storage while maintaining image clarity and color accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI PROVINCIAL CHEST HOSPITAL (TUBERCULOSIS PREVENTION & CONTROL INST)
- Filing Date
- 2026-04-15
- Publication Date
- 2026-07-10
AI Technical Summary
In current bronchoscopy procedures, image acquisition, processing, and storage rely on manual operation, resulting in insufficient automation and intelligence, leading to low acquisition efficiency and high labor costs.
An automatic acquisition and storage method for bronchial imaging data based on voice interaction is adopted. The voice signal issued by the operator is converted into a text sequence through a voice recognition model, and word segmentation and keyword extraction are performed to match control commands. The real-time video stream is then processed and stored using an image enhancement model.
It enables automated acquisition, annotation, and storage of bronchoscopic images, reducing manual intervention, improving acquisition efficiency, lowering labor costs, and maintaining natural color reproduction while enhancing brightness and clarity.
Smart Images
Figure CN122369458A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent voice interaction technology, specifically to a method and system for automatic acquisition and storage of bronchial imaging data based on voice interaction. Background Technology
[0002] Bronchoscopy is an important means of diagnosing and intervening in respiratory diseases. In clinical practice, it is usually necessary to observe the bronchial structure, lesion morphology and operation process in real time, and to collect, label and store key images for subsequent diagnostic analysis, case archiving and teaching research.
[0003] Current methods for acquiring bronchial images largely rely on manual operation via buttons or touch controls. Typically, a clinical workflow requires two people: one operator performs the bronchoscopy, while the other handles computer operations, including starting and stopping recording, taking screenshots, and saving image data. While this division of labor reduces the workload of the primary operator to some extent, the reliance on manual operation for image acquisition, annotation, and storage increases labor costs, and the level of automation and intelligence is limited, resulting in low acquisition efficiency. Summary of the Invention
[0004] The purpose of this invention is to address the problem mentioned in the background art that the image acquisition, processing and storage process during bronchoscopy relies on manual operation and lacks automation and intelligence. Therefore, this invention proposes an automatic acquisition and storage method and system for bronchial image data based on voice interaction.
[0005] A first aspect of this invention provides a method for automatic acquisition and storage of bronchial imaging data based on voice interaction, the method comprising: Collect the voice signals emitted by the operator; The speech signal is input into a pre-trained speech recognition model, which outputs the corresponding text sequence. The text sequence is segmented and keywords are extracted to obtain instruction keywords; The command keywords are matched with a preset command template library to determine the corresponding control commands; the control commands include image acquisition type commands and / or image annotation commands. The real-time video stream is processed according to the control instructions, and the processing results are stored in a preset bronchial image database.
[0006] By implementing this technical solution, the acquisition, annotation, and storage of bronchoscopic images have been automated, reducing the need for manual intervention and operator collaboration, and improving clinical acquisition efficiency.
[0007] Optionally, the image acquisition type command includes a screenshot command, which triggers a screenshot and enhancement processing procedure; the enhancement processing procedure is implemented through a pre-trained image enhancement model; the image enhancement model includes: The color space conversion module is used to convert the input image to the YUV color space to obtain the Y channel image, U channel image and V channel image; The brightness processing branch processes the Y-channel image to obtain the brightness-perceived features F. Y ; The chroma processing branch is used to process the U-channel and V-channel images to obtain the chroma-perceptual features F. UV ; A fusion-generating network is used to process the luminance-perceived features and the chrominance-perceived features to obtain an enhanced YUV image; The color space inverse conversion module is used to convert the YUV image back to RGB space to obtain the final enhanced image.
[0008] By implementing this technical solution, while improving brightness, contrast, and detail clarity, it is possible to maintain natural color reproduction and improve the color distortion problem that may be caused by traditional brightness enhancement.
[0009] Optionally, the brightness processing branch includes an initial convolutional layer, a global feature extraction module, a local feature extraction module, an adaptive fusion module, and a first branch supervision module, wherein: The initial convolutional layer uses a 1×1 convolution to process the input Y-channel image, expanding the channel dimension to obtain the feature map Y1; The global feature extraction module uses two Swing Transformer blocks to extract global features from feature map Y1 to obtain global feature Y2. The local feature extraction module uses four residual blocks to extract features from the feature map Y1 to obtain local features Y3. The adaptive fusion module performs a weighted summation of the global feature Y2 and the local feature Y3 to obtain the feature map Y4. The first branch supervision module uses a residual attention mechanism to fuse the input Y-channel image and feature map Y4 to obtain the brightness perception feature F. Y .
[0010] By implementing this technical solution, multi-scale modeling and adaptive fusion mechanism based on global and local feature extraction are used to achieve fine enhancement of brightness information, while preserving image structure and texture details, improving the stability and robustness of the brightness enhancement effect, and providing reliable brightness features for subsequent chromaticity fusion and final image enhancement.
[0011] Optionally, the calculation process of the first branch supervision module includes: A 3×3 convolution layer is used to process the feature map Y4 to compress the channel dimension, resulting in the feature map F1. After adding feature map F1 element by element to the Y channel image, attention weight map A1 is generated by a 3×3 convolution and a sigmoid function. A 3×3 convolution layer is used to process feature map Y4 to obtain feature map F2; The feature map F2 is weighted using the attention weight map A1 to obtain the feature map F3; A 3×3 convolution layer is used to process the input Y-channel image to expand the channel dimension and obtain the feature map F4. Adding feature maps F3, F4, and Y4 element by element yields the brightness perception feature F. Y .
[0012] By implementing this technical solution, the residual attention mechanism is used to fuse the original brightness information with the enhanced features, so as to maintain the consistency of the original structure during the brightness enhancement process, effectively suppress noise and over-enhancement, and improve the reliability and stability of the brightness perception features.
[0013] Optionally, the chroma processing branch includes a first feature denoising module, a second feature denoising module, a detail enhancement module, and a second branch supervision module; wherein: The first feature denoising module and the second feature denoising module have the same structure. They process the U-channel image and the V-channel image in parallel using multi-layer convolution and attention mechanisms to obtain feature maps U1 and V1. U1 and V1 are then concatenated along the channels to obtain feature map T1. The detail enhancement module uses discrete wavelet transform to convert feature map T1 into frequency domain features, and uses residual channel attention module to enhance the frequency domain features. The enhanced frequency domain features are then reconstructed back into the spatial domain to obtain feature map T2. The second branch supervision module uses a residual attention mechanism to fuse the input UV channel image and feature map T2 to obtain the chromaticity perception feature F. UV The second branch supervision module has the same structure as the first branch supervision module.
[0014] By implementing this technical solution, multi-scale denoising and frequency domain enhancement are performed on chromaticity information. At the same time, the original color structure is maintained by combining a supervision module, so as to achieve natural color restoration and detail enhancement, avoid color shift problems caused by brightness enhancement, and improve the color accuracy of bronchoscopic images.
[0015] Optionally, the computation process of any feature denoising module includes: The input image is processed using four 3×3 convolution layers to obtain multi-scale feature maps X1, X2, X3, and X4; the stride of the convolution from the second to the fourth layer is 2. A multi-head attention mechanism is used to process feature map X4 and model global dependencies to obtain feature map X5; Feature map X6 is obtained by upsampling feature maps X1, X2, X3 and X5 element by element; X6 is processed by a 3×3 convolution layer and residual connections are performed to obtain feature map X7; The feature map X7 is processed by two 3×3 convolution layers to obtain the denoised feature map X8, and X8 is output.
[0016] By implementing this technical solution, the combination of multi-layer convolution and multi-head attention mechanism can effectively suppress noise interference while taking into account both local details and global structure, making the chroma features more stable and providing reliable input for subsequent detail enhancement and fusion processing.
[0017] Optionally, the computation process of the detail enhancement module includes: A three-level discrete wavelet transform is performed on the input feature map T1 to obtain the sub-bands of each level; The residual channel attention module is used to process each sub-band to obtain the enhanced frequency domain sub-band; Performing an inverse discrete wavelet transform on the frequency domain subband of the third-level enhancement yields the enhanced subband. ; For the enhanced subband The enhanced subband is obtained by performing an inverse discrete wavelet transform on the frequency domain subband of the second-level enhancement. ; For the enhanced subband The spatial domain feature T0 is obtained by performing inverse discrete wavelet transform on the frequency domain sub-band of the first-level enhancement. The spatial domain feature T0 is processed using the residual channel attention module to obtain the feature map T1.
[0018] By implementing this technical solution, chromaticity features are introduced into the frequency domain for multi-level modeling, and key frequency bands are enhanced through residual channel attention, thereby improving edge and texture details.
[0019] Optionally, the fusion generation network includes a fusion module and a generation module; wherein: The fusion module employs a multi-level attention mechanism to integrate the brightness perception features F Y and color perception features F UV Interactive fusion yields feature map Z1; The generation module uses a multi-layered cascaded 3×3 convolutional layer to extract features from feature map Z1 to obtain feature map Z2. Then, a single 3×3 convolutional layer and a sigmoid function are used to compress the number of channels in feature map Z2 to 3, and inverse normalization is performed to obtain an enhanced YUV image.
[0020] Optionally, the calculation process of the fusion module includes: Brightness perception feature F Y and color perception features F UV The feature map S is obtained by adjusting the number of channels through a 3×3 convolution layer. Y and S UV ; Feature map S Y and S UV Adding each element together, we obtain the initial fusion feature S1; Channel attention and spatial attention are performed in parallel on the initial fusion feature S1 to obtain channel attention feature S2 and spatial attention feature S3; The channel attention feature S2 and spatial attention feature S3 are combined with the initial fusion feature S1, and independent weights are assigned to each pixel position through pixel attention to obtain the channel attention weight W1 and spatial attention weight W2. The channel attention weight W1 and spatial attention weight W2 are dynamically fused through learnable parameters to obtain the final refined attention weight W3; Attention weight W3 is used to adjust the contribution ratio of luminance and chrominance features, and combined with residual connections, the final fused feature Z1 is obtained.
[0021] By implementing this technical solution, brightness perception features and chromaticity perception features are finely integrated through a multi-level attention mechanism to achieve an adaptive balance between brightness and chromaticity, highlighting key structures while maintaining global color consistency and the integrity of local details.
[0022] A second aspect of this invention provides an automatic acquisition and storage system for bronchial imaging data based on voice interaction, the system comprising: The voice acquisition module is used to acquire the voice signals emitted by the operator. The speech recognition module is used to input the speech signal into a pre-trained speech recognition model and output the corresponding text sequence; The semantic analysis module is used to segment and extract keywords from the text sequence to obtain instruction keywords; The instruction matching module is used to match the instruction keywords with a preset instruction template library to determine the corresponding control instructions; the control instructions include image acquisition type instructions and / or image annotation instructions; The instruction execution module is used to process the real-time video stream according to the control instructions and store the processing results in a preset bronchial image database. Attached Figure Description
[0023] Figure 1 A flowchart illustrating an automatic acquisition and storage method for bronchial imaging data based on voice interaction, provided as an embodiment of the present invention; Figure 2 A network architecture diagram of an image enhancement model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a brightness processing branch provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a chroma processing branch provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a feature denoising module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a detail enhancement module provided in an embodiment of the present invention; Figure 7 A schematic diagram of the structure of a monitoring module provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of a fusion generative network provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a fusion module provided in an embodiment of the present invention. Detailed Implementation
[0024] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features and effects of the present invention is provided in conjunction with the accompanying drawings and preferred embodiments.
[0025] This invention provides a method for automatic acquisition and storage of bronchial imaging data based on voice interaction. See also... Figure 1 , Figure 1 This is a flowchart illustrating an automatic acquisition and storage method for bronchial imaging data based on voice interaction, provided as an embodiment of the present invention. The method includes the following steps: S1 collects the voice signals emitted by the operator.
[0026] S2 inputs the speech signal into the pre-trained speech recognition model and outputs the corresponding text sequence.
[0027] S3 performs word segmentation and keyword extraction on the text sequence to obtain instruction keywords.
[0028] S4 matches the instruction keywords with the preset instruction template library to determine the corresponding control instruction.
[0029] S5 processes the real-time video stream according to control commands and stores the processing results in a preset bronchial image database.
[0030] The present invention provides an automatic acquisition and storage method for bronchial image data based on voice interaction. By introducing voice interaction, the method realizes automated control of the image acquisition, annotation and storage process, reduces reliance on manual button or touch operation, and enables the main operator to complete the automatic acquisition of images when issuing instructions. This reduces reliance on other operators, effectively saves labor costs and improves overall efficiency.
[0031] In one implementation, the speech recognition model can be an end-to-end speech recognition model based on deep learning, such as the Conformer speech recognition model or DeepSpeech.
[0032] In one implementation, the control commands include image acquisition type commands and image annotation commands. Image acquisition type commands include start recording, stop recording, and screenshot. Image annotation commands include lesion type annotation, anatomical structure annotation, examination stage annotation, and remarks information annotation.
[0033] In one embodiment, a screenshot command triggers a screenshot and enhancement process. The enhancement process is implemented using a pre-trained image enhancement model. See also Figure 2 , Figure 2 This is a network architecture diagram of an image enhancement model provided in an embodiment of the present invention. The image enhancement model includes: The color space conversion module is used to convert the input image to the YUV color space to obtain the Y channel image, U channel image, and V channel image.
[0034] The brightness processing branch processes the Y-channel image to obtain the brightness-perceived features F. Y .
[0035] The chroma processing branch is used to process the U-channel and V-channel images to obtain the chroma-perceptual features F. UV .
[0036] A fusion-generative network is used to process luminance-perceived features and chrominance-perceived features to obtain an enhanced YUV image.
[0037] The color space inverse conversion module is used to convert YUV images back to RGB space to obtain the final enhanced image.
[0038] This embodiment automatically performs image enhancement processing after a screenshot command is triggered, enabling real-time quality optimization of acquired bronchoscopic images without increasing the operator's workload. The image enhancement model proposed in this embodiment converts the image to the YUV color space and performs separate modeling and collaborative processing of the luminance and chrominance components. While enhancing luminance information, it maintains relative stability of chrominance information, effectively solving the color distortion problem easily caused in traditional luminance enhancement processes. Therefore, while improving image clarity and contrast, it ensures the accuracy and consistency of color reproduction in bronchoscopic images.
[0039] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of a brightness processing branch provided in an embodiment of the present invention. The brightness processing branch includes an initial convolutional layer, a global feature extraction module, a local feature extraction module, an adaptive fusion module, and a first branch supervision module, wherein: The initial convolutional layer uses a 1×1 convolution to process the input Y-channel image, expanding the channel dimension to obtain the feature map Y1.
[0040] The global feature extraction module uses two Swin Transformer blocks to extract global features from feature map Y1, resulting in global features Y2. Each Swin Transformer block uses 11 attention heads.
[0041] The local feature extraction module uses four residual blocks to extract features from the feature map Y1, resulting in local features Y3. Each residual block consists of two 3×3 convolutional layers.
[0042] The adaptive fusion module weighted sums of global feature Y2 and local feature Y3 to obtain feature map Y4. . These are learnable parameters.
[0043] The first branch supervision module uses a residual attention mechanism to fuse the input Y-channel image and feature map Y4 to obtain the brightness perception feature F. Y .
[0044] This embodiment achieves multi-scale, global and local collaborative modeling of brightness information by simultaneously introducing a global feature extraction module based on Swin Transformer and a local feature extraction module based on residual blocks into the brightness processing branch. This enables the model to capture the overall brightness distribution in bronchoscopic images while effectively preserving fine structural and textural details. The adaptive fusion module dynamically balances global and local features through learnable weights to avoid over-enhancement or loss of details. Combined with the residual attention mechanism of the supervision module, the consistency of the original brightness structure is maintained while enhancing brightness, thereby significantly improving the stability and robustness of the brightness enhancement effect and providing high-quality brightness-perceived features for subsequent chromaticity fusion and final image enhancement.
[0045] In one embodiment, see Figure 4 , Figure 4 This is a schematic diagram of a chroma processing branch provided in an embodiment of the present invention. The chroma processing branch includes a first feature denoising module, a second feature denoising module, a detail enhancement module, and a second branch supervision module; wherein: The first and second feature denoising modules have the same structure. They process the U-channel and V-channel images in parallel using multi-layer convolution and attention mechanisms to obtain feature maps U1 and V1. U1 and V1 are then concatenated along the channels to obtain feature map T1. For details, see [link to documentation]. Figure 5 , Figure 5 This is a schematic diagram of a feature denoising module provided in an embodiment of the present invention. The operation process of the feature denoising module includes: Step 1: Process the input image using four 3×3 convolutional layers to obtain multi-scale feature maps X1, X2, X3, and X4. The stride of the convolutions from the second to the fourth layer is 2 for downsampling. Unless otherwise specified, the stride of the convolutional layers is 1 by default.
[0046] Step two: Multi-head attention (MHSA) is used to process feature map X4, modeling global dependencies to obtain feature map X5. MHSA is a common self-attention mechanism, which will not be elaborated upon here.
[0047] Step 3: Add feature maps X1, X2, X3, and X5 element-wise by upsampling to obtain feature map X6. Specifically, feature maps X2, X3, and X5 are upsampled to the spatial resolution of X1 using transposed convolution.
[0048] Step 4: Process X6 using a 3×3 convolution layer and perform residual connections to obtain feature map X7.
[0049] Step 5: Process the feature map X7 using two 3×3 convolution layers to obtain the denoised feature map X8, and output X8.
[0050] The detail enhancement module uses Discrete Wavelet Transform (DWT) to convert feature map T1 into frequency domain features, and then uses the Residual Channel Attention (RCAB) module to enhance the frequency domain features. The enhanced frequency domain features are then reconstructed back into the spatial domain to obtain feature map T2. For details, see [link to documentation]. Figure 6 , Figure 6 This is a schematic diagram of a detail enhancement module provided in an embodiment of the present invention. The operation process of the detail enhancement module includes: Step 1: Perform a three-level discrete wavelet transform (DWT) on the input feature map T1 to obtain the sub-bands (LL) of each level. i LH i HL i HH i ).
[0051] Step two involves using a residual channel attention module to process each sub-band, resulting in enhanced frequency domain sub-bands. The RCAB module is a well-known basic module that integrates channel attention (CA) and residual concepts, and will not be elaborated upon here.
[0052] Step 3, for the third-level enhanced frequency domain subband ( Perform inverse discrete wavelet transform (IDWT) to obtain the enhanced subband. .
[0053] Step 4, strengthen the subband Second-level enhanced frequency domain subband ( Perform inverse discrete wavelet transform to obtain the enhanced subband. .
[0054] Step 5, strengthen the subband and the first-level enhanced frequency domain subband ( Perform inverse discrete wavelet transform to obtain spatial domain feature T0.
[0055] Step 6: Process the spatial domain feature T0 using the residual channel attention module to obtain the feature map T1.
[0056] The second branch supervision module uses a residual attention mechanism to fuse the input UV channel image and feature map T2 to obtain the chroma perception feature F. UV .
[0057] This embodiment effectively suppresses noise interference in the chromaticity component of bronchoscopic images by setting a feature denoising module for the UV channel, and balances local details and global color consistency through the synergistic effect of multi-scale convolution and multi-head attention mechanism. The detail enhancement module based on discrete wavelet transform introduces chromaticity features into the frequency domain for hierarchical modeling, and combines it with the residual channel attention module to target and enhance key high-frequency and low-frequency sub-bands, effectively enhancing color details and edge information during the reconstruction back to the spatial domain. Combined with the residual attention fusion method of the supervision module, the original color structure is kept stable while improving chromaticity performance, thereby avoiding color shift and over-enhancement problems and significantly improving the naturalness of color reproduction in bronchoscopic images.
[0058] In one implementation, the second branch supervision module has the same structure as the first branch supervision module. See also... Figure 7 , Figure 7 This is a schematic diagram of a supervision module provided in an embodiment of the present invention. The operation process of any branch supervision module includes: Step 1: Process the input feature map F0 (F0 is Y4 or T2) using a 3×3 convolution layer to compress the channel dimensions and obtain the feature map F1; Step 2: After element-wise addition of feature map F1 and channel image CC (CC is either Y channel image or UV channel image), attention weight map A1 is generated by passing a 3×3 convolution and sigmoid function. Step 3: Process feature map F0 using a 3×3 convolution layer to obtain feature map F2; Step 4: Use attention weight map A1 to weight feature map F2 to obtain feature map F3; Step 5: Process the input channel image CC using a 3×3 convolution layer to expand the channel dimension and obtain the feature map F4; Step 6: Add feature maps F3, F4, and F0 element by element to obtain the perceptual feature F5 (F5 is the brightness perceptual feature F). Y or color perception feature F UV ).
[0059] This supervision module generates attention weights using the original channel images and adaptively modulates the feature maps, highlighting key brightness or chromaticity information while suppressing irrelevant noise. Through multiple residual connections, it fuses the enhanced features, original features, and channel mapping results, effectively maintaining the consistency of the original structure and color distribution, avoiding artifacts and distortions introduced by over-enhancement, thereby improving the stability and reliability of brightness and chromaticity perception features.
[0060] In one embodiment, see Figure 8 , Figure 8This is a schematic diagram of a fusion-generative network provided in an embodiment of the present invention. The fusion-generative network includes a fusion module and a generation module; wherein: The fusion module employs a multi-level attention mechanism to integrate the brightness perception features F Y and color perception features F UV Interactive fusion yields feature map Z1. For details, see [link to documentation]. Figure 9 , Figure 9 This is a schematic diagram of a fusion module provided in an embodiment of the present invention. The operation process of the fusion module includes: Step 1: Assign brightness perception features F Y and color perception features F UV The feature map S is obtained by adjusting the number of channels through a 3×3 convolution layer. Y and S UV .
[0061] Step 2, transfer feature map S Y and S UV Adding each element together yields the initial fusion feature S1.
[0062] Step 3: Perform channel attention (CA) and spatial attention (SA) in parallel on the initial fused feature S1 to obtain channel attention feature S2 and spatial attention feature S3.
[0063] Step four: Combine the channel attention feature S2 and spatial attention feature S3 with the initial fusion feature S1, and assign independent weights to each pixel position through pixel attention (PA) to obtain the channel attention weight W1 and spatial attention weight W2. .
[0064] Step 5, pass the channel attention weights W1 and spatial attention weights W2 through learnable parameters ( Dynamic fusion yields the final refined attention weight W3: .
[0065] Step 6: Adjust the contribution ratio of luminance and chrominance features using attention weight W3, and combine with residual connections to obtain the final fused feature Z1: .
[0066] The generation module uses a multi-layered cascaded 3×3 convolutional layer to extract features from feature map Z1 to obtain feature map Z2. Then, a single 3×3 convolutional layer and a sigmoid function are used to compress the number of channels in feature map Z2 to 3, and inverse normalization is performed to obtain an enhanced YUV image.
[0067] This embodiment uses a fusion generative network to analyze the brightness perception feature F. Y and color perception features FUV Multi-level attentional interaction fusion adaptively balances the contribution ratio of luminance and chromaticity information in the final image. The combined application of channel attention, spatial attention, and pixel attention enables the network to highlight key structures and lesion areas while maintaining global color consistency and local detail integrity; residual connections further ensure that original image information is not lost during enhancement. The generation module extracts and integrates features through multi-layer convolution to achieve high-quality YUV image output, significantly improving the contrast and color reproduction of bronchoscopic images, providing visually friendly image data for clinical diagnosis, case archiving, and teaching research.
[0068] This invention provides an automatic acquisition and storage system for bronchial imaging data based on voice interaction. The system includes: The voice acquisition module is used to collect the voice signals emitted by the operator.
[0069] The speech recognition module is used to input speech signals into a pre-trained speech recognition model and output the corresponding text sequence.
[0070] The semantic analysis module is used to segment and extract keywords from text sequences to obtain instruction keywords.
[0071] The instruction matching module is used to match instruction keywords with a preset instruction template library to determine the corresponding control instruction.
[0072] The instruction execution module is used to process the real-time video stream according to the control instructions and store the processing results in a preset bronchial image database.
[0073] The present invention provides an automatic bronchial image data acquisition and storage system based on voice interaction. By introducing voice interaction, the system automates the image acquisition, annotation, and storage process, reducing reliance on manual button or touch operation. This allows the main operator to complete the automatic acquisition of images simply by issuing a command, reducing dependence on other operators, effectively saving labor costs and improving overall efficiency.
[0074] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention are within the scope of the claims of the present invention.
Claims
1. A method for automatic acquisition and storage of bronchial imaging data based on voice interaction, characterized in that, The method includes: Collect the voice signals emitted by the operator; The speech signal is input into a pre-trained speech recognition model, which outputs the corresponding text sequence. The text sequence is segmented and keywords are extracted to obtain instruction keywords; The command keywords are matched with a preset command template library to determine the corresponding control commands; the control commands include image acquisition type commands and / or image annotation commands. The real-time video stream is processed according to the control instructions, and the processing results are stored in a preset bronchial image database.
2. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 1, characterized in that, The image acquisition type command includes a screenshot command, which triggers the screenshot and enhancement processing; The enhancement process is implemented through a pre-trained image enhancement model; the image enhancement model includes: The color space conversion module is used to convert the input image to the YUV color space to obtain the Y channel image, U channel image and V channel image; The brightness processing branch processes the Y-channel image to obtain the brightness-perceived features F. Y ; The chroma processing branch is used to process the U-channel and V-channel images to obtain the chroma-perceptual features F. UV ; A fusion-generating network is used to process the luminance-perceived features and the chrominance-perceived features to obtain an enhanced YUV image; The color space inverse conversion module is used to convert the YUV image back to RGB space to obtain the final enhanced image.
3. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 2, characterized in that, The brightness processing branch includes an initial convolutional layer, a global feature extraction module, a local feature extraction module, an adaptive fusion module, and a first branch supervision module, wherein: The initial convolutional layer uses a 1×1 convolution to process the input Y-channel image, expanding the channel dimension to obtain the feature map Y1; The global feature extraction module uses two Swing Transformer blocks to extract global features from feature map Y1 to obtain global feature Y2. The local feature extraction module uses four residual blocks to extract features from the feature map Y1 to obtain local features Y3. The adaptive fusion module performs a weighted summation of the global feature Y2 and the local feature Y3 to obtain the feature map Y4. The first branch supervision module uses a residual attention mechanism to fuse the input Y-channel image and feature map Y4 to obtain the brightness perception feature F. Y .
4. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 3, characterized in that, The calculation process of the first branch supervision module includes: A 3×3 convolution layer is used to process the feature map Y4 to compress the channel dimension, resulting in the feature map F1. After adding feature map F1 element by element to the Y channel image, attention weight map A1 is generated by a 3×3 convolution and a sigmoid function. A 3×3 convolution layer is used to process feature map Y4 to obtain feature map F2; The feature map F2 is weighted using the attention weight map A1 to obtain the feature map F3; A 3×3 convolution layer is used to process the input Y-channel image to expand the channel dimension and obtain the feature map F4. Adding feature maps F3, F4, and Y4 element by element yields the brightness perception feature F. Y .
5. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 2, characterized in that, The chroma processing branch includes a first feature denoising module, a second feature denoising module, a detail enhancement module, and a second branch supervision module; wherein: The first feature denoising module and the second feature denoising module have the same structure. They process the U-channel image and the V-channel image in parallel using multi-layer convolution and attention mechanisms to obtain feature maps U1 and V1. U1 and V1 are then concatenated along the channels to obtain feature map T1. The detail enhancement module uses discrete wavelet transform to convert feature map T1 into frequency domain features, and uses residual channel attention module to enhance the frequency domain features. The enhanced frequency domain features are then reconstructed back into the spatial domain to obtain feature map T2. The second branch supervision module uses a residual attention mechanism to fuse the input UV channel image and feature map T2 to obtain the chromaticity perception feature F. UV The second branch supervision module has the same structure as the first branch supervision module.
6. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 5, characterized in that, The computation process of any feature denoising module includes: The input image is processed using four 3×3 convolution layers to obtain multi-scale feature maps X1, X2, X3, and X4; the stride of the convolution from the second to the fourth layer is 2. A multi-head attention mechanism is used to process feature map X4 and model global dependencies to obtain feature map X5; Feature map X6 is obtained by upsampling feature maps X1, X2, X3 and X5 element by element; X6 is processed by a 3×3 convolution layer and residual connections are performed to obtain feature map X7; The feature map X7 is processed by two 3×3 convolution layers to obtain the denoised feature map X8, and X8 is output.
7. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 5, characterized in that, The operation process of the detail enhancement module includes: A three-level discrete wavelet transform is performed on the input feature map T1 to obtain the sub-bands of each level; The residual channel attention module is used to process each sub-band to obtain the enhanced frequency domain sub-band; Performing an inverse discrete wavelet transform on the frequency domain subband of the third-level enhancement yields the enhanced subband. ; For the enhanced subband The enhanced subband is obtained by performing an inverse discrete wavelet transform on the frequency domain subband of the second-level enhancement. ; For the enhanced subband The spatial domain feature T0 is obtained by performing inverse discrete wavelet transform on the frequency domain sub-band of the first-level enhancement. The spatial domain feature T0 is processed using the residual channel attention module to obtain the feature map T1.
8. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 2, characterized in that, The fusion generation network includes a fusion module and a generation module; wherein: The fusion module employs a multi-level attention mechanism to integrate the brightness perception features F Y and color perception features F UV Interactive fusion yields feature map Z1; The generation module uses a multi-layered cascaded 3×3 convolutional layer to extract features from feature map Z1 to obtain feature map Z2. Then, a single 3×3 convolutional layer and a sigmoid function are used to compress the number of channels in feature map Z2 to 3, and inverse normalization is performed to obtain an enhanced YUV image.
9. The method for automatic acquisition and storage of bronchial imaging data based on voice interaction according to claim 8, characterized in that, The calculation process of the fusion module includes: Brightness perception feature F Y and color perception features F UV The feature map S is obtained by adjusting the number of channels through a 3×3 convolution layer. Y and S UV ; Feature map S Y and S UV Adding each element together, we obtain the initial fusion feature S1; Channel attention and spatial attention are performed in parallel on the initial fusion feature S1 to obtain channel attention feature S2 and spatial attention feature S3; The channel attention feature S2 and spatial attention feature S3 are combined with the initial fusion feature S1, and independent weights are assigned to each pixel position through pixel attention to obtain the channel attention weight W1 and spatial attention weight W2. The channel attention weight W1 and spatial attention weight W2 are dynamically fused through learnable parameters to obtain the final refined attention weight W3; Attention weight W3 is used to adjust the contribution ratio of luminance and chrominance features, and combined with residual connections, the final fused feature Z1 is obtained.
10. An automatic acquisition and storage system for bronchial imaging data based on voice interaction, characterized in that, The system includes: The voice acquisition module is used to acquire the voice signals emitted by the operator. The speech recognition module is used to input the speech signal into a pre-trained speech recognition model and output the corresponding text sequence; The semantic analysis module is used to segment and extract keywords from the text sequence to obtain instruction keywords; The instruction matching module is used to match the instruction keywords with a preset instruction template library to determine the corresponding control instructions; the control instructions include image acquisition type instructions and / or image annotation instructions; The instruction execution module is used to process the real-time video stream according to the control instructions and store the processing results in a preset bronchial image database.