Uterine cavity image automatic lesion detection method and system

Abstract

The invention discloses a uterine cavity image automatic lesion detection method and system, and the method comprises the following steps: 1, collecting an analog signal in a uterine cavity through a CMOS image sensor, and providing illumination through an LED; 2, converting the analog image signal into a digital image signal through an analog-to-digital conversion chip; 3, performing ISP image processing on the image signal by using an ISP processor; 4, evaluating the image quality through an image quality control module, and dynamically adjusting the LED brightness; 5, carrying out image AI uterine cavity lesion detection on the processed image by using an NPU chip; and 6, outputting an image in real time, and superposing a detection result for display. According to the system, advanced image collecting, processing and displaying technologies are integrated, an automatic uterine cavity image lesion detection method is provided, the speed and accuracy of uterine cavity lesion detection are remarkably improved in an automatic and intelligent mode, and powerful support is provided for clinical diagnosis and treatment.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing and medical automation technology, and in particular to a method and system for automatic lesion detection of uterine cavity images. Background Art

[0002] Common diagnostic methods for uterine cavity lesions include ultrasound examination, hysteroscopy, diagnostic curettage, serological examination and other examinations. Doctors will choose the appropriate method according to the patient's condition, and the diagnostic methods for special populations may be different. However, the diagnosis of the above uterine cavity diseases mainly depends on the doctor's naked eye observation combined with hysteroscopic images. Hysteroscopy is also the "gold standard". This method not only relies on the doctor's experience, but also has problems such as strong subjectivity and high missed diagnosis rate.

[0003] With the development of medical imaging technology and artificial intelligence, especially the breakthrough of deep learning in the field of image recognition, automatic lesion detection in uterine cavity images has become possible. However, there are still some shortcomings: 1. Most existing systems are based on large-scale server processing, which has shortcomings such as poor real-time performance, complex equipment and high cost.

[0004] 2. Lack of efficient embedded solutions makes it difficult to achieve portability and real-time detection.

[0005] 3. Deep learning algorithms operate inefficiently in resource-constrained environments, affecting detection speed and accuracy.

[0006] 4. The existing system has limited ability to identify complex lesions, and has high false alarm and missed alarm rates.

[0007] In view of this, the present invention proposes a method and system for automatic lesion detection in uterine cavity images, aiming to solve the above problems. Summary of the invention

[0008] The purpose of the present invention is to provide a method and system for automatic lesion detection in uterine cavity images. The method and system are based on the image sensor to collect intrauterine cavity image information, transmit it to the embedded image processor, and use the NPU processing unit, which integrates the deep learning AI detection algorithm, analyzes the uterine cavity lesion tissue in real time, and accurately detects lesions. The automatic lesion detection system for uterine cavity images of the present invention significantly improves the diagnostic efficiency and accuracy of uterine cavity lesions through integrated design, efficient data processing capabilities and accurate lesion recognition capabilities. At the same time, it reduces the dependence on doctors' experience and reduces the risk of misdiagnosis due to human factors. The specific technical scheme is as follows: A method for automatic lesion detection of uterine cavity images comprises the following steps: Step 1: The CMOS image sensor collects analog signals in the uterine cavity and uses LED to provide lighting; Step 2: Convert the analog image signal into a digital image signal through an analog-to-digital conversion chip; Step 3: Use the ISP processor to perform ISP image processing on the image signal; Step 4: Evaluate the image quality through the image quality control module and dynamically adjust the LED brightness; Step 5: Use the NPU chip to perform AI uterine cavity lesion detection on the processed image; Step 6: Output the image in real time and overlay the detection results.

[0009] Preferably, in step 1, during hysteroscopy and surgery, a CMOS image sensor at the front end of a hysteroscope is used to collect analog signals in the uterine cavity, and an LED is used to provide lighting.

[0010] Preferably, in step 2, the analog image signal is converted into a digital image signal by an analog-to-digital conversion chip.

[0011] Preferably, in step 3, in order to obtain high-quality uterine cavity image signals, an ISP processor is used to perform ISP image processing on the image signals.

[0012] Preferably, in step 4, the image quality is evaluated by the image quality control module, and the LED brightness is dynamically adjusted. The following steps are included: Step (1) Preset the average brightness range .in is the minimum brightness, is the maximum brightness, is the threshold interval when the image quality is high, and ; Preset LED brightness adjustment step S.

[0013] Step (2) Calculate the average brightness of the current frame image .

[0014] Step (3) calculates the required LED brightness increment based on the current value. The specific method is:

[0015] in, is the LED brightness increment; is the incremental weight, ranging from 1 to 10. exist When When adjusting the LED brightness, the incremental weight will be added , adjust to the appropriate brightness faster.

[0016] Preferably, in step 5, the processed image is subjected to image AI uterine cavity lesion detection using an NPU chip, comprising the following steps: Step (1): Input feature map (where H is the height, W is the width, is the number of input channels), the feature map is input into the depth convolution and a separate convolution kernel is applied to each channel. The convolution kernel size of the depth convolution is K×K. For the cth input channel, the cth channel of the output feature map It can be calculated by the following formula:

[0017] in, is the K×K convolution kernel applied to the cth input channel, is the value of the cth channel at position (i, j) of the output feature map after the depthwise convolution. Step (2): Perform a 1x1 point convolution operation on the output of the depthwise convolution in step (1) to mix channel information and adjust the number of output channels. If the number of output channels is , then the output feature map of the point convolution It can be calculated by the following formula:

[0018] in, yes The point convolution kernel is is the output feature map of the deep convolution, It is the output feature map at position (i, j) after point convolution. The above two steps are used to perform deep convolution, which achieves fast and efficient feature extraction, while reducing the number of parameters and improving computing performance. Step (3): After performing step (2), the output feature map is subjected to BatchNorm2d operation and then connected to the Relu6 operator. Replace the original LeakRelu operator. This simple calculation logic helps to achieve fast calculation on NPU. Step (4): Take the output features in step (3) as input , perform MaxPool2d operation to obtain , Perform the MaxPool2d operation to obtain: , Perform the MaxPool2d operation to obtain , and Feature splicing . For the output features Then perform standard convolution Conv2d, BatchNorm2d and Relu6 operations. The above is the complete structure of the ESPSCN structure.

[0019] Preferably, in step 6, the prototype distance of each category is calculated based on the test image to obtain the category of the uterine cavity lesion image.

[0020] An automatic uterine cavity image lesion detection system adopts the above detection method. The detection system integrates an image acquisition unit, collects intrauterine cavity image data through a CMOS image sensor and an LED light source; converts image analog signals into digital signals through analog-to-digital conversion on an output processing host; dynamically adjusts LED brightness through an image quality control module; proposes an efficient spatial pyramid separation convolutional network structure, namely, an ESPSCN network structure, based on a yolov5 detection network through an AI uterine cavity lesion detection algorithm to replace the SPPF module in the yolov5 structure; and performs real-time target detection and analysis through NPU module acceleration, and transmits the processed images and detection results to a display for display.

[0021] Preferably, the ESPSCN network structure changes the convolution CONV operation to a depth-wise separable convolution, and changes the LeakyReLU activation function in the structure to the activation function Relu6 commonly used on embedded devices.

[0022] Compared with the closest prior art, the technical solution provided by the present invention has the following beneficial effects: The automatic lesion detection method for uterine cavity images of the present invention helps to reduce problems such as image blur, overexposure or darkness by dynamically adjusting the LED brightness, ensures the consistency and accuracy of image acquisition, and ensures that the image quality meets the detection requirements, thereby improving the reliability of lesion detection; the automatic lesion detection system for uterine cavity images of the present invention provides an efficient and accurate automated solution for uterine cavity lesion detection, and through integrated design, efficient data processing capabilities and accurate lesion recognition capabilities, the use of NPU chips for image AI processing can significantly improve the detection speed and realize real-time detection; it helps to improve the diagnosis level and service quality of medical institutions; at the same time, it reduces the dependence on doctors' experience and reduces the risk of misdiagnosis due to human factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a processing flow chart of a method for automatic lesion detection in uterine cavity images; Figure 2 It is the image quality control module that controls the process; Figure 3 This is the structure diagram of ESPSCN in the improved YOLOv5 network. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Please refer to Figure 1-3 .

[0025] like Figure 1 As shown, a method for automatic lesion detection of uterine cavity images of the present invention comprises the following steps: Step 1: The CMOS image sensor collects analog signals in the uterine cavity and uses LED to provide lighting; Step 2: Convert the analog image signal into a digital image signal through an analog-to-digital conversion chip; Step 3: Use the ISP processor to perform ISP image processing on the image signal; Step 4: Evaluate the image quality through the image quality control module and dynamically adjust the LED brightness; like Figure 2 As shown, the control process is as follows: Step (1) Preset the average brightness range .in is the minimum brightness, is the maximum brightness, is the threshold interval when the image quality is high, and ; Preset LED brightness adjustment step S.

[0026] Step (2) Calculate the average brightness of the current frame image .

[0027] Step (3) calculates the required LED brightness increment based on the current value. The specific method is: in, is the LED brightness increment; is the incremental weight, ranging from 1 to 10. exist When When adjusting the LED brightness, the incremental weight will be added , adjust to the appropriate brightness faster.

[0028] Step 5: Use the NPU chip to perform AI uterine cavity lesion detection on the processed image; like Figure 3 As shown in the figure, based on the improved YOLOv5 network, the SPPF structure is replaced with the ESPSCN structure, and the network construction process is as follows: Step (1): Input feature map (where H is the height, W is the width, is the number of input channels), the feature map is input into the depth convolution and a separate convolution kernel is applied to each channel. The convolution kernel size of the depth convolution is K×K. For the cth input channel, the cth channel of the output feature map It can be calculated by the following formula:

[0029] in, is the K×K convolution kernel applied to the cth input channel, It is the value of the cth channel at position (i, j) of the output feature map after depthwise convolution.

[0030] Step (2): Perform a 1x1 point convolution operation on the output of the depthwise convolution in step (1) to mix channel information and adjust the number of output channels. If the number of output channels is , then the output feature map of the point convolution It can be calculated by the following formula: in, yes The point convolution kernel is is the output feature map of the deep convolution, It is the output feature map at position (i, j) after point convolution. Through the above two steps of deep convolution, fast and efficient feature extraction is achieved, which reduces the number of parameters and improves the computing performance.

[0031] Step (3): After step (2), the output feature map is subjected to the BatchNorm2d operation and then connected to the Relu6 operator. Replace the original LeakRelu operator. This simple calculation logic helps to achieve fast calculation on NPU. Step (4): Take the output features in step (3) as input , perform MaxPool2d operation to obtain , Perform the MaxPool2d operation to obtain: , Perform the MaxPool2d operation to obtain , and Feature splicing . For the output features Then perform standard convolution Conv2d, BatchNorm2d and Relu6 operations. The above is the complete structure of the ESPSCN structure.

[0032] Step 6: Output the image in real time and overlay the detection results.

[0033] An automatic uterine cavity image lesion detection system adopts the above detection method. The detection system integrates an image acquisition unit, collects intrauterine cavity image data through a CMOS image sensor and an LED light source; converts image analog signals into digital signals through analog-to-digital conversion on an output processing host; dynamically adjusts LED brightness through an image quality control module; proposes an Efficient Spatial Pyramid Separable Convolutional Network (ESPSCN) structure based on the YOLOv5 detection network through an AI uterine cavity lesion detection algorithm to replace the SPPF module in the YOLOv5 structure; and performs real-time target detection and analysis through NPU module acceleration, and transmits the processed images and detection results to a display for display.

[0034] Among them, the ESPSCN network structure changes the convolution CONV operation to a depth-separable convolution, and changes the LeakyReLU activation function in the structure to the activation function Relu6 commonly used on embedded devices, because it helps to limit the output to a smaller range, helps to reduce the amount of calculation and improve numerical stability. Using the NPU module acceleration, real-time target detection and analysis are carried out to achieve real-time, efficient and accurate lesion detection of uterine cavity images, improve diagnostic efficiency and accuracy, and reduce the risk of misdiagnosis. The above is only a preferred embodiment of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the idea of ​​the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and embellishments without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for automatic lesion detection in uterine cavity images, characterized in that: The following steps are involved: Step 1: The CMOS image sensor collects analog signals in the uterine cavity and uses LED to provide lighting; Step 2: Convert the analog image signal into a digital image signal through an analog-to-digital conversion chip; Step 3: Use the ISP processor to perform ISP image processing on the image signal; Step 4: Evaluate the image quality through the image quality control module and dynamically adjust the LED brightness; The image quality of the processed image is controlled, and the LED brightness is dynamically adjusted in real time according to the image quality, which specifically includes the following steps: Step (1) Preset the average brightness range ;in is the minimum brightness, is the maximum brightness, is the threshold interval when the image quality is high, and ;Preset LED brightness adjustment step S; Step (2) Calculate the average brightness of the current frame image ; Step (3) calculates the brightness increment of the LED to be adjusted based on the current value; the specific method is: in, is the LED brightness increment; is the incremental weight, ranging from 1 to 10. exist and When or When adjusting the LED, the incremental weight will be added , so as to adjust to the appropriate brightness more quickly; Step 5: Use the NPU chip to perform AI uterine cavity lesion detection on the processed image; Step 6: Output the image in real time and overlay the detection results.

2. The method for automatic lesion detection in uterine cavity images according to claim 1, characterized in that: In the step 1, the CMOS image sensor at the front end of the hysteroscope collects analog signals in the uterine cavity, and an LED is used to provide illumination.

3. The method for automatic lesion detection in uterine cavity images according to claim 1, characterized in that: In step 2, the collected analog image signal is converted into a digital image signal through a collection chip.

4. The method for automatic lesion detection in uterine cavity images according to claim 1, characterized in that: In the step 3, the ISP image processing is performed on the digital image signal ISP chip.

5. The method for automatic lesion detection in uterine cavity images according to claim 1, characterized in that: In step 5, the NPU chip is used to perform AI uterine cavity lesion detection. To improve the detection speed, this detection method improves the SPPF module in yolov5 and proposes an ESPSCN network structure, which specifically includes the following steps: Step (1): Input feature map (where H is the height, W is the width, is the number of input channels), the feature map is input into the deep convolution and a separate convolution kernel is applied to each channel; the convolution kernel size of the deep convolution is K×K, and for the cth input channel, the cth channel of the output feature map It can be calculated by the following formula: in, is the K×K convolution kernel applied to the cth input channel, is the value of the cth channel at position (i, j) of the output feature map after depth convolution; Step (2): Perform a 1x1 point convolution operation on the output of the depthwise convolution in step (1) to mix channel information and adjust the number of output channels; if the number of output channels is , then the output feature map of the point convolution It can be calculated by the following formula: in, yes The point convolution kernel is is the output feature map of the deep convolution, It is the output feature map at position (i, j) after point convolution. The above two steps are used to perform deep convolution, which enables fast and efficient feature extraction, while reducing the number of parameters and improving computing performance. Step (3): After step (2), the output feature map is subjected to the BatchNorm2d operation and then connected to the Relu6 operator. Replace the original LeakRelu operator. This simple calculation logic helps to achieve fast calculation on NPU. Step (4): Take the output features in step (3) as input , perform MaxPool2d operation to obtain , Perform the MaxPool2d operation to obtain , Perform the MaxPool2d operation to obtain , and Feature splicing ; Output features Then perform standard convolution Conv2d, BatchNorm2d and Relu6 operations; the above is the complete structure of the ESPSCN structure.

6. The method for automatic lesion detection of uterine cavity images according to claim 1, characterized in that: The image and detection result processed in step 6 are transmitted to the display for display.

7. A system for automatic lesion detection of uterine cavity images, using the above detection method, characterized in that: The detection system integrates an image acquisition unit, which collects intrauterine image data through CMOS image sensors and LED light sources; converts image analog signals into digital signals through analog-to-digital conversion on the output processing host; dynamically adjusts LED brightness through the image quality control module; and proposes an efficient spatial pyramid separation convolutional network structure, namely the ESPSCN network structure, based on the YOLOv5 detection network through the AI ​​intrauterine lesion detection algorithm to replace the SPPF module in the YOLOv5 structure; and performs real-time target detection and analysis through NPU module acceleration, and transmits the processed images and detection results to the display for display.

8. The automatic lesion detection system for uterine cavity images according to claim 7, characterized in that: The ESPSCN network structure changes the convolution CONV operation into a depth-wise separable convolution, and changes the LeakyReLU activation function in the structure into the activation function Relu6 commonly used on embedded devices.

Images