A hysteroscopy image automatic lesion detection method and system

Abstract

The application discloses a kind of hysterosalpingogram automatic lesion detection method, system, comprising the following steps: 1: CMOS image sensor gathers analog signal in uterine cavity, and illumination is provided using LED;2: convert analog image signal into digital image signal by analog-digital conversion chip;3: using ISP processor, image signal is carried out ISP image processing;4: image quality is evaluated by image quality control module, and LED brightness is dynamically adjusted, 5: using NPU chip, image AI uterine cavity lesion detection is carried out to the image after processing;6: real-time output image and superimposed detection result display.The application system integrates advanced image acquisition, processing and display technology, proposes a kind of hysterosalpingogram automatic lesion detection method, through automation, intelligent way, the speed and accuracy of hysterosalpingography lesion detection are significantly improved, and strong support is provided for clinical diagnosis and treatment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing and medical automation, in particular to a hysteroscopy image automatic lesion detection method and system. BACKGROUND

[0002] Common diagnostic methods for hysteroscopy lesions include ultrasound examination, hysteroscopy examination, diagnostic curettage, serological examination and other examinations. Doctors will choose appropriate methods according to the patient's condition, and the diagnostic method of special groups may be different. However, the diagnosis of the above hysteroscopy diseases mainly depends on the naked eye observation of doctors combined with hysteroscopy images, and hysteroscopy examination is also the "gold standard". This method not only depends on the experience of doctors, but also has the problems of strong subjectivity, high missed diagnosis rate, etc.

[0003] With the development of medical imaging technology and artificial intelligence, especially the breakthrough of deep learning in image recognition, it is possible to automatically detect lesions in hysteroscopy images. However, there are still some deficiencies:

[0004] 1. Existing systems are mostly based on large servers for processing, which have the disadvantages of poor real-time performance, complex equipment and high cost.

[0005] 2. There is a lack of efficient embedded solutions, making it difficult to achieve portability and real-time detection.

[0006] 3. The running efficiency of deep learning algorithms in resource-constrained environments is low, affecting the detection speed and accuracy.

[0007] 4. The existing system has limited recognition ability for complex lesions, with high false positive rate and false negative rate.

[0008] In view of the above problems, the present application provides a hysteroscopy image automatic lesion detection method and system. SUMMARY

[0009] The purpose of the present application is to provide a hysteroscopy image automatic lesion detection method and system. The method and system are based on image sensors to collect image information inside the hysteroscopy, transmit it to an embedded image processor, and use an NPU processing unit that integrates a deep learning AI detection algorithm to analyze hysteroscopy lesions in real time and accurately detect lesions. The hysteroscopy image automatic lesion detection system of the present application significantly improves the diagnosis efficiency and accuracy of hysteroscopy lesions through integrated design, efficient data processing capability and precise lesion recognition capability. At the same time, it reduces the dependence on doctors' experience and reduces the risk of misdiagnosis caused by human factors. The specific technical solutions are as follows:

[0010] A hysteroscopy image automatic lesion detection method, comprising the following steps:

[0011] Step 1: CMOS image sensor collects analog signals in the uterine cavity, and LED provides illumination;

[0012] Step 2: Analog image signals are converted into digital image signals by an analog-to-digital conversion chip;

[0013] Step 3: ISP image processing is performed on the image signals using an ISP processor;

[0014] Step 4: Image quality is evaluated by an image quality control module, and LED brightness is dynamically adjusted;

[0015] Step 5: AI uterine cavity lesion detection is performed on the processed image using an NPU chip;

[0016] Step 6: Real-time image output and superimposed detection result display.

[0017] Preferably, in step 1, analog signals in the uterine cavity are collected using a CMOS image sensor at the front end of a hysteroscope during hysteroscopy and surgery, and LED provides illumination.

[0018] Preferably, in step 2, analog image signals are converted into digital image signals by an analog-to-digital conversion chip.

[0019] Preferably, in step 3, in order to obtain high-quality uterine cavity image signals, ISP image processing is performed on the image signals using an ISP processor.

[0020] Preferably, in step 4, image quality is evaluated by an image quality control module, and LED brightness is dynamically adjusted. The following steps are included:

[0021] Step (1) Preset brightness average value range . Wherein is the minimum brightness, is the maximum brightness, is the threshold interval when the image quality is high, and ; the preset LED brightness adjustment step S.

[0022] Step (2) Calculate the current frame image brightness average value LA.

[0023] Step (3) Calculate the brightness increment of the LED that needs to be adjusted according to the value of the current LA. The specific method is:

[0024]

[0025] Where, is the LED brightness increment; W is the increment weight, which is valued at 1~10, when LA is between , no processing is performed; when When adjusting LED brightness, an incremental weight W is applied to achieve the desired brightness more quickly.

[0026] Preferably, step 5 involves using an NPU chip to perform AI-based intrauterine lesion detection on the processed image, including the following steps:

[0027] Step (1): Input feature map Where H represents the height and W represents the width. This represents the number of input channels. The feature map is input into a depthwise convolution, which applies a separate kernel to each channel. The kernel size of the depthwise convolution is K×K. For the c-th input channel, the c-th channel of the output feature map is generated. It can be calculated using the following formula:

[0028]

[0029] in, It is a K×K convolution kernel applied to the c-th input channel. It is the value of the c-th channel of the output feature map after depthwise convolution at position (i,j).

[0030] Step (2): Perform a 1x1 pointwise convolution operation on the output of the depthwise convolution in step (1) to mix channel information and adjust the number of output channels. If the number of output channels is... The output feature map of point convolution. It can be calculated using the following formula:

[0031]

[0032] in, yes Point convolution kernel, It is the output feature map of depthwise convolution. The output feature map after point convolution is at position (i,j) of the first position. The values ​​of each channel are used. Through the above two steps of depthwise convolution, fast and efficient feature extraction is achieved, improving computational performance while reducing the number of parameters.

[0033] Step (3): After performing step (2), the output feature map is subjected to the BatchNorm2d operation and then connected to the Relu6 operator. Replacing the original LeakRelu operator, this simple computational logic facilitates fast computation on the NPU.

[0034] Step (4): Use the output features from step (3) as input. Perform the MaxPool2d operation to obtain , The MaxPool2d operation is performed to obtain , The MaxPool2d operation is performed to obtain , and The features are spliced to obtain . The output features are subjected to standard convolution Conv2d, BatchNorm2d and Relu6 operations again. The above is the complete structure of the ESPSCN structure.

[0035] Preferably, the prototype distance of each class is calculated according to the test picture in step 6, and the class of the uterine cavity lesion image is obtained.

[0036] An automatic uterine cavity lesion detection system adopts the above detection method, integrates an image acquisition unit, acquires uterine cavity image data through a CMOS image sensor and an LED light source, converts an image analog signal into a digital signal through an analog-to-digital conversion on an output processing host, dynamically adjusts the LED brightness through an image quality control module, proposes an efficient spatial pyramid separation convolution network structure, i.e., an ESPSCN network structure, based on a yolov5 detection network, and replaces an SPPF module in the yolov5 structure through the ESPSCN network structure, and performs real-time target detection analysis and detection through an NPU module acceleration, and transmits the processed image and the detection result to a display for display.

[0037] Preferably, the ESPSCN network structure changes a convolution CONV operation into a depth separable convolution, and changes a LeakyReLU activation function in the structure into a commonly used activation function Relu6 on an embedded device.

[0038] Compared with the closest prior art, the technical scheme provided by the present application has the following beneficial effects:

[0039] The automatic uterine cavity lesion detection method of the present application helps to reduce problems such as image blur, overexposure or overdarkness, ensures the consistency and accuracy of image acquisition, ensures that the image quality meets the detection requirements, and thus improves the reliability of lesion detection; the automatic uterine cavity lesion detection system of the present application provides an efficient, accurate and automatic solution for uterine cavity lesion detection, and through integrated design, efficient data processing capability and accurate lesion recognition capability, image AI processing is performed by using an NPU chip, which can significantly improve the detection speed and realize real-time detection; helps to improve the diagnosis level and service quality of medical institutions; at the same time, reduces the dependence on the experience of doctors and reduces the risk of misdiagnosis caused by human factors. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1It is a hysteroscopy image automatic lesion detection method processing flow chart.

[0041] Figure 2 It is an image quality control module control process.

[0042] Figure 3 It is an improved ESPSCN structure diagram in YOLOv5 network. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application. Please refer to Figures 1-3 .

[0044] As Figure 1 shown, the hysteroscopy image automatic lesion detection method of the present application comprises the following steps:

[0045] Step 1: The CMOS image sensor collects the analog signal in the uterine cavity, and the LED provides illumination;

[0046] Step 2: Convert the analog image signal to digital image signal through the analog-digital conversion chip;

[0047] Step 3: Use ISP processor to process the image signal;

[0048] Step 4: Evaluate the image quality through the image quality control module, and dynamically adjust the LED brightness;

[0049] As Figure 2 shown, the control process is as follows:

[0050] Step (1) preset the average value range of brightness . Wherein is the minimum value of brightness, is the maximum value of brightness, is the threshold interval when the image quality is high, and ; preset LED brightness adjustment step S.

[0051] Step (2) calculate the average value LA of the current frame image brightness.

[0052] Step (3) calculate the brightness increment of the LED to be adjusted according to the value of the current LA. The specific method is:

[0053]

[0054] Wherein, LA represents the LED brightness increment; W is the increment weight, ranging from 1 to 10. In the meantime, no action is taken; when When adjusting LED brightness, an incremental weight W is applied to achieve the desired brightness more quickly.

[0055] Step 5: Use the NPU chip to perform AI-based uterine cavity lesion detection on the processed images;

[0056] like Figure 3 As shown, the SPPF structure is replaced with the ESPSCN structure in the improved YOLOv5 network. The network construction process is as follows:

[0057] Step (1): Input feature map Where H represents the height and W represents the width. This represents the number of input channels. The feature map is input into a depthwise convolution, which applies a separate kernel to each channel. The kernel size of the depthwise convolution is K×K. For the c-th input channel, the c-th channel of the output feature map is generated. It can be calculated using the following formula:

[0058]

[0059] in, It is a K×K convolution kernel applied to the c-th input channel. It is the value of the c-th channel of the output feature map after depthwise convolution at position (i,j).

[0060] Step (2): Perform a 1x1 pointwise convolution operation on the output of the depthwise convolution in step (1) to mix channel information and adjust the number of output channels. If the number of output channels is... The output feature map of point convolution. It can be calculated using the following formula:

[0061]

[0062] in, yes Point convolution kernel, It is the output feature map of depthwise convolution. The output feature map after point convolution is at position (i,j) of the first position. The values ​​of each channel are used. Through the above two steps of depthwise convolution, fast and efficient feature extraction is achieved, improving computational performance while reducing the number of parameters.

[0063] Step (3): After performing step (2), the output feature map is subjected to the BatchNorm2d operation and then connected to the Relu6 operator. Replace the original LeakRelu operator. This simple calculation logic helps to achieve fast calculation on NPU.

[0064] Step (4): Take the output features in step (3) as input , and perform MaxPool2d operation to obtain , perform MaxPool2d operation to obtain , perform MaxPool2d operation to obtain , and concatenate features to obtain . Perform standard convolution Conv2d, BatchNorm2d and Relu6 operations on the output features . The above is the complete structure of ESPSCN structure.

[0065] Step 6: Real-time output image and superimpose detection results for display.

[0066] An automatic uterine cavity image lesion detection system adopts the above detection method. The detection system integrates an image acquisition unit, which acquires uterine cavity image data through a CMOS image sensor and an LED light source; converts the image analog signal to a digital signal through an analog-to-digital conversion on the output processing host; dynamically adjusts the LED brightness through the image quality control module; through the AI uterine cavity lesion detection algorithm, based on the yolov5 detection network, an efficient spatial pyramid separable convolutional network (Efficient Spatial Pyramid Separable Convolutional Network, ESPSCN) structure is proposed to replace the SPPF module in the yolov5 structure; through the NPU module acceleration, real-time target detection analysis and detection are performed, and the processed image and detection result are transmitted to the display for display.

[0067] Among them, the ESPSCN network structure changes the convolution CONV operation to depth separable convolution, and changes the LeakyReLU activation function in the structure to the commonly used activation function Relu6 on the embedded device, because it helps to limit the output to a smaller range, which helps to reduce the calculation amount and improve the numerical stability. Utilizing NPU module acceleration, real-time target detection analysis is performed, realizing real-time, efficient and accurate lesion detection of uterine cavity images, improving the diagnosis efficiency and accuracy, and reducing the risk of misdiagnosis.

[0068] The above are only preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical scheme falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application shall be considered as the protection scope of the present application.

Claims

1. A method for automatic lesion detection of hysteroscopic images, characterized in that, The method comprises the following steps: Step 1: the CMOS image sensor collects the analog signal in the uterine cavity, and the LED provides illumination; Step 2: the analog image signal is converted into a digital image signal through an analog-to-digital conversion chip; Step 3: the ISP processor is used for ISP image processing of the image signal; Step 4: the image quality is evaluated through an image quality control module, and the LED brightness is dynamically adjusted; The processed image is subjected to image quality control, and the LED brightness is dynamically adjusted in real time according to the image quality, which specifically comprises the following steps: Step (1) preset luminance average value range ; wherein is a luminance minimum value, is a luminance maximum value, is a threshold interval when the image quality is high, and ; a preset LED luminance adjustment step S; Step (2) calculates the average value LA of the current frame image brightness; Step (3) calculates the brightness increment of the LED to be adjusted according to the value of the current LA; the specific method is as follows: ; Wherein, is the LED brightness increment; W is the increment weight, and the value is 1-10, when LA is between , no processing is performed; when , the increment weight W is superimposed when adjusting the LED, so as to adjust to the appropriate brightness more quickly. Step 5: the NPU chip is used for image AI uterine cavity lesion detection of the processed image; Step 6: the image is output in real time and the detection result is displayed.

2. The method of claim 1, wherein: In step 1, the analog signal in the uterine cavity is collected from the front-end CMOS image sensor of the hysteroscope, and the LED is used for illumination.

3. The method of claim 1, wherein the method further comprises: In step 5, the NPU chip is used for image AI uterine cavity lesion detection, in order to improve the detection speed, the SPPF module in yolov5 is improved, and the ESPSCN network structure is proposed, which specifically comprises the following steps: Step (1): inputting a feature map where H denotes height and W denotes width, is an input channel number, a separate convolution kernel is applied to each channel by inputting a feature map into a deep convolution; the convolution kernel size of the deep convolution is KxK, and the c-th channel of the output feature map is which can be calculated by the following formula: ; wherein, is a KxK convolution kernel applied to the c-th input channel, is the value of the c-th channel of the output feature map after the deep convolution at position (i,j). Step (2): 1x1 point convolution operation is performed on the output of step (1) deep convolution to mix channel information and adjust the number of output channels; if the number of output channels is , then the output feature map of point convolution can be calculated by the following formula: ; wherein, is a point convolution kernel, is an output feature map of the deep convolution, is the value of the output feature map after the point convolution at the (i, j)th position of the first channel; through the above two steps, fast and efficient feature extraction is achieved, and the operation performance is improved while the parameter quantity is reduced; Step (3): After step (2), the output feature map is subjected to BatchNorm2d operation, followed by Relu6 operator Replace the original LeakRelu operator. This simple calculation logic helps to achieve fast calculation on NPU; Step (4): taking the output feature in step (3) as input , MaxPool2d operation is performed to obtain , MaxPool2d operation is performed to obtain , MaxPool2d operation is performed to obtain , and the features are spliced to obtain ; the output feature is subjected to standard convolution Conv2d, BatchNorm2d and Relu6 operations again; the above is the complete structure of the ESPSCN structure.

4. The method of claim 1, wherein the method further comprises: In step 6, the processed image and the detection result are transmitted to the display for display.

5. A system for automatic lesion detection of hysteroscopic images, employing the detection method of claims 1-4, characterized in that: The detection system integrates an image acquisition unit, collects image data in the uterine cavity through a CMOS image sensor and a LED light source, converts the analog image signal into a digital signal through an analog-to-digital conversion on the output processing host, dynamically adjusts the LED brightness through an image quality control module, and performs real-time target detection analysis and detection through an AI uterine cavity lesion detection algorithm based on a yolov5 detection network and an efficient spatial pyramid separation convolutional network structure (ESPSCN network structure) for replacing the SPPF module in the yolov5 structure. The ESPSCN network structure changes the convolution CONV operation to a depth separable convolution, and changes the LeakyReLU activation function in the structure to the commonly used activation function Relu6 on the embedded device; through the NPU module acceleration, the processed image and the detection result are transmitted to the display for display.

Images