Automatic washing method based on neural network model and endoscope system

Abstract

This application relates to the field of medical device technology and discloses an automatic flushing method and endoscope system based on a neural network model. The method includes: acquiring endoscopic images of the surgical area using an endoscope; identifying the endoscopic images using a preset neural network model to obtain the current blood concentration in the surgical area; the preset neural network model being trained using sample images and corresponding sample blood concentrations; determining a corresponding first adjustment signal based on the current blood concentration; and controlling a first flushing pump to flush the surgical area according to the first adjustment signal. Because this application can identify and obtain the current blood concentration in the surgical area through a preset neural network model and automatically control the first flushing pump to flush based on the current blood concentration, compared to existing methods that require manual control, this application eliminates the need for manual intervention, thereby improving the timeliness of flushing and thus enhancing safety.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to an automatic flushing method and endoscope system based on a neural network model. Background Technology

[0002] Currently, during endoscopic surgery, the surgical area typically requires instillation of saline solution. Some endoscopes have saline delivery lines in their sheaths, with flushing pumps installed on these lines to inject saline solution into the surgical area. During the procedure, surgical manipulation inevitably involves touching blood vessels, leading to bleeding. The mixture of blood and saline solution reduces the water's transparency, causing the images captured by the endoscope to be blurry. This severely impacts the surgeon's ability to identify the anatomical structures of the surgical area and the precision of the surgical procedure.

[0003] Currently, to promptly remove blood and fluid, a manual method is generally used. This involves the doctor or assistant adjusting the speed of the irrigation pump via a foot pedal or hand control while observing the surgical procedure. However, this method is prone to problems such as delayed irrigation, prolonged surgical time, and lower safety standards. Summary of the Invention

[0004] The main purpose of this application is to provide an automatic flushing method and endoscope system based on a neural network model, which aims to solve the technical problems of existing manual flushing methods, which are prone to untimely flushing and low safety.

[0005] To achieve the above objectives, this application provides an automatic flushing method based on a neural network model, which is applied to the endoscope host in an endoscope system. The endoscope system further includes a water supply pipeline and an endoscope scope. A first flushing pump is provided on the water supply pipeline, which controls the flushing speed of physiological saline solution in the water supply pipeline on the surgical area. The method includes: Endoscopic images of the surgical area are acquired using the endoscope. The endoscopic image is identified by a preset neural network model to obtain the current blood concentration in the surgical area. The preset neural network model is trained using sample images and corresponding sample blood concentrations. A first adjustment signal is determined based on the current blood concentration, and the first flushing pump is controlled to flush the surgical area according to the first adjustment signal.

[0006] In one embodiment, the step of identifying the endoscopic image using a preset neural network model to obtain the current blood concentration in the surgical area includes: Determine the red channel intensity value and the green channel intensity value of each pixel in the endoscopic image; The blood sensitivity intensity of each pixel is determined based on the intensity values ​​of each red channel and the corresponding intensity values ​​of the green channel. Channel replication is performed on each of the blood sensitivity intensities, and the replication results are stitched together to obtain a stitched image; The current blood concentration in the surgical area is obtained by recognizing the stitched image using a preset neural network model.

[0007] In one embodiment, the step of determining the blood sensitivity intensity corresponding to each pixel based on the intensity values ​​of each red channel and the corresponding intensity values ​​of the green channel includes: Each red channel intensity value is compared with a preset small normal number to obtain a first comparison result, and the larger value in each first comparison result is used as the corresponding normalized reference value. Divide the preset normalization constant by each of the normalization reference values ​​to obtain the corresponding scaling factor; The blood sensitivity intensity corresponding to each pixel is obtained by multiplying the intensity value of each green channel by the corresponding scaling factor.

[0008] In one embodiment, the step of channel replication for each of the blood sensitivity intensities includes: Determine the cutoff threshold for each of the aforementioned blood sensitivity intensities; Each of the blood sensitivity values ​​is compared with the cropping threshold to obtain a second comparison result, and the smaller value in each of the second comparison results is taken as the corresponding pixel intensity value; Divide each pixel intensity value by the cropping threshold to obtain the corresponding normalized blood sensitivity intensity, and perform channel copying on each normalized blood sensitivity intensity.

[0009] In one embodiment, the step of identifying the endoscopic image using a preset neural network model to obtain the current blood concentration in the surgical area includes: The endoscopic image is identified by a preset neural network model to obtain the original scores of the blood concentration in the surgical area belonging to each preset concentration level. Perform probability transformation on each of the original scores to obtain the predicted probability distribution belonging to each of the preset concentration levels; Select the target preset concentration level corresponding to the highest predicted probability from each of the predicted probability distributions, and use the target preset concentration level as the current blood concentration in the surgical area.

[0010] In one embodiment, before the step of recognizing the endoscopic image using a preset neural network model, the method further includes: Each sample image and its corresponding preset concentration level are acquired, and each sample image is identified through an initial neural network model to obtain the sample prediction probability distribution of each sample image belonging to each preset concentration level. The model loss value is determined based on the preset loss function, the preset concentration level of each sample, and the predicted probability distribution of each sample. The parameters of the initial neural network model are adjusted based on the model loss value, and the process of recognizing each sample image using the initial neural network model is repeated until the preset training conditions are met, thereby obtaining the preset neural network model.

[0011] In one embodiment, the step of determining the model loss value based on a preset loss function, preset concentration levels of each sample, and predicted probability distributions of each sample includes: Determine the occurrence ratio of each preset concentration level of the sample, and assign a corresponding category weight to each preset concentration level of the sample according to the occurrence ratio, wherein the occurrence ratio is negatively correlated with the category weight; The original loss value is determined based on the preset loss function, the preset concentration level of each sample, and the predicted probability distribution of each sample. The original loss values ​​are weighted and summed according to the corresponding category weights, and then averaged to obtain the model loss value.

[0012] In one embodiment, after the step of acquiring each sample image and the corresponding preset concentration level of the sample, the method further includes: The preset concentration level of each sample image is taken as the true preset concentration level. The label value representing the true preset concentration level is set as a first value, and the label values ​​representing the concentration levels of each sample other than the true preset concentration level are set as second values. The first value is 1- The second value is / n, The preset smoothing coefficient is n, which is the number of sample concentration levels other than the actual preset concentration level. Construct probability distribution labels for each of the sample images based on the first value and each of the second values; The step of determining the model loss value based on a preset loss function, preset concentration levels of each sample, and predicted probability distributions of each sample includes: The model loss value is determined based on the preset loss function, the probability distribution labels, and the predicted probability distribution of each sample.

[0013] In one embodiment, the preset training conditions include: The overall accuracy of the initial neural network model reaches a preset accuracy threshold, and the overall accuracy is obtained by statistically analyzing the proportion of the number of correctly predicted sample images to the total number of sample images. And / or, the macro-average F1 score of the initial neural network model reaches a preset threshold, the macro-average F1 score is obtained by the arithmetic mean of the F1 scores of each preset concentration level of the samples, and the arithmetic mean of the F1 scores of each preset concentration level of the samples is obtained by the precision and recall corresponding to each preset concentration level of the samples.

[0014] In one embodiment, the step of determining a corresponding first adjustment signal based on the current blood concentration and controlling the first flushing pump to flush the surgical area according to the first adjustment signal includes: The corresponding rotation speed increment is determined based on the current blood concentration, and the preset initial rotation speed is increased by the rotation speed increment. The current blood concentration is positively correlated with the rotation speed increment. The first adjustment signal is determined based on the preset initial rotation speed after the speed increase, and the first flushing pump is controlled to flush the surgical area according to the first adjustment signal.

[0015] In one embodiment, the step of controlling the first irrigation pump to irrigate the surgical area according to the first adjustment signal includes: The target speed range of the preset initial speed after the acceleration is determined according to the first adjustment signal, and the current bleeding state is determined based on the target speed range; The target time duration is determined based on the current bleeding status, and the first flushing pump is controlled according to the first adjustment signal to continuously flush the surgical area for the target time duration.

[0016] In one embodiment, after the step of controlling the first irrigation pump according to the first adjustment signal to continuously irrigate the surgical area for the target time period, the method further includes: When the target timeout period ends and the current blood concentration decreases to a preset concentration threshold, the preset initial rotation speed after acceleration is decelerated according to a preset step size until the preset initial rotation speed is reached.

[0017] In one embodiment, the endoscope system further includes: a pressure acquisition component for acquiring the current cavity pressure within the surgical area; and a second flushing pump provided on the water supply line for controlling the cavity pressure within the surgical area. After the step of controlling the first irrigation pump to irrigate the surgical area according to the first adjustment signal, the method further includes: Determine the pressure difference between the current cavity pressure and the preset pressure threshold; Based on the pressure difference, a corresponding second adjustment signal is determined, and the second flushing pump is controlled according to the second adjustment signal to maintain the cavity pressure in the surgical area at the preset pressure threshold.

[0018] In addition, to achieve the above objectives, this application also proposes an endoscope system, which includes: an endoscope main unit, an endoscope scope, and a water supply pipeline; The water supply pipeline is equipped with a first flushing pump, which is used to control the flushing speed of the saline solution in the water supply pipeline on the surgical area. The endoscope host includes: a memory, a processor, and an automatic flushing program based on a neural network model stored in the memory and executable on the processor. When the automatic flushing program based on the neural network model is executed by the processor, it implements the steps of the automatic flushing method based on the neural network model described above.

[0019] This application provides an automatic flushing method and endoscope system based on a neural network model. The method is applied to the endoscope host in the endoscope system. The endoscope system further includes: a water supply pipeline and an endoscope scope. A first flushing pump is provided on the water supply pipeline, which is used to control the flushing speed of physiological saline in the water supply pipeline on the surgical area. The method includes: acquiring endoscopic images of the surgical area using the endoscope scope; identifying the endoscopic images using a preset neural network model to obtain the current blood concentration of the surgical area, wherein the preset neural network model is trained using sample images and corresponding sample blood concentrations; determining a corresponding first adjustment signal based on the current blood concentration, and controlling the first flushing pump to flush the surgical area according to the first adjustment signal.

[0020] This application pre-trains a preset neural network model using sample images and corresponding blood concentrations. During actual use, an endoscopic image of the surgical area is acquired via an endoscope. The preset neural network model then identifies this image to determine the current blood concentration in the surgical area. Based on this concentration, a first adjustment signal for the first flushing pump is determined, controlling the pump to flush the surgical area at the speed corresponding to that signal. Because this application can automatically control the flushing pump based on the current blood concentration obtained through the preset neural network model, it eliminates the need for manual control compared to existing methods, thus improving the timeliness of flushing and enhancing safety. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the endoscope host structure of the hardware operating environment involved in the embodiments of this application; Figure 2 This is a schematic flowchart of the first embodiment of the automatic flushing method based on a neural network model of this application; Figure 3 This is a schematic diagram of the endoscope system in the first embodiment of the automatic flushing method based on a neural network model in this application; Figure 4 This is a schematic diagram of the algorithm flow in the first embodiment of the automatic flushing method based on a neural network model in this application; Figure 5 This is a flowchart illustrating the second embodiment of the automatic flushing method based on a neural network model according to this application; Figure 6 This is a flowchart illustrating the third embodiment of the automatic flushing method based on a neural network model in this application; Figure 7 This is a schematic diagram of the control strategy flow of the automatic flushing method based on a neural network model in this application; Figure 8 This is a flowchart of the PID algorithm in the third embodiment of the automatic flushing method based on a neural network model in this application.

[0024] Explanation of icon numbers:

[0025] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0027] Reference Figure 1 , Figure 1 This is a schematic diagram of the endoscope host 1, which is part of the hardware operating environment involved in the embodiments of this application.

[0028] like Figure 1 As shown, the endoscope host 1 may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include an interface for connecting to a display screen; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. In this application, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0029] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the endoscope host 1, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0030] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an automatic flushing program based on a neural network model.

[0031] exist Figure 1In the endoscope host 1 shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the endoscope host 1 calls the automatic flushing program based on the neural network model stored in the memory 1005 through the processor 1001 and executes the automatic flushing method based on the neural network model provided in the embodiments of this application.

[0032] It should be noted that currently, during endoscopic surgery, the surgical area typically requires infusion of saline solution. Generally, a dedicated saline solution delivery line 21 is provided, which can be independently installed or utilize the existing channels of the endoscope 2. In this embodiment, a saline solution delivery line 21 can be provided in the sheath portion of the endoscope 2, and an irrigation pump can be installed on the delivery line 21 to inject saline solution into the surgical area. During the surgery, the surgical procedure inevitably touches blood vessels 7, leading to bleeding. The mixture of blood and saline solution reduces the transparency of the water, making the images captured by the endoscope 2 blurry, severely affecting the surgeon's identification of the anatomical structures in the surgical area and the precision of the surgical procedure.

[0033] Currently, to promptly remove blood and fluid, a manual method is generally used. This involves the doctor or assistant adjusting the speed of the irrigation pump via a foot pedal or hand control while observing the surgical procedure. However, this method is prone to problems such as delayed irrigation, prolonged surgical time, and lower safety standards.

[0034] Therefore, to address the aforementioned shortcomings, this embodiment provides an automatic flushing method based on a neural network model. This embodiment pre-trains a preset neural network model using sample images and corresponding blood concentrations. During actual use, an endoscopic image of the surgical area is acquired via an endoscope. The preset neural network model then identifies this endoscopic image to obtain the current blood concentration in the surgical area. Based on this current blood concentration, a first adjustment signal for the first flushing pump 3 is determined, controlling the pump to flush the surgical area at the speed corresponding to the first adjustment signal. Since this embodiment can identify the current blood concentration in the surgical area using a preset neural network model and automatically control the first flushing pump 3 accordingly, compared to existing methods requiring manual control, this embodiment eliminates the need for manual intervention, thereby improving the timeliness of flushing and ultimately enhancing safety.

[0035] For ease of understanding, the following is combined with Figures 2 to 8 The automatic flushing method based on a neural network model provided in the embodiments of this application will be described in detail.

[0036] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the automatic rinsing method based on a neural network model according to this application. The first embodiment of the automatic rinsing method based on a neural network model according to this application is presented as follows: Figure 2 As shown, in this embodiment, the specific method includes: Step S10: Use the endoscope to acquire endoscopic images of the surgical area.

[0037] It is understood that the method of this embodiment can be applied to any device with data processing, program execution and positioning functions, such as the endoscope host 1 in an endoscope system, and this embodiment does not limit it. However, for ease of understanding, this embodiment uses the endoscope host 1 (hereinafter referred to as the device) as the execution subject to describe this embodiment and the following embodiments.

[0038] It is also understood that the endoscope system described in this embodiment further includes: a water supply line 21 and an endoscope sight. The endoscope 2 is equipped with an endoscope sight and a water supply line 21. Of course, the water supply line 21 may not be part of the endoscope 2 and may be set up separately. This embodiment uses an endoscope 2 with an endoscope sight and a water supply line 21 for description. The water supply line 21 may be set at the endoscope sheath. The water supply line 21 is equipped with a first flushing pump 3. The first flushing pump 3 is used to control the flushing speed of the saline solution in the water supply line 21 on the surgical area.

[0039] For ease of understanding, please refer to Figure 3 , Figure 3 This is a schematic diagram of the endoscope system in the first embodiment of the automatic flushing method based on a neural network model according to this application. Figure 3 As shown, the aforementioned endoscope system can be a complete medical device system for performing minimally invasive endoscopic surgery. For example, it typically includes an endoscope host 1, an endoscope 2, a light source, etc.

[0040] Figure 3 The aforementioned endoscope 2 can be inserted through natural body cavities or tiny incisions to observe the interior, such as... Figure 3 The endoscope 2 contains internal tissue 6, and its front end can be equipped with an endoscope sight and lighting equipment. The endoscope 2 is connected to the endoscope host 1 and is used to transmit the internal images collected by the endoscope sight to the endoscope host 1 for processing and display on an external display screen.

[0041] It should be emphasized that the endoscope sight in this embodiment can be an optical lens assembly located on the endoscope 2 itself, i.e., it is used in optical endoscope scenarios. Of course, it can also be a sight located in front of the camera on the endoscope 2, i.e., it is used in electronic endoscope scenarios. The specific position of the endoscope sight can be set according to the actual situation, and this embodiment does not limit it.

[0042] It should also be emphasized that the aforementioned water supply pipeline 21 can be used to supply water to the surgical area (i.e., Figure 3 6) Channels for perfusing physiological saline into the body's tissues, such as Figure 3 As shown, a three-channel system can be used inside the endoscope sheath. The middle channel is the endoscope channel 2, and water supply pipes 21 can be set on both sides, including inlet and outlet channels. An opening can be set at the front end of the endoscope sheath between the two channels, so that physiological saline can enter from the inlet of the inlet channel and flow out from the front end of the endoscope sheath to the surgical area in the internal tissue 6. At the same time, the physiological saline in the surgical area flows from the front end of the endoscope sheath into the outlet channel and is discharged from the outlet through the outlet channel, thereby achieving flushing.

[0043] In order to control the rinsing speed with saline solution, such as Figure 3 As shown, a flushing pump can generally be installed on the water supply pipeline 21, either on the inlet channel or the outlet channel. In this embodiment, the first flushing pump 3 is installed at the inlet of the inlet channel. The first flushing pump 3 can be electrically connected to the endoscope host 1 and receive control signals from the endoscope host 1 to precisely drive the saline solution to be injected into the surgical area through the water supply pipeline 21 at a specific flow rate.

[0044] It should be understood that the aforementioned surgical area can be a localized area within the body where the surgical procedure is being performed, as seen under the view of endoscope 2. For example... Figure 3 As shown, the surgical area can be filled with infused saline solution to form the operating space. The endoscopic images described above are images representing the real-time state of the surgical area, acquired by the endoscope.

[0045] In practical use, refer to Figure 4 , Figure 4 This is a schematic diagram of the algorithm flow in the first embodiment of the automatic flushing method based on a neural network model of this application. Figure 4 The endoscope located in endoscope 2, as shown, can acquire observation images of the surgical area, and then... Figure 4 The camera in the middle is converted into endoscopic images, by Figure 4 The image acquisition module of the central endoscope host 1 receives the image.

[0046] Step S20: The endoscopic image is identified by a preset neural network model to obtain the current blood concentration in the surgical area. The preset neural network model is trained using sample images and corresponding sample blood concentrations.

[0047] It should be noted that the aforementioned preset neural network model can be a pre-trained model with fixed parameters, or a residual network, etc., and this embodiment does not impose any limitations on it. In this embodiment, the aforementioned preset neural network model can have the ability to identify and evaluate blood concentration in endoscopic images. Furthermore, before use, the aforementioned device can use a large number of sample images (which can be images acquired by historical endoscopic endoscopes) and their corresponding sample blood concentrations (which can be the corresponding real blood concentrations) pre-annotated by experts as training data to train the model and obtain the aforementioned preset neural network model.

[0048] It should also be noted that the aforementioned current blood concentration can be the blood content within the surgical area. It can be a specific concentration value (such as 0-100%), or it can be a discrete level (such as level 0-4, where level 0 is the clearest and level 4 is the most blurry, etc.). This embodiment uses discrete levels, specifically level 0 to level 4 from clear to blurry, but this is not a limitation.

[0049] In practical use, the endoscope host 1 can input endoscopic images into a preset neural network model, which can then recognize the images and ultimately generate a recognition result at its output layer, namely the aforementioned current blood concentration (i.e., Figure 4 (Medium resolution evaluation algorithm).

[0050] Step S30: Determine the corresponding first adjustment signal based on the current blood concentration, and control the first flushing pump 3 to flush the surgical area according to the first adjustment signal.

[0051] It is understood that the aforementioned first adjustment signal can be a signal corresponding to a first target rotation speed, which can be the rotation speed executed by the first flushing pump 3. This can be obtained based on the current blood concentration according to a preset mapping table, or through real-time calculation. This embodiment does not limit the specific mapping table or real-time calculation method.

[0052] In actual use, during surgery, it is inevitable to encounter blood vessels 7, causing bleeding that flows into the surgical cavity and affects viewing. After obtaining the current blood concentration, the endoscopic host 1 can determine the corresponding first adjustment signal (i.e., ...) based on the current blood concentration. Figure 4 The control strategy module generates the water injection flow rate. Generally, the higher the current blood concentration, the stronger the flushing needs to be, and thus the higher the first target rotation speed corresponding to the first adjustment signal. Then, the endoscope host 1 will send the first adjustment signal containing the first target rotation speed (i.e., Figure 4 The water pump control module sends the first adjustment signal to the first flushing pump 3. After receiving the first adjustment signal, the first flushing pump 3 immediately adjusts its speed to the first target speed (i.e., the first target speed). Figure 4The first flushing pump 3 is located at the water inlet, allowing for adjustments to the saline injection rate and thus improving the flushing of the surgical area. Furthermore, this embodiment utilizes a preset neural network model to identify the current blood concentration in the surgical area and automatically controls the first flushing pump 3 accordingly. Compared to existing methods requiring manual control, this embodiment eliminates the need for manual intervention, thereby improving the timeliness of flushing and ultimately enhancing safety.

[0053] Furthermore, considering that human blood absorbs light primarily in the 400nm~600nm range within the visible light spectrum, while 530nm falls under the green light category, in bleeding / blood-covered scenarios, the red light channel absorbs less strongly to concentration changes, while the green channel is more sensitive to concentration. Therefore, the blood concentration can be characterized by the intensity of green light. Consequently, in this embodiment, the step of identifying the endoscopic image using a preset neural network model to obtain the current blood concentration in the surgical area includes: Step S21: Determine the red channel intensity value and the green channel intensity value of each pixel in the endoscopic image.

[0054] It should be noted that in digital color images, the color of each pixel is typically composed of the intensity values ​​of three channels: red (R), green (G), and blue (B). The intensity value of each channel is a numerical value (e.g., 0-255 in an 8-bit image), representing the strength of that color component. Therefore, in this embodiment, the aforementioned red channel intensity value can be the intensity value of the red (R) channel in the endoscopic image, and the aforementioned green channel intensity value can be the intensity value of the green (G) channel in the endoscopic image.

[0055] In practical use, the above-mentioned devices acquire endoscopic images. Then, for each pixel (x, y) in the endoscopic image, the intensity value of its R channel can be obtained. As mentioned above, the intensity value of the red channel and the intensity value of the G channel are obtained. This serves as the strength value for the aforementioned green channel.

[0056] Step S22: Determine the blood sensitivity intensity corresponding to each pixel based on the intensity values ​​of each red channel and the corresponding intensity values ​​of the green channel.

[0057] Understandably, the aforementioned blood sensitivity intensity can be a blood concentration-sensitive intensity constructed based on the red channel intensity value and the green channel intensity value. Since the green intensity differs under different lighting intensities at the same blood concentration, and red absorption is minimal in bleeding scenes, the red intensity remains the same under different blood concentrations, the difference in green intensity caused by varying lighting intensities can be corrected using red intensity. By calculating the red and green channel intensity values ​​of each pixel, the visual changes caused by blood are amplified while suppressing interference from lighting changes, thus obtaining the aforementioned blood sensitivity intensity for each pixel. For ease of subsequent explanation, the pixel... The intensity of blood sensitivity can be recorded as .

[0058] Step S23: Perform channel replication on each of the blood sensitivity intensities, and stitch the replication results together to obtain a stitched image; Step S24: The stitched image is identified using a preset neural network model to obtain the current blood concentration in the surgical area.

[0059] It should be understood that, in order to adapt to the pre-trained neural network model, since the aforementioned preset neural network model usually requires a three-channel image as input, in this embodiment, after obtaining the blood sensitivity intensity of each pixel, the three-channel amplitudes can be performed and then stitched together to form... The stitched image. Then the obtained stitched image... The blood is input into a preset neural network model, which then yields the corresponding current blood concentration.

[0060] Similarly, when training the model, we can first determine the intensity values ​​of the red channel and the green channel of the sample image, determine the blood sensitivity intensity of each sample based on the intensity values ​​of the red channel and the green channel, then perform channel amplitude calculation on the blood sensitivity intensity of each sample to obtain a stitched image of the samples, and finally train the model with the blood concentration of the samples.

[0061] Furthermore, to eliminate the influence of differences in imaging brightness and exposure, in this embodiment, the step of determining the blood sensitivity intensity corresponding to each pixel based on the intensity values ​​of each red channel and the corresponding intensity values ​​of the green channel includes: Step S221: Compare the intensity values ​​of each red channel with a preset small normal number to obtain a first comparison result, and take the larger value in each of the first comparison results as the corresponding normalized reference value.

[0062] It should be noted that the preset small normal number can be a pre-set positive number with a very small value. For ease of understanding, this embodiment uses ε as the preset small normal number (ε > 0). The preset small normal number can be used to prevent the denominator from being zero and ensure the stability of mathematical calculations. That is, when the red channel intensity value of a pixel in the endoscopic image is 0, directly using it as the denominator will lead to calculation overflow or generate an infinitely large invalid value. Therefore, the above-mentioned preset small normal number can be introduced to ensure that the denominator has a safe minimum positive value under any circumstances.

[0063] It should also be noted that the aforementioned normalization benchmark can be a reference benchmark established for the red channel intensity value of each pixel. In practical use, the device can first compare each red channel intensity value with a preset small normal value, and take the larger value as the aforementioned normalization benchmark. Specifically, it can be... Where max is the larger value among the first comparison results mentioned above.

[0064] Step S222: Divide the preset normalization constant by each of the normalization reference values ​​to obtain the corresponding scaling factor.

[0065] It is understood that the aforementioned preset normalization constant can be used to adjust the numerical range of the calculation result to a standard interval. It can be the maximum value of the image data type; for example, for an 8-bit image, the preset normalization constant could be 255. Of course, it can also be other values, and this embodiment does not impose any limitations on this. For ease of subsequent explanation, this embodiment will use the preset normalization constant α = 255 for explanation.

[0066] It is also understood that the aforementioned scaling factor can be a scaling factor required to increase (or decrease) the normalized reference value of a pixel to the standard intensity α, i.e. .

[0067] Step S223: Multiply each of the green channel intensity values ​​by the corresponding scaling factor to obtain the blood sensitivity intensity corresponding to each pixel.

[0068] In practical use, after obtaining the scaling factor for each pixel, it can be multiplied by the corresponding green channel intensity value to obtain the result after normalizing the green channel intensity value using the red channel intensity value. This result serves as the aforementioned blood sensitivity intensity, i.e., the blood sensitivity intensity. .

[0069] Furthermore, in this embodiment, by normalizing the green channel intensity value pixel-by-pixel based on the red channel intensity value, the calculated feature value remains unchanged. This enhances the system's adaptability and recognition stability under different surgical environments and different patient tissue reflectivity characteristics. Moreover, by pre-setting a small normal value and taking the maximum value, it ensures that even in extreme pixels where pure black or red channel information is missing, a reasonable finite value can still be output. This significantly improves reliability and security.

[0070] Furthermore, to suppress the influence of extreme brightness / saturation points, in this embodiment, the step of channel replication for each of the aforementioned blood sensitivity intensities includes: Step S231: Determine the cutoff threshold for each of the blood sensitivity intensities.

[0071] It should be understood that the aforementioned cropping threshold can be a threshold used to distinguish normal tissue / blood pixels from abnormally bright pixels (such as reflective points). For ease of subsequent explanation, the aforementioned cropping threshold in this embodiment can be denoted as h. Furthermore, in this embodiment, the aforementioned cropping threshold can be taken as the upper quantile (e.g., the 95th or 99th percentile) of all blood sensitivity intensities, which means that 95% (or 99%) of the blood sensitivity intensities are less than or equal to this threshold.

[0072] Specifically, the preset percentile clipping parameter p can be obtained first. (This embodiment uses p=99 for illustration). Furthermore, in actual use, the above device obtains the blood sensitivity intensity of each pixel. Next, you can first set the values ​​of each pixel. Sort the data, then find its p-th percentile (e.g., p=99). This percentile value can then be set as the aforementioned clipping threshold h. , This is a percentile operation.

[0073] Step S232: Compare each of the blood sensitivity values ​​with the cropping threshold to obtain a second comparison result, and take the smaller value in each of the second comparison results as the corresponding pixel intensity value.

[0074] It should also be understood that the aforementioned pixel intensity value can be a new intermediate intensity value for each pixel after comparing it with the cropping threshold and taking the smaller value. In this embodiment, after obtaining the cropping threshold, the cropping threshold can be compared with the blood sensitivity intensity of each pixel, and the smaller value can be taken as the aforementioned pixel intensity value. The `min` operation involves taking the smaller value. Furthermore, by taking the smaller value, all abnormally high values ​​exceeding the clipping threshold `h` can be limited to a level consistent with the surrounding normal area, thus eliminating extreme values.

[0075] Step S233: Divide each pixel intensity value by the cropping threshold to obtain the corresponding normalized blood sensitivity intensity, and perform channel copying on each normalized blood sensitivity intensity.

[0076] It is important to emphasize that after obtaining the pixel intensity value for each pixel, in order to unify the scale, each pixel intensity value can be divided by the cropping threshold, and the result can be used as the normalized blood sensitivity intensity mentioned above. That is, if the normalized blood sensitivity intensity is denoted as... ,but After obtaining the normalized blood sensitivity intensity for each pixel, three-channel copying and stitching can be performed to obtain a format that meets the input requirements of the preset neural network model, i.e., a stitched image. .

[0077] In endoscopic surgery, specular reflections from tissue surfaces, fat, or instruments are common interferences, distorting color and brightness information and being unrelated to blood. Therefore, in this embodiment, the aforementioned cropping threshold eliminates abnormally high values, preventing misleading the neural network model and improving its accuracy and robustness in complex surgical scenarios. Furthermore, the cropping threshold is calculated in real-time for each frame (e.g., using the 99th percentile), not a fixed preset value. This allows for adaptive adjustment to the overall brightness level of images under different surgical stages and tissue characteristics, achieving fully automatic and adaptive preprocessing without requiring any parameter adjustments by doctors or engineers. This simplifies operation and improves the system's adaptability and versatility.

[0078] This embodiment pre-trains a preset neural network model using sample images and corresponding blood concentrations. During actual use, an endoscopic image of the surgical area is acquired via an endoscope. The preset neural network model then identifies this image to determine the current blood concentration in the surgical area. Based on this concentration, a first adjustment signal for the first flushing pump 3 is determined, controlling the pump to flush the surgical area at the corresponding rotational speed. Because this embodiment uses the preset neural network model to identify the current blood concentration and automatically controls the first flushing pump 3 accordingly, it eliminates the need for manual control compared to existing methods, thus improving the timeliness of flushing and enhancing safety.

[0079] Reference Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the automatic flushing method based on a neural network model of this application. Based on the first embodiment described above, the second embodiment of the automatic flushing method based on a neural network model of this application is proposed.

[0080] In the above embodiments, the current blood concentration is divided into levels 0 to 4, corresponding to no bleeding (level 0) to massive bleeding (level 4). To enable the preset neural network model to output the aforementioned current blood concentration, such as... Figure 4 As shown, in this embodiment, the step of identifying the endoscopic image using a preset neural network model to obtain the current blood concentration in the surgical area includes: Step S241: The endoscopic image is identified by a preset neural network model to obtain the original scores of the blood concentration in the surgical area belonging to each preset concentration level.

[0081] It should be noted that the aforementioned raw scores can be the bias scores of the preset neural network model in determining whether the input endoscopic image belongs to each preset concentration level. A higher score indicates a stronger bias towards that preset concentration level. In this embodiment, levels 0 to 4 are used as the preset concentration levels for explanation. Therefore, during recognition, the preset neural network model can output 5 raw scores, each corresponding to a preset concentration level. For ease of understanding, the raw score corresponding to the k-th preset concentration level can be denoted as... .

[0082] Step S242: Perform probability transformation on each of the original scores to obtain the predicted probability distribution belonging to each of the preset concentration levels; Step S243: Select the target preset concentration level corresponding to the highest predicted probability from each of the predicted probability distributions, and use the target preset concentration level as the current blood concentration of the surgical area.

[0083] It should also be noted that the above probability transformation can be a process used to convert all original scores into probability values ​​between 0 and 1. The above predicted probability distribution can be the probability distribution obtained after probability transformation, representing the probability distribution of the input endoscopic image X belonging to each preset concentration level.

[0084] Understandably, the above probability transformation can be implemented using the Softmax function, although it can also be implemented in other ways. This embodiment uses the Softmax function for illustration. Furthermore, if the probability distribution of the input endoscopic image X belonging to the k-th preset concentration level is denoted as... ,but ,in It is a natural constant.

[0085] After obtaining the predicted probability distribution of each preset concentration level, the highest predicted probability can be selected and the corresponding preset concentration level can be taken as the target preset concentration level. This target preset concentration level can then be the current blood concentration of the input endoscopic image X. Specifically, if the target preset concentration level is denoted as... ,but .

[0086] Furthermore, in order to train and obtain the aforementioned preset neural network model, in this embodiment, before the step of recognizing the endoscopic image using the preset neural network model, the following steps are also included: Step S01: Obtain each sample image and its corresponding preset concentration level, and use an initial neural network model to identify each sample image to obtain the sample prediction probability distribution of each sample image belonging to each preset concentration level.

[0087] It should be noted that the aforementioned sample images can be pre-collected historical images of endoscopic surgeries used for training and testing the neural network during the model development phase. The preset concentration levels of the samples can be real-world label levels (e.g., 0, 1, 2, 3, 4) manually assigned by medical experts to each sample image, representing the blood concentration in that image. In this embodiment, the aforementioned sample images and preset concentration levels can constitute a training set, denoted as... ,in For the i-th sample image, The preset concentration level is set for the sample corresponding to the i-th sample image, and N is the total number of N.

[0088] It should also be noted that the initial neural network model mentioned above can be a neural network model that does not yet have accurate recognition capabilities at the beginning of the training process. The sample prediction probability distribution mentioned above can be the probability distribution of each preset concentration level output after the sample image is input into the initial neural network model, undergoes forward propagation and Softmax transformation.

[0089] In practical use, the above-mentioned device can extract a batch of sample images and input them into an initial neural network model. The model can then infer from the sample images and output a sample prediction probability distribution for each sample image.

[0090] Step S02: Determine the model loss value based on the preset loss function, the preset concentration level of each sample, and the predicted probability distribution of each sample; Step S03: Adjust the parameters of the initial neural network model according to the model loss value, and return to the step of recognizing each sample image through the initial neural network model until the preset training conditions are met to obtain the preset neural network model.

[0091] It is understood that the aforementioned preset loss function can be a function used to quantify the difference between the model's predicted results (sample predicted probability distribution) and the actual situation (sample preset concentration level). In this embodiment, cross-entropy can be used as the preset loss function, but other functions can also be used, and this embodiment does not limit this. The aforementioned model loss value can be a value representing the degree of deviation obtained by calculating the sample image using the aforementioned preset loss function.

[0092] In practical use, the sample prediction probability distribution of the initial neural network model for the sample images, along with the corresponding preset concentration levels, can be input into the preset loss function to obtain the model loss value. Then, using the backpropagation algorithm, starting from the final model loss value, the gradient of the model loss value with respect to each trainable parameter (θ) of the initial neural network model is calculated backwards. Next, an optimizer (such as stochastic gradient descent or Adam) is used to update the parameters of the initial neural network model according to the calculated gradient and a preset learning rate, thereby generating a lower model loss value in the next iteration. After completing one parameter update, the process returns to the step of recognizing each sample image using the initial neural network model, processing the sample images with the updated initial neural network model, starting a new loop until the preset training conditions are met. The training loop then stops, saving the model parameters at this point as the final deployable preset neural network model.

[0093] Furthermore, considering that there may be relatively few sample images of massive hemorrhage (level 4), which would cause the model to ignore this category during training, in order to improve the accuracy of the model, in this embodiment, the step of determining the model loss value based on a preset loss function, preset concentration levels of each sample, and predicted probability distributions of each sample includes: Step S021: Determine the occurrence ratio of each preset concentration level of the sample, and assign a corresponding category weight to each preset concentration level of the sample according to the occurrence ratio, wherein the occurrence ratio is negatively correlated with the category weight.

[0094] It should be noted that the above occurrence ratio can be the percentage of sample images belonging to a specific sample predefined concentration level (e.g., level 4: severe hemorrhage) in the entire training dataset, out of the total number of sample images. For example, if the occurrence ratio of level k is denoted as... ,but =Number of sample images with level k / Total number of sample images.

[0095] It should also be noted that the aforementioned category weights can be importance coefficients assigned to each preset concentration level, which can be used to amplify or reduce the contribution of samples of that category in the total loss calculation. In this embodiment, to avoid ignoring the importance of fewer samples, the occurrence ratio can be negatively correlated with the category weight, that is, the fewer samples a category has (the lower the occurrence ratio), the larger its assigned weight value. Specifically, an inverse proportional weighting can be used, that is, the category weight of the preset concentration of the k-th sample can be... Of course, there are other methods as well, and this embodiment does not limit them.

[0096] Step S022: Determine the original loss value based on the preset loss function, the preset concentration level of each sample, and the predicted probability distribution of each sample.

[0097] It is understandable that the original loss value mentioned above can be the loss value calculated for a single sample image based on a preset loss function (such as cross-entropy) without considering class weights.

[0098] In practical use, the above-mentioned equipment can first determine the occurrence ratio of preset concentration levels for each sample. Then, based on the proportion of each occurrence Assign corresponding category weights Then, predictions are made on the sample images to obtain the sample prediction probability distribution. For each sample, the cross-entropy formula is used. Calculate its original loss value, where To predict the i-th sample image This is a real sample preset concentration level The probability of.

[0099] Step S023: The original loss values ​​are weighted and summed according to the corresponding category weights and then averaged to obtain the model loss value.

[0100] After obtaining the raw loss value for each sample image, the corresponding class weight can be found based on its actual sample preset concentration level. Then, the original loss value of the sample image is multiplied by the class weight, and the loss values ​​of all sample images are summed. Dividing this sum by the number of sample images N yields the final model loss value for this iteration. ,in For predicting only the i-th sample image preset concentration level A weighted summation is performed when the concentration level of the k-th real sample is preset. After obtaining the model loss value, backpropagation and parameter updates are performed, and the preset neural network model can be obtained when the preset training requirements are met.

[0101] Furthermore, considering that using hard labels when setting preset concentration levels for samples may force the model to learn annotation noise and random features that may exist in the training data, resulting in poor generalization ability when facing new data, this embodiment, in order to improve the robustness of the model, after the step of obtaining each sample image and the corresponding preset concentration level, further includes: Step S011: Take the preset concentration level of each sample image as the true preset concentration level.

[0102] It should be noted that the aforementioned true preset concentration levels can be concentration levels labeled by medical experts for each sample image as a training gold standard. For example, for a sample image, if its original label is a hard label and the true preset concentration level is 2, it can be represented as [0, 0, 1, 0, 0]. That is, the five values ​​from left to right correspond to levels 0 to 4, and the value of 1 indicates that the true preset concentration level is that level.

[0103] Step S012: Set the label value representing the true preset concentration level to a first value, and set the label values ​​representing the concentration levels of each sample other than the true preset concentration level to a second value, where the first value is 1- The second value is / n, The preset smoothing coefficient is n, which is the number of sample concentration levels other than the actual preset concentration level. Step S013: Construct probability distribution labels for each of the sample images based on the first value and each of the second values.

[0104] It should also be noted that the preset smoothing coefficient mentioned above can be a pre-set small value between 0 and 1 (e.g., 0.1). This can be used to control the intensity of label smoothing. The larger the value, the smoother the surface, and the more conservative the model's predictions will be. The smaller the size, the closer the label is to the original hard label.

[0105] Understandably, the aforementioned first value could be a label value assigned to the true preset concentration level within the smoothed new label. Since even with expert annotation, the probability of a sample image belonging to the true category is not absolutely 100%, but rather has a very high probability, such as 0.9 (i.e., when...). Therefore, in this embodiment, a preset smoothing coefficient can be set (e.g., when =0.1). And set the label value representing the actual preset concentration level to 1- This ensures noise and randomness are minimized.

[0106] It is also understood that the aforementioned second value can be the label value of each category other than the true category, evenly distributed among the smoothed new labels. In this embodiment, the second value can be evenly distributed according to the number of levels of each sample concentration level other than the true preset concentration level, based on a preset smoothing coefficient. For example, when When the value is 0.1, there are a total of 4 categories remaining. The second value... / n=0.1 / 4=0.025. Therefore, the probability distribution label of this sample image can be changed from [0, 0, 1, 0, 0] to [0.025, 0.025, 0.9, 0.025, 0.025].

[0107] The step of determining the model loss value based on a preset loss function, preset concentration levels of each sample, and predicted probability distributions of each sample includes: Step S024: Determine the model loss value based on the preset loss function, the probability distribution labels, and the predicted probability distribution of each sample.

[0108] After smoothing each preset sample concentration level to obtain the corresponding probability distribution label, when it is necessary to calculate the loss value, the original hard label [0, 0, 1, 0, 0] is no longer used. Instead, the newly generated probability distribution label [0.025, 0.025, 0.9, 0.025, 0.025] is used. That is, the difference between the sample prediction probability distribution predicted by the model and the probability distribution label is calculated to obtain the model loss value.

[0109] Because label smoothing was performed in this embodiment, the model is prevented from becoming overconfident in the training set, thereby learning more essential and generalized blood concentration features, which significantly improves the stability of the model under different surgical scenarios and different doctors' operating habits.

[0110] Furthermore, in order to obtain a higher quality model, in this embodiment, the preset training conditions include: The overall accuracy of the initial neural network model reaches a preset accuracy threshold, and the overall accuracy is obtained by statistically analyzing the proportion of the number of correctly predicted sample images to the total number of sample images. And / or, the macro-average F1 score of the initial neural network model reaches a preset threshold, the macro-average F1 score is obtained by the arithmetic mean of the F1 scores of each preset concentration level of the samples, and the arithmetic mean of the F1 scores of each preset concentration level of the samples is obtained by the precision and recall corresponding to each preset concentration level of the samples.

[0111] It should be noted that the overall accuracy mentioned above can be the ratio between the number of correctly predicted sample images and the total number of sample images, i.e., the overall accuracy. , The preset concentration level is set for the predicted sample image of the i-th sample. For example, if the model correctly classifies 920 out of 1000 validated sample images, the overall accuracy is 92%. The preset accuracy threshold can be a threshold set to determine whether the model is qualified, such as 90% or 95%, etc. It can be set according to the actual situation, and this embodiment does not limit it.

[0112] In practical use, after the model completes one iteration, it can calculate the overall accuracy based on the verification results and determine whether the overall accuracy reaches the preset accuracy threshold. If it does, it means that the model training is qualified, and training is stopped to obtain the preset neural network model. If it does not reach the threshold, it means that the model training is not qualified, and training continues.

[0113] It should also be noted that, in addition to using the overall accuracy, this embodiment can also use the macro-average F1 score as a criterion for determining whether training is successful. The aforementioned macro-average F1 score can be a more equitable, rigorous, and applicable comprehensive performance metric than the overall accuracy score, and is more suitable for class imbalance scenarios.

[0114] The calculation principle of the above macro-average F1 index can be achieved by first pre-setting the concentration level (e.g., 0, 1, 2, 3, 4) for each blood concentration sample, and then calculating the accuracy (denoted as F1) separately. Precision (i.e., the precision of the k-th sample at the preset concentration level) and recall (denoted as ) , which is the recall rate of the k-th sample at the preset concentration level.

[0115] The aforementioned accuracy rate can be calculated as the number of samples correctly predicted to the preset concentration level of that sample divided by the total number of samples predicted to the preset concentration level of that sample. That is... ,in To predict the number of samples that correctly set the concentration level for the k-th sample, The number of samples that are predicted to be the k-th sample is preset to a certain concentration level.

[0116] The recall rate mentioned above can be calculated as the number of samples correctly predicted to be at the preset concentration level of the sample divided by the total number of samples that actually reached the preset concentration level of the sample. That is... ,in To predict the number of samples that correctly set the concentration level for the k-th sample, The number of samples whose actual concentration level is the k-th sample.

[0117] After obtaining precision and recall, the arithmetic mean of the F1 scores for each of the preset concentration levels of the samples can be calculated, and this arithmetic mean can be used as the macro-average F1 index, i.e., the macro-average F1 index. .

[0118] In practical use, after the model completes one iteration, the precision and recall rates mentioned above can be calculated based on the validation results. Then, the macro-average F1 score is obtained based on the precision and recall rates, and it is determined whether the macro-average F1 score reaches the preset threshold. The preset threshold can be a threshold set to determine whether the model is qualified. If it is reached, it means that the model training is qualified, and training is stopped to obtain the preset neural network model. If it is not reached, it means that the model training is not qualified, and training continues.

[0119] Of course, in this embodiment, the overall accuracy and the macro average F1 index can be used together as preset training conditions. That is, only when the macro average F1 index reaches the preset index threshold and the overall accuracy reaches the preset accuracy threshold can the model be determined to be trained.

[0120] Reference Figure 6 , Figure 6 This is a flowchart illustrating the third embodiment of the automatic flushing method based on a neural network model of this application. Based on the above embodiments, the third embodiment of the automatic flushing method based on a neural network model of this application is proposed.

[0121] In order to determine a suitable first target rotation speed based on different current blood concentrations, in this embodiment, such as Figure 6 As shown, the step of determining a corresponding first adjustment signal based on the current blood concentration and controlling the first flushing pump 3 to flush the surgical area according to the first adjustment signal includes: Step S31: Determine the corresponding rotation speed increment based on the current blood concentration, and increase the preset initial rotation speed by the rotation speed increment, wherein the current blood concentration is positively correlated with the rotation speed increment; Step S32: Determine the corresponding first adjustment signal based on the preset initial rotation speed after the speed increase, and control the first flushing pump 3 to flush the surgical area according to the first adjustment signal.

[0122] It should be noted that the aforementioned rotation speed increment can be an increase in rotation speed based on a preset initial rotation speed. This increment can be dynamically obtained based on the current blood concentration. The larger the rotation speed increment, the more intense the flushing. In this embodiment, the current blood concentration is positively correlated with the rotation speed increment; that is, the higher the detected current blood concentration, the blurrier the image, and thus the larger the rotation speed increment can be. Conversely, the lower the concentration, the smaller or even zero the rotation speed increment. The aforementioned preset initial rotation speed can be a pre-set, relatively low base rotation speed, suitable for situations where there is no significant bleeding during surgery and only basic visual clarity and fluid exchange are required.

[0123] In practical use, a mapping table can be pre-set within the aforementioned device, storing the rotation speed increments corresponding to different current blood concentrations. For ease of understanding, refer to... Figure 7 , Figure 7 This is a schematic diagram of the control strategy flow of the automatic flushing method based on a neural network model in this application. Figure 7 As shown, when the current blood concentration is 0, the corresponding rotation speed increment can be 0; when the current blood concentration is 1, the corresponding rotation speed increment can be 1, and so on. When the current blood concentration is 4, the corresponding rotation speed increment can be 4. After determining the corresponding rotation speed increment, the preset initial rotation speed can be increased. A corresponding first adjustment signal is generated according to the increased preset initial rotation speed and transmitted to the first flushing pump 3 to control the first flushing pump 3 to perform flushing. This gradual increase in speed prevents the shock to the patient caused by sudden speed increases, thus improving safety.

[0124] Furthermore, to avoid continuous rinsing, in this embodiment, the step of controlling the first rinsing pump 3 to rinse the surgical area according to the first adjustment signal includes: Step S321: Determine the target speed range of the preset initial speed after the acceleration based on the first adjustment signal, and determine the current bleeding state based on the target speed range; Step S322: Determine the corresponding target time duration based on the current bleeding status, and control the first flushing pump 3 according to the first adjustment signal to continuously flush the surgical area for the target time duration.

[0125] It is understood that in this embodiment, the entire speed range can be pre-divided into several continuous intervals, such as... Figure 7 As shown, this may include, but is not limited to, 0 to 50 rpm, 50 to 100 rpm, 100 to 150 rpm, 150 to 200 rpm, and 200 to 280 rpm. Furthermore, after the device determines the preset initial speed after the increase in speed based on the first adjustment signal, the speed range within which the preset initial speed after the increase in speed falls can be determined as the aforementioned target speed range. Different speed ranges correspond to different current bleeding states; that is, 0 to 50 rpm corresponds to bleeding state 0, 50 to 100 rpm corresponds to bleeding state 1, 100 to 150 rpm corresponds to bleeding state 2, 150 to 200 rpm corresponds to bleeding state 3, and 200 to 280 rpm corresponds to bleeding state 4.

[0126] It is also understood that, in this embodiment, the device may also include a timer to time the duration for which the first flushing pump 3 rotates at the preset initial speed after the speed increase. Furthermore, after the device determines the corresponding current bleeding state, different timing durations can be set for different current bleeding states, such as... Figure 7 As shown, when the bleeding state switches to 0, the timer's duration is 0; when the bleeding state switches to 1, the timer's duration is 150 (corresponding to 30s); when the bleeding state switches to 2, the timer's duration is 300 (corresponding to 60s); when the bleeding state switches to 3, the timer's duration is 450 (corresponding to 90s); and when the bleeding state switches to 4, the timer's duration is 600 (corresponding to 120s). If the bleeding state is the same as the previous identification, it indicates that the bleeding has not changed, and the timer can be directly decremented by 1 (i.e., a time interval of 200ms), that is, the timer can start counting down.

[0127] Once the target time duration corresponding to the current bleeding state is determined, the first flushing pump 3 can be controlled according to the preset initial speed after the increase, and the flushing can continue for the target time duration.

[0128] Furthermore, considering that a significant reduction in the rotation speed of the first irrigation pump 3 after the timed period may easily lead to a drastic change in the pressure inside the surgical cavity, in this embodiment, after the step of controlling the first irrigation pump 3 according to the first adjustment signal to continuously irrigate the surgical area for the target timed duration, the method further includes: Step S323: When the target timeout period ends and the current blood concentration decreases to a preset concentration threshold, the preset initial rotation speed after the acceleration is decelerated according to a preset step size until the preset initial rotation speed is reached.

[0129] It should be understood that the aforementioned preset concentration threshold can be a threshold used to characterize the safety of the current blood concentration; in this embodiment, 0 can be used for illustration. The aforementioned preset step size can be a value used to control the rate of decrease in the rotation speed of the first flushing pump 3; in this embodiment, 4 revolutions per minute can be used for illustration.

[0130] like Figure 7 As shown, in actual use, when the above-mentioned equipment determines the target timing duration interpretation (i.e. Figure 7 The timer is 0), and the current blood concentration has decreased to the preset concentration threshold (i.e., Figure 7 If the concentration is 0%, it indicates that the blood has been largely flushed out. Therefore, the initial rotation speed can be reduced by a preset step size of -4 rpm until the next sampling. If this is not achieved, the rotation speed can continue at the preset initial speed for the next sampling.

[0131] Therefore, in this embodiment, the higher the blurriness, the higher the blood concentration, and the higher the blood concentration, the faster the speed. Then, the current bleeding amount (i.e., bleeding state) is determined based on the rotation speed. When the bleeding amount is small, the scene will become clearer after the rotation speed accumulates to a certain level, at which point the rotation speed will not continue to accumulate. Therefore, the bleeding amount can be determined by the rotation speed. Next, when the bleeding amount is large, the high-speed operation needs to be maintained for a longer period. The timer duration is determined based on the bleeding amount. When the timer duration accumulates to 0, the rotation speed gradually decreases. Finally, the water pump speed adjustment process under a bleeding scenario is completed.

[0132] Furthermore, in order to achieve constant pressure within the surgical area and reduce bleeding, in this embodiment, the endoscope system further includes: a pressure acquisition component 5, which is used to acquire the current cavity pressure within the surgical area; and a second flushing pump 4 is provided on the water supply pipeline 21, which is used to control the cavity pressure within the surgical area.

[0133] like Figure 3 As shown, this embodiment can also include a second flushing pump 4, which is installed on the water supply pipeline 21. It is only necessary to ensure that the surgical area is between the first flushing pump 3 and the second flushing pump 4; that is, the first flushing pump 3 can be located at the inlet, and the second flushing pump 4 at the outlet, or vice versa. This embodiment uses the second flushing pump 4 located at the outlet for illustration. Furthermore, the pressure within the surgical area cavity can be controlled by controlling the rotational speed of the second flushing pump 4.

[0134] It should also be noted that the pressure acquisition component 5 mentioned above can be a component that measures the fluid pressure inside the surgical cavity in real time, such as a pressure sensor. Figure 3 As shown, in this embodiment, the pressure acquisition component 5 can be set separately from the endoscope 2 and directly placed in the cavity of the surgical area and connected to the endoscope host 1. Alternatively, it can be set on the endoscope sheath and connected to the endoscope host 1 through the endoscope 2. There are no specific limitations on this.

[0135] After the step of controlling the first irrigation pump 3 to irrigate the surgical area according to the first adjustment signal, the method further includes: Step S40: Determine the pressure difference between the current cavity pressure and the preset pressure threshold; Step S50: Determine the corresponding second adjustment signal based on the pressure difference, and control the second flushing pump 4 according to the second adjustment signal to maintain the cavity pressure in the surgical area at the preset pressure threshold.

[0136] Understandably, the aforementioned preset pressure threshold can be the pressure value that the user sets to be maintained. (Refer to...) Figure 8 , Figure 8This is a flowchart of the PID algorithm in the third embodiment of the automatic flushing method based on a neural network model in this application. Figure 4 and Figure 8 As shown, users can set a preset pressure threshold (i.e. Figure 4 In the middle user control, the system interaction module sets the target water pressure, that is... Figure 8 The target concentration is set in the middle, which can then indicate that the pressure in the cavity needs to reach the preset pressure threshold.

[0137] After setting the preset pressure threshold, the device can continuously measure the current cavity pressure (i.e., pressure) within the surgical cavity through the pressure acquisition component 5. Figure 4 The medium pressure sensor, the water pressure acquisition component, collects the current water pressure, that is... Figure 8 (obtain the current concentration), and then determine the pressure difference e(t) between the current cavity pressure and the preset pressure threshold (i.e., Figure 8 The difference e(t) is calculated. Then, constant pressure control is achieved according to a preset control algorithm. In this embodiment, a PID control algorithm (i.e., ...) can be used. Figure 8 In the PID calculation, the pressure difference e(t) is input into the PID control algorithm (i.e., the constant pressure PID module). The algorithm can comprehensively calculate the second target speed (i.e., the constant pressure PID module) based on the current value (proportional P), historical cumulative value (integral I), and changing trend (derivative D) of the pressure difference e(t). Figure 8 (Output of medium speed value). The specific calculation process is as follows: .

[0138] Among them, K p For proportional gain; T t T is the integration time constant; D Let e(t) be the differential time constant; u(t) be the output adjustment signal of the PID controller (i.e., the second target speed). If e(t) > 0 (pressure too high), the algorithm can output a higher second adjustment signal, corresponding to the second target speed, instructing the water pump to accelerate pumping and quickly discharge the liquid to reduce pressure. If e(t) < 0 (pressure too low), the algorithm can output a lower or even zero second target speed second adjustment signal, instructing the water pump to decelerate or stop, allowing the injection water (first flushing pump 3) to naturally raise the pressure.

[0139] After obtaining the second target rotational speed (i.e.) Figure 4 (Medium pumping flow rate), the endoscope host 1 sends the second adjustment signal to the second flushing pump 4 (i.e., Figure 4 The water pump control module controls the water pump, and the second flushing pump 4 immediately adjusts its own speed to the value corresponding to the second adjustment signal, thereby changing the suction flow rate.

[0140] In addition, to achieve the above objectives, this application also provides an endoscope system, which may include: an endoscope host 1, an endoscope scope, and a water supply pipeline 21; The water supply pipeline 21 is equipped with a first flushing pump 3, which is used to control the flushing speed of the saline solution in the water supply pipeline 21 on the surgical area. The endoscope host 1 includes: a memory 1005, a processor 1001, and an automatic flushing program based on a neural network model stored in the memory 1005 and executable on the processor 1001. When the automatic flushing program based on the neural network model is executed by the processor, it implements the steps of the automatic flushing method based on the neural network model as described above.

[0141] Since the endoscope host 1 in this embodiment can be implemented in accordance with the specific implementation of the above method embodiment, the specific implementation of the endoscope system in this embodiment and the beneficial effects thereon can be referred to the above method embodiment. This embodiment will not elaborate on this.

[0142] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0143] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0145] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A neural network model-based automatic flushing method, characterized by, The method is applied to the endoscope host of an endoscope system; the endoscope system further includes: a water supply pipeline and an endoscope, wherein the water supply pipeline is equipped with a first flushing pump, the first flushing pump being used to control the flushing speed of physiological saline in the water supply pipeline on the surgical area; the method includes: Endoscopic images of the surgical area are acquired using the endoscope. The endoscopic image is identified by a preset neural network model to obtain the current blood concentration in the surgical area. The preset neural network model is trained using sample images and corresponding sample blood concentrations. A first adjustment signal is determined based on the current blood concentration, and the first flushing pump is controlled to flush the surgical area according to the first adjustment signal.

2. The method of claim 1, wherein, The step of identifying the endoscopic image using a preset neural network model to obtain the current blood concentration in the surgical area includes: Determine the red channel intensity value and the green channel intensity value of each pixel in the endoscopic image; The blood sensitivity intensity of each pixel is determined based on the intensity values ​​of each red channel and the corresponding intensity values ​​of the green channel. Channel replication is performed on each of the blood sensitivity intensities, and the replication results are stitched together to obtain a stitched image; The current blood concentration in the surgical area is obtained by recognizing the stitched image using a preset neural network model.

3. The method of claim 2, wherein, The step of determining the blood sensitivity intensity corresponding to each pixel based on the intensity values ​​of each red channel and the corresponding intensity values ​​of the green channel includes: Each red channel intensity value is compared with a preset small normal number to obtain a first comparison result, and the larger value in each first comparison result is used as the corresponding normalized reference value. Divide the preset normalization constant by each of the normalization reference values ​​to obtain the corresponding scaling factor; The blood sensitivity intensity corresponding to each pixel is obtained by multiplying the intensity value of each green channel by the corresponding scaling factor.

4. The method of claim 2, wherein, The step of channel replication for each of the blood sensitivity intensities includes: Determine the cutoff threshold for each of the aforementioned blood sensitivity intensities; Each of the blood sensitivity values ​​is compared with the cropping threshold to obtain a second comparison result, and the smaller value in each of the second comparison results is taken as the corresponding pixel intensity value; Divide each pixel intensity value by the cropping threshold to obtain the corresponding normalized blood sensitivity intensity, and perform channel copying on each normalized blood sensitivity intensity.

5. The method of claim 1, wherein, The step of identifying the endoscopic image using a preset neural network model to obtain the current blood concentration in the surgical area includes: The endoscopic image is identified by a preset neural network model to obtain the original scores of the blood concentration in the surgical area belonging to each preset concentration level. Perform probability transformation on each of the original scores to obtain the predicted probability distribution belonging to each of the preset concentration levels; Select the target preset concentration level corresponding to the highest predicted probability from each of the predicted probability distributions, and use the target preset concentration level as the current blood concentration in the surgical area.

6. The method of claim 1, wherein, Before the step of recognizing the endoscopic image using a preset neural network model, the method further includes: Each sample image and its corresponding preset concentration level are acquired, and each sample image is identified through an initial neural network model to obtain the sample prediction probability distribution of each sample image belonging to each preset concentration level. The model loss value is determined based on the preset loss function, the preset concentration level of each sample, and the predicted probability distribution of each sample. The parameters of the initial neural network model are adjusted based on the model loss value, and the process of recognizing each sample image using the initial neural network model is repeated until the preset training conditions are met, thereby obtaining the preset neural network model.

7. The method of claim 6, wherein, The step of determining the model loss value based on a preset loss function, preset concentration levels of each sample, and predicted probability distributions of each sample includes: Determine the occurrence ratio of each preset concentration level of the sample, and assign a corresponding category weight to each preset concentration level of the sample according to the occurrence ratio, wherein the occurrence ratio is negatively correlated with the category weight; The original loss value is determined based on the preset loss function, the preset concentration level of each sample, and the predicted probability distribution of each sample. The original loss values ​​are weighted and summed according to the corresponding category weights, and then averaged to obtain the model loss value.

8. The method of claim 6, wherein, After the step of acquiring each sample image and the corresponding preset concentration level of the sample, the method further includes: The preset concentration level of each sample image is taken as the true preset concentration level. a first value is set to a label value representing the real preset concentration level, and a second value is set to a label value representing each of the rest of the sample concentration levels except the real preset concentration level, the first value being 1 , and the second value being / n, is a preset smoothing coefficient, and n is the number of levels of the rest of the sample concentration levels except the real preset concentration level; Construct probability distribution labels for each of the sample images based on the first value and each of the second values; The step of determining the model loss value based on a preset loss function, preset concentration levels of each sample, and predicted probability distributions of each sample includes: The model loss value is determined based on the preset loss function, the probability distribution labels, and the predicted probability distribution of each sample.

9. The method of claim 6, wherein, The preset training conditions include: The overall accuracy of the initial neural network model reaches a preset accuracy threshold, and the overall accuracy is obtained by statistically analyzing the proportion of the number of correctly predicted sample images to the total number of sample images. And / or, the macro-average F1 score of the initial neural network model reaches a preset threshold, the macro-average F1 score is obtained by the arithmetic mean of the F1 scores of each preset concentration level of the samples, and the arithmetic mean of the F1 scores of each preset concentration level of the samples is obtained by the precision and recall corresponding to each preset concentration level of the samples.

10. The method of claim 1, wherein, The step of determining a corresponding first adjustment signal based on the current blood concentration and controlling the first flushing pump to flush the surgical area according to the first adjustment signal includes: The corresponding rotation speed increment is determined based on the current blood concentration, and the preset initial rotation speed is increased by the rotation speed increment. The current blood concentration is positively correlated with the rotation speed increment. The first adjustment signal is determined based on the preset initial rotation speed after the speed increase, and the first flushing pump is controlled to flush the surgical area according to the first adjustment signal.

11. The method of claim 10, wherein, The step of controlling the first irrigation pump to irrigate the surgical area according to the first adjustment signal includes: The target speed range of the preset initial speed after the acceleration is determined according to the first adjustment signal, and the current bleeding state is determined based on the target speed range; The target time duration is determined based on the current bleeding status, and the first flushing pump is controlled according to the first adjustment signal to continuously flush the surgical area for the target time duration.

12. The method according to any one of claims 1 to 11, characterized in that, The endoscope system further includes: a pressure acquisition component for acquiring the current cavity pressure within the surgical area; and a second flushing pump on the water supply line for controlling the cavity pressure within the surgical area. After the step of controlling the first irrigation pump to irrigate the surgical area according to the first adjustment signal, the method further includes: Determine the pressure difference between the current cavity pressure and the preset pressure threshold; Based on the pressure difference, a corresponding second adjustment signal is determined, and the second flushing pump is controlled according to the second adjustment signal to maintain the cavity pressure in the surgical area at the preset pressure threshold.

13. An endoscope system, characterized in that, The endoscope system includes: an endoscope main unit, an endoscope scope, and a water supply pipeline; The water supply pipeline is equipped with a first flushing pump, which is used to control the flushing speed of the saline solution in the water supply pipeline on the surgical area. The endoscope host includes: a memory, a processor, and an automatic flushing program based on a neural network model stored in the memory and executable on the processor. When the automatic flushing program based on the neural network model is executed by the processor, it implements the steps of the automatic flushing method based on a neural network model as described in any one of claims 1 to 12.

Images