Target phenomenon monitoring method, device, computer equipment and storage medium

Abstract

The present application relates to the field of medical technology, and in particular to a method for monitoring a target phenomenon, including: obtaining a current medical image captured by a medical mirror; obtaining an identification result corresponding to the target phenomenon and / or a contamination result of the medical mirror based on the current medical image; obtaining processing control information of the current medical image based on the identification result and / or the contamination result of the medical mirror; and processing the current medical image according to the processing control information to obtain monitoring information of the target phenomenon. The method for monitoring target phenomena during surgery of a surgical robot of the present application solves the technical problem in the prior art of requiring the addition of additional sensors, which affects the operating space of the doctor and the robotic arm. It effectively improves the safety and reliability of surgery, does not require additional sensors, reduces production, design, and use costs, and increases the operating space of the robot compared to traditional non-contact methods.

Description

Technical Field

[0001] The present application relates to the field of medical simulation control and graphics processing technology, and in particular to a target phenomenon monitoring method, apparatus, computer equipment and storage medium. Background Art

[0002] Minimally invasive surgery, primarily using endoscopes and various imaging technologies, allows surgeons to perform procedures without inflicting extensive wounds on patients. Due to its minimal trauma and rapid recovery, it has become a trend in surgical development. Intraoperative bleeding is a major complication of surgery. Excessive bleeding directly threatens the patient's life, increasing intraoperative danger and surgical risk. Therefore, there is an urgent need for technologies and equipment to monitor intraoperative bleeding in real time.

[0003] However, in existing robotic surgery systems, intraoperative bleeding monitoring relies on a bleeding detection unit analyzing pixel data from the captured scene to determine the presence of one or more initial blood sites. After detecting these initial blood sites, the initial site icon is displayed on the display unit to identify the area. Therefore, no tissue information regarding the area where blood has accumulated is available. Summary of the Invention

[0004] Based on this, it is necessary to provide a target phenomenon monitoring method, device, computer equipment and storage medium to address the above technical problems.

[0005] In a first aspect, the present application provides a target phenomenon monitoring method, the target phenomenon monitoring method comprising:

[0006] Acquire the current medical image captured by the medical mirror;

[0007] Obtaining, based on the current medical image, a recognition result corresponding to a target phenomenon and / or a contamination result of the medical mirror;

[0008] Obtaining processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror;

[0009] The current medical image is processed according to the processing control information to obtain monitoring information of the target phenomenon.

[0010] In the second aspect, the present application also provides a target phenomenon monitoring system, which is characterized in that it includes a medical mirror and an image processing device, the medical mirror communicates with the image processing device, the medical mirror is used to collect the current medical image; the image processing device is used to execute the above-mentioned target phenomenon monitoring method.

[0011] In a third aspect, the present application further provides a target phenomenon monitoring device, characterized in that the target phenomenon monitoring device comprises:

[0012] An acquisition unit, configured to acquire a current medical image captured by the medical mirror;

[0013] an analyzing unit, configured to obtain, based on the current medical image, an identification result corresponding to a target phenomenon and / or a contamination result of the medical mirror;

[0014] a processing control information generating unit, configured to obtain processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror;

[0015] A monitoring information generating unit is configured to process the current medical image according to the processing control information to obtain monitoring information of the target phenomenon.

[0016] In a fourth aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in any one of the above embodiments when executing the computer program.

[0017] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any one of the above embodiments.

[0018] In a sixth aspect, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the method in any one of the above embodiments.

[0019] The above-mentioned target phenomenon monitoring method, device, computer equipment and storage medium obtain the current medical image captured by the medical mirror; obtain the recognition result corresponding to the target phenomenon and / or the contamination result of the medical mirror based on the current medical image; obtain the processing control information of the current medical image based on the recognition result and / or the contamination result of the medical mirror; and process the current medical image according to the processing control information to obtain the monitoring information of the target phenomenon. The intraoperative bleeding monitoring method of the surgical robot of the present application solves the technical problem in the prior art that additional sensors need to be added, which affects the operating space of the doctor and the robotic arm. It effectively improves the safety and reliability of the operation, does not require additional sensors, reduces the production, design and use costs, and increases the operating space of the robot compared to the traditional non-contact method, and improves the efficiency of collaboration with the doctor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A diagram showing an application environment of a target phenomenon monitoring method in one embodiment;

[0022] Figure 2 Schematic diagram of a target phenomenon monitoring method according to an embodiment;

[0023] Figure 3 Schematic diagram of a flow chart of a method for generating pollution results of a target phenomenon monitoring method in one embodiment;

[0024] Figure 4 1. A schematic diagram of a process flow for identifying a target phenomenon monitoring method according to an embodiment;

[0025] Figure 5 A schematic diagram of generating processing control information for a target phenomenon monitoring method according to an embodiment;

[0026] Figure 6 A schematic diagram of a monitoring information flow of a target phenomenon of a target phenomenon monitoring method in one embodiment;

[0027] Figure 7 A schematic diagram of target phenomenon monitoring application of a target phenomenon monitoring method in one embodiment;

[0028] Figure 8 Schematic diagram of an image fusion process of a target phenomenon monitoring method in one embodiment;

[0029] Figure 9 A schematic diagram of an image fusion application of a target phenomenon monitoring method in one embodiment;

[0030] Figure 10 FIG. 1 is a schematic diagram A of the first image fusion of a target phenomenon monitoring method in one embodiment; FIG.

[0031] Figure 11 FIG. 1 is a schematic diagram B of the first image fusion of a target phenomenon monitoring method in one embodiment;

[0032] Figure 12 FIG3 is a schematic diagram C of the first image fusion of a target phenomenon monitoring method in one embodiment;

[0033] Figure 13 Schematic diagram A of image fusion in a target phenomenon monitoring method according to an embodiment, in which a bleeding monitoring display mode is not enabled;

[0034] Figure 14 Schematic diagram B of image fusion in a target phenomenon monitoring method according to an embodiment, in which the bleeding monitoring display mode is not enabled;

[0035] Figure 15 Schematic diagram C of image fusion in a target phenomenon monitoring method in an embodiment in which the bleeding monitoring display mode is not enabled;

[0036] Figure 16 FIG. 1 is a schematic diagram A showing bleeding of a target phenomenon monitoring method in one embodiment;

[0037] Figure 17 FIG. 1 is a schematic diagram B showing bleeding of a target phenomenon monitoring method in one embodiment;

[0038] Figure 18 FIG. C is a schematic diagram showing bleeding of a method for monitoring a target phenomenon in one embodiment;

[0039] Figure 19 A schematic diagram of image fusion and reconstruction of a target phenomenon monitoring method in one embodiment;

[0040] Figure 20 A schematic diagram of a feature point registration process of a target phenomenon monitoring method in one embodiment;

[0041] Figure 21 Schematic diagram of image matching of a target phenomenon monitoring method in one embodiment;

[0042] Figure 22 A schematic diagram of superimposed splicing of a target phenomenon monitoring method in one embodiment;

[0043] Figure 23 A schematic diagram of image superposition of a target phenomenon monitoring method in one embodiment;

[0044] Figure 24 A schematic diagram of a target phenomenon monitoring process of a target phenomenon monitoring method in one embodiment;

[0045] Figure 25 A schematic diagram of current medical image reconstruction of a target phenomenon monitoring method in one embodiment;

[0046] Figure 26 A schematic diagram of current medical image reconstruction of a target phenomenon monitoring method in another embodiment;

[0047] Figure 27 A data enhancement diagram of a target phenomenon monitoring method according to an embodiment;

[0048] Figure 28 A data enhancement diagram of a target phenomenon monitoring method according to an embodiment;

[0049] Figure 29 Schematic diagram of the exposure control process of a target phenomenon monitoring method in one embodiment;

[0050] Figure 30 Schematic diagram of an application of a target phenomenon monitoring method in one embodiment;

[0051] Figure 31 A schematic diagram of image acquisition and display hardware for a target phenomenon monitoring method according to an embodiment;

[0052] Figure 32 A control block diagram of a target phenomenon monitoring method in one embodiment;

[0053] Figure 33 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] The target phenomenon monitoring method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. The target phenomenon monitoring method can be applied to a surgical system. The following is an example of a surgical system. In the surgical system, the surgical system may include a doctor's console, a patient operating trolley, and an image platform. The doctor's console is a platform for the doctor to operate, and the entire surgical process is achieved through operations on the doctor's console. The patient operating trolley is used to place a bed. The image platform includes an endoscope, an image processor, and a central processing unit, wherein the endoscope communicates with the image processor, and the image processor communicates with the central processing unit. The central processing unit can also receive control instructions sent by the doctor's console, and process the image sent by the image processor and send it to the doctor's console for display. The surgeon can use the doctor's console to control the endoscope arm to drive the endoscope's movement and observe the scene inside the patient's body cavity based on the image displayed on the doctor's control platform. The endoscope simultaneously captures the scene observed by the left and right eyes of the body cavity and sends it to the image processor. The image processor processes the image and obtains the recognition result corresponding to the target phenomenon and / or the contamination result of the medical mirror. Based on the recognition result corresponding to the target phenomenon and / or the contamination result of the medical mirror, image fusion and other processing are performed, and the processed image is sent to the central processing unit. The central processing unit then performs video splicing and other processing on the fused image and sends it to the doctor's console for display. In one embodiment, the central processing unit and the image processor can use the same processor to reduce hardware costs. In another embodiment, the central processing unit and the image processor can be set independently to improve processing efficiency. For example, a separate image processor, such as a GPU, is used to perform image processing, and the central processing unit, such as a CPU, performs subsequent processing on the image processed by the image processor.

[0056] In one embodiment, Figure 2 As shown, a target phenomenon monitoring method is provided, which is applied to Figure 1 The image processor and the central processing unit shown in the figure are used as an example to illustrate the process, which includes the following steps:

[0057] S202: Acquire the current medical image captured by the medical mirror.

[0058] The medical scope is an endoscope used in endoscopic surgery. The current medical image is the current view inside the patient's body cavity during endoscopic surgery.

[0059] Specifically, the image processor obtains the current medical image captured by the medical mirror. Here, the image processor may capture the current scene inside the patient's body cavity through an endoscope.

[0060] S204: Obtaining a recognition result corresponding to the target phenomenon and / or a contamination result of the medical mirror according to the current medical image.

[0061] Specifically, the image processor acquires the current medical image, and based on the current medical image, the image processor further determines whether the target phenomenon exists and / or the image processor determines the contamination status of the medical mirror based on the current medical image. The contamination status here may be contamination of the medical mirror caused by bleeding inside the patient's body cavity.

[0062] The target phenomenon may refer to bleeding, that is, whether bleeding occurs inside the patient's body cavity, and contamination refers to the contamination of the medical mirror, for example, the medical mirror is contaminated due to bleeding.

[0063] S206: Obtaining processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror.

[0064] Specifically, the image processor first determines whether a target phenomenon exists and / or whether the medical mirror is contaminated. For example, bleeding within a patient's body cavity may cause the target phenomenon to occur and / or the patient's bleeding may contaminate the medical mirror. Based on the recognition result and / or the contamination result, the image processor then obtains processing control information for the current medical image to form a fused image, which may be a 3D image.

[0065] S208: Process the current medical image according to the processing control information to obtain monitoring information of the target phenomenon.

[0066] Specifically, the image processor processes the current medical image according to the obtained processing control information. For example, the image processor forms an image according to the obtained processing control information so as to present the target phenomenon in real time through the image. The image processor then monitors the target phenomenon through the image to obtain monitoring information of the target phenomenon.

[0067] In this embodiment, the image processor obtains a current medical image captured by a medical mirror; then determines whether a bleeding state exists by obtaining an identification result corresponding to a target phenomenon and / or a contamination result of the medical mirror based on the current medical image; obtains processing control information of the current medical image based on the identification result and / or the contamination result of the medical mirror, and forms a real-time image of the current scene inside the patient's body cavity; processes the current medical image based on the processing control information to obtain monitoring information of the target phenomenon, monitors and displays the current medical image based on the image of the processing control information, and obtains monitoring information.

[0068] like Figure 3 As shown, in one embodiment, the method for generating the contamination result of the medical mirror includes:

[0069] S302: Calculate the image information entropy of the current medical image.

[0070] Specifically, the image processor transmits the image data inside the patient's body cavity to the lens contamination monitoring module through the medical mirror. The lens contamination monitoring module of the image processor receives the current medical image and sends the current medical image to the information entropy extraction module. The information entropy extraction module extracts the information entropy of the image to calculate the information entropy of the current medical image.

[0071] S304: Compare the image information entropy with the information entropy threshold obtained in advance. When the image information entropy is less than or equal to the information entropy threshold obtained in advance, generate a contamination result indicating that the medical mirror is contaminated; when the image information entropy is greater than the information entropy threshold obtained in advance, generate a contamination result indicating that the medical mirror is not contaminated.

[0072] Specifically, the image processor compares the extracted information entropy with a pre-trained information entropy threshold of image data of a medical mirror to obtain a contamination result of the medical mirror, so as to obtain processing control information of the current medical image based on the contamination result.

[0073] In this embodiment, the image processor calculates the image information entropy of the current medical image, and then compares the obtained information entropy with the information entropy threshold obtained by pre-training, thereby obtaining the contamination result of the medical mirror, so that the contamination result of the medical mirror obtained after comparison is transmitted to the image fusion module of the image processor and the 3D reconstruction module of the image processor.

[0074] like Figure 4 As shown, in one embodiment, obtaining a recognition result corresponding to a target phenomenon based on a current medical image includes:

[0075] S402: Extracting objects from the current medical image.

[0076] Specifically, the image processor receives the current medical image and sends it to the candidate target extraction module of the image processor so that the candidate target extraction module can further determine the target. For example, the bleeding monitoring module of the image processor receives image data and sends it to the candidate target extraction module. The image data received by the bleeding monitoring module is sent by the endoscope control unit of the target phenomenon monitoring device.

[0077] S404: When the target is extracted, a judgment result of whether the target phenomenon exists is obtained; when the target is not extracted, a recognition result of whether the target phenomenon does not exist is obtained.

[0078] Specifically, when the image processor extracts a target from the current medical image, it screens the target and transmits the screened target to the image fusion module and the 3D reconstruction module. For example, the image processor's candidate target extraction module extracts image data representing the current scene inside a patient's body cavity, determines whether the image data contains bleeding, screens out the image data containing bleeding, and then fuses the screened image data through the fusion module to form an image. The 3D reconstruction module then reconstructs a 3D model.

[0079] In this embodiment, the image processor extracts targets from the acquired current medical image, and then further judges the acquired targets to screen out targets containing preset results. The obtained targets containing preset results can facilitate subsequent processing of the targets.

[0080] In one embodiment, before performing target extraction on the current medical image, the method further includes: adjusting the brightness and / or contrast of the current medical image.

[0081] Specifically, after receiving the current medical image, the image processor adjusts the brightness and / or contrast of the current medical image to facilitate subsequent assessment of the current medical image. For example, after receiving image data from the lens contamination monitoring module, the bleeding monitoring module of the image processor first sends the image data to the image processor's preprocessing module, which adjusts the brightness and contrast of the image data. The preprocessed image data is then sent to the candidate object extraction module.

[0082] In this embodiment, the pre-processing module of the image processor adjusts the brightness and / or contrast of the current medical image to improve the accuracy of the judgment of the current medical image in subsequent steps.

[0083] like Figure 5 As shown, in one embodiment, obtaining the processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror includes:

[0084] S502: When the recognition result is existence, processing control information for starting image fusion is generated.

[0085] Specifically, the image processor identifies the presence of the target phenomenon based on the current medical image and generates processing control information for the current medical image fusion. For example, the image processor's bleeding monitoring module monitors the left and right scenes for bleeding. Upon detecting bleeding, the image processor sends an image fusion start command to the image fusion module and the 3D reconstruction module.

[0086] S504: When the recognition result is existence and the contamination result of the medical mirror is contamination, processing control information for terminating image fusion is generated.

[0087] Specifically, if the image processor determines that the medical mirror is contaminated based on the acquired contamination status, it will generate processing control information to terminate the current medical image fusion. For example, the image processor's lens contamination monitoring module monitors the lens contamination of the received left and right scene images separately. If it detects that the lens contamination exceeds a preset threshold, it will send a command to terminate the current medical image fusion to the image fusion module and the 3D reconstruction module.

[0088] In this embodiment, the image processor generates processing control information to initiate image fusion based on the target phenomenon recognition result and / or generates processing control information to terminate image fusion based on the contamination result of the medical mirror. This allows the image processor to present the current medical image in real time. For example, the image currently reconstructed by the 3D reconstruction module can be mapped onto a screen to display tissue information of the blood clot area.

[0089] like Figure 6 and Figure 7 As shown, in one embodiment, processing the current medical image according to the processing control information to obtain monitoring information of the target phenomenon includes:

[0090] S602: When the processing control information is to perform fusion processing on the current medical image, obtain a display mode of the target phenomenon.

[0091] Specifically, when the image processor obtains the recognition result of the existence of the target phenomenon based on the previous medical image and / or the contamination result of the medical mirror, it fuses the current medical image and obtains a display mode for displaying the current fusion result. For example, the bleeding monitoring module of the image processor monitors the bleeding of the patient's affected area during surgery and / or the lens contamination detection module monitors the contamination of the endoscope lens. When bleeding is detected, the intraoperative image fusion function is started, and then the bleeding monitoring display mode function of the image processor is obtained to display the fusion result.

[0092] S604: Monitor and display the target phenomenon in the current medical image according to the display mode.

[0093] Specifically, the image processor monitors the target phenomenon according to the current medical image fusion status displayed by the display function of the display mode, so as to obtain the real-time status of the target phenomenon in real time. For example, the bleeding monitoring display mode function reconstructs the latest fusion result in 3D, forming a real-time 3D image mapped to the screen to display the tissue information of the patient's blood accumulation area. The doctor can monitor and display the tissue information of the blood accumulation area.

[0094] In this embodiment, when the image processor obtains the recognition result that the target phenomenon exists and / or the contamination result of the medical mirror based on the previous medical image, the current medical image is fused and the fusion result is displayed through the display mode so that the fusion result can be monitored in real time. For example, in an endoscopic surgery system, the current medical image of the patient during surgery is fused to display the tissue information of the blood accumulation area, and the fusion result is monitored in real time to solve the problem of the tissue information of the blood accumulation area being covered up when the patient is bleeding heavily.

[0095] like Figure 8 and Figure 9 As shown, in one embodiment, monitoring and displaying a target phenomenon in a current medical image according to a display mode includes:

[0096] S702: Extract feature points from the current frame image and the previous frame image of the first channel.

[0097] Specifically, the image processor obtains the current frame image and the previous frame image of the first channel, and extracts feature points from the two images so that the image processor can compare them based on the feature points. For example, the lens contamination monitoring module of the image processor obtains the current frame visible light image and the previous frame visible light image of the left scene, and extracts feature points from the two images. The lens contamination monitoring module of the image processor processes the current frame image and the previous frame image of the right scene in the same way as the left scene, which will not be repeated here. Figure 10 and Figure 11 , Figure 11 It can be the current frame image of the first channel, Figure 10 It can be obtained by the lens contamination monitoring module of the image processor for the previous frame of image. Figure 10 and Figure 11 , so that Figure 10 and Figure 11 Do subsequent processing.

[0098] S704: Registering feature points of the current frame image and the previous frame image.

[0099] Specifically, the image processor aligns the extracted feature points of the current frame image of the first channel and the previous frame image. For example, when the edge image of the current frame image of the first channel does not overlap with the edge image of the previous frame image, the two images are aligned based on the feature points of the two images so as to determine the positional relationship between the two images in subsequent operations.

[0100] S706: Superimposing the overlapping parts of the current frame image and the previous frame image according to the registration result, and superimposing and stitching the boundary areas of the non-overlapping parts to obtain a fused image.

[0101] Specifically, the image processor determines the positional relationship between the two images based on the alignment of the feature points of the current frame image of the first channel and the previous frame image, and superimposes the two images, that is, Figure A and Figure B can be superimposed so that the same image content in the subsequent processing of the two images can overlap.

[0102] Specifically, after the image processor obtains the superposition of the current frame image of the first channel and the previous frame image, there may be non-overlapping parts on the edges of the two images. The boundary areas of the non-overlapping parts of the two images are superimposed and spliced to fuse the two images to form a fused image. Figure 10 and Figure 11 Formed after fusion Figure 12 .

[0103] S710: Monitor and display the target phenomenon based on the fused image.

[0104] Specifically, after the image processor obtains the image fused by the current frame image of the first channel and the previous frame image, the target phenomenon is monitored and displayed according to the fused image. For example, the lens contamination monitoring module of the image processor obtains the fused image of the current frame visible light image of the left scene and the previous frame visible light image, and the fused image can be stored in the memory, that is, Figure 12 The fused image is stored in the memory so that it can be used to perform image registration with the current visible light image. Based on the fused image, the bleeding condition of the patient during surgery can be known, monitored, and displayed.

[0105] Specifically, the image processor does not enable the image fusion of the bleeding monitoring display mode. Figure 13 、 Figure 14 and Figure 15 The image processor starts the image fusion of the bleeding monitoring display mode. Figure 16 、 Figure 17 and Figure 18 .

[0106] In this embodiment, the image processor obtains the current frame image and the previous frame image of the first channel, extracts the feature points of the two images, and aligns the feature points. The two images are then superimposed and the boundary areas of the non-overlapping parts are spliced, so that the image processor can monitor and display the target phenomenon in real time.

[0107] like Figure 19 and Figure 20 As shown, in one embodiment, registering feature points of a current frame image and a previous frame image includes:

[0108] S802: Acquire a first preset number of first pixel points to be processed in the current frame image and a first preset number of second pixel points to be processed in the previous frame image.

[0109] like Figure 21 As shown, specifically, the target phenomenon monitoring device obtains a first preset number of first pixels to be processed in the current frame image of the first channel and a first preset number of second pixels to be processed in the previous frame image, so as to be used for subsequent registration of the current frame image and the previous frame image of the first channel. For example, Figure 10 , four pixel points preset in the current frame image of the first channel are exemplarily shown, the four pixel points include the first pixel point to be processed p1, where the coordinates of p1 are (x1, y), and four pixel points preset in the previous frame image, the four pixel points include the second pixel point to be processed q1, where the coordinates of q1 are (x1, y1).

[0110] S804: Taking one of the to-be-processed pixels as the origin, calculate a second preset number of pixels in another frame image that meet the requirements.

[0111] Specifically, the image processor may use the first pixel point to be processed as the origin to calculate a second preset number of pixel points of the current frame image of the first channel.

[0112] Specifically, the image processor first generates a template for each pixel, for example, a template consisting of 2×2 pixels near the first pixel to be processed p1 and the second pixel to be processed q1. Then, based on the template processing, a second preset number of pixels meeting the requirements are obtained in another frame of image.

[0113] For example, the image processor may use the first pixel point to be processed p1 as the origin to calculate the four pixel points with the smallest template pixel difference in the pixel point template in another frame of image.

[0114] S806: Screening pixels that meet the requirements to obtain target pixels.

[0115] For the convenience of explanation, Dmin is introduced, where Dmin is the sum of numerical differences of pixel values within the template, and the template may be composed of 2×2 pixels.

[0116] Specifically, the image processor obtains the average absolute difference that occurs most frequently among the average absolute differences of the second preset number of pixel points that meet the requirement, or obtains the average absolute difference that is closest to the average of the average absolute differences of the second preset number of pixel points. For example, the image processor obtains the four pixels with the smallest pixel difference in a 2×2 pixel template near the first pixel point to be processed p1 in the current frame image of the first channel, and obtains Dmin with the highest frequency among the four minimum average absolute differences, or obtains Dmin that is closest to the average of the four minimum average absolute differences Dmin.

[0117] S808: Taking the distance between the origin and the target pixel as the disparity.

[0118] Specifically, the image processor obtains the pixel points corresponding to the sum of the numerical differences, and uses the distance between their positions in the current frame image and the previous frame image as the disparity. For example, the image processor obtains the first pixel point to be processed, p1(x1, y1), calculates the four pixels in Figure A with the smallest template pixel difference, and takes the Dmin with the highest frequency among the four minimum mean absolute differences, or takes the Dmin closest to the average of the four minimum mean absolute differences Dmin. The distance between the corresponding pixel points in Figures A and B is the disparity.

[0119] S810: performing movement processing on the pixels to be processed of the current frame image and / or the previous frame image according to the disparity.

[0120] Specifically, after the image processor obtains the disparity formed by the distance between the positions of corresponding pixels in images A and B, it moves the pixels to be processed in the current frame image and / or the previous frame image. For example, after the image processor obtains the disparity, it moves the pixels in image B relative to image A based on the horizontal and vertical disparities.

[0121] S812: Superimposing the current frame image after the pixel point movement processing with the previous frame image to align feature points of the current frame image and the previous frame image.

[0122] Specifically, the image processor superimposes the current frame image after pixel point movement processing with the previous frame image, and the image processor aligns feature points of the current frame image and the previous frame image so that the current frame image and the previous frame image are fused.

[0123] In this embodiment, after the image processor obtains the current frame image and the previous frame image, it moves the pixels in one image so that the image is superimposed on the other image, thereby achieving alignment of the two images, so that the boundary area where the two images are superimposed is superimposed and spliced, and fused into a whole image formed by the current frame image of the left scene or the right scene and the previous frame image.

[0124] like Figure 22 As shown, in one embodiment, the current frame image and the boundary area of the non-overlapping portion of the previous frame image are superimposed and spliced to obtain a fused image, including:

[0125] S902: Obtain a first brightness weight of a current frame image and a second brightness weight of a previous frame image.

[0126] Specifically, the image processor obtains the first brightness weight of the current frame image and the second brightness weight of the previous frame image respectively, so as to process the subsequent current frame image and the previous frame image according to the brightness weight of the current frame image and the brightness weight of the previous frame image.

[0127] S904: Superimpose the brightness of the overlapping area between the current frame image and the previous frame image based on the first brightness weight and the second brightness weight and the pixel value difference. Superimpose the brightness of the overlapping area between the first image to be superimposed and the second image to be superimposed between the current frame image and the previous frame image based on the first brightness weight and the second brightness weight and the pixel value difference.

[0128] like Figure 23 As shown, specifically, the image processor performs superposition processing on the brightness of the overlapping area of the first image to be superimposed of the current frame image and the second image to be superimposed of the previous frame image based on the first brightness weight of the current frame image and the second brightness weight of the previous frame image. For example, according to the preset normal distribution curve schematic diagram, the image processor preferably has a mathematical expectation U=128 and a variance δ=50 of the normal distribution curve. The closer the brightness value is to the mathematical expectation, the greater its weight. Therefore, by using the above-mentioned normal distribution method to superimpose and splice the two images, a suitable superposition effect can be obtained. The calculation formula is as follows:

[0129] Non-overlapping regions of two images:

[0130] result = p1 or p2

[0131] The image superposition method 1 of two images is calculated using the following formula:

[0132]

[0133] Among them, P1 is the first brightness; P2 is the second brightness; W1 is the weight of the first brightness P1; W2 is the weight of the second brightness P2; △ is the difference threshold of pixel values, obtained through debugging, and d1 and d2 are the pixel values of the pixels in the two images.

[0134] Image overlay method 2 is calculated using the following formula:

[0135] result=(p1*w1+p2*w2) / (w1+w2)

[0136] In this embodiment, the image processor obtains the brightness weights of the current frame image and the previous frame image respectively, and superimposes the brightness of the overlapping area of the first image to be superimposed and the second image to be superimposed according to the brightness weights of the two images, so as to adjust the difference in pixel values of the brightness of the non-overlapping area of the first image to be superimposed and the second image to be superimposed, thereby completing the fusion of the two images.

[0137] like Figure 24 As shown, in one embodiment, monitoring and displaying a target phenomenon in a current medical image according to a display mode includes:

[0138] S1002: When the display mode is normal display, the current medical image and the previous frame image are superimposed and stored.

[0139] Specifically, when the display mode obtained by the image processor is normal display, the current medical image is superimposed. The current medical image may include the current medical image and a visible light image that can be read from the memory. The superimposed image is stored and displayed. For example, when the image processor receives a normal display command, the feature points of the current frame visible light image of the left scene image and the visible light image read from the memory are first extracted, and then the feature points of the two images are aligned. Based on the alignment results, the two images are subjected to image superposition method 1, and the boundary areas of the non-overlapping parts of the two images are superimposed and spliced. Finally, the fused image is stored in the memory, and this cycle is repeated. The right scene image is processed in the same way as the left scene image and will not be repeated here.

[0140] Furthermore, when image overlay mode 1 is used without the bleeding monitoring display mode enabled, the result is not directly applied to the display. With this method, the information about the blood accumulation area will not be weakened as the number of overlays increases.

[0141] S1004: When the display mode is bleeding display, the current medical image is reconstructed and the bleeding monitoring mode is displayed.

[0142] Specifically, when the display mode obtained by the image processor is bleeding display, the current medical image is reconstructed. The current medical image may include the current medical image and the visible light image that can be read from the memory, and the bleeding monitoring mode is displayed. When the image processor receives the command to turn on the bleeding monitoring display, the feature points of the current frame visible light image of the left scene image and the visible light image read from the memory are first extracted, and then the feature points of the two images are aligned. Based on the alignment result, the two images are subjected to image superposition method 2. The latest superposition effect of image superposition method 2 is not stored in the memory. That is, the fused image formed by the current frame visible light image and the visible light image read from the memory will not be stored in the memory, but will form the image data in the 3D reconstruction. Finally, the image after superimposing the left and right scene images is 3D reconstructed.

[0143] Furthermore, when the image overlay mode 2 is applied in the bleeding monitoring display mode, the result will not be stored in the memory, but will be directly applied to the display, and the user can observe the range of blood accumulation and tissue information at the same time.

[0144] In this embodiment, the image processor performs different processing on the fused image according to different commands received. For example, when the image processor receives a normal display command, the fused image is stored in the memory, and this cycle is repeated. When the image processor receives a bleeding monitoring display command, the latest superimposed fused image is not stored in the memory, but is used as image data for 3D reconstruction, and the image is superimposed with the left and right scene images for 3D reconstruction.

[0145] like Figure 25 and Figure 26 As shown, in one embodiment, when the display mode is bleeding display, the current medical image is reconstructed and the bleeding monitoring mode is displayed, including: when the display mode is bleeding display, the current medical image and the previous frame image captured by each acquisition device are superimposed, and the images after superposition of each acquisition device are superimposed.

[0146] Specifically, the image processor acquires the relative motion images of the two current medical images and then reconstructs a 3D model of the current medical image based on the motion images of the two current medical images. For example, in Figure A, the focal plane of the video captured by the two parallel sensors is equivalent to infinity, and stereoscopic images cannot be obtained without parallax processing. For this reason, the image processor needs to perform relative motion on the acquired two current medical images to achieve 3D reconstruction. As shown in Figure B, the left eye scene is captured on the right image, and the right eye scene is captured on the left image. This ensures that the left and right images have positive parallax, ultimately achieving better 3D reconstruction results.

[0147] In this embodiment, the image processor obtains two current medical images, that is, obtains relative motion images of the two current medical images, and then performs 3D reconstruction based on the motion images of the two current medical images, so as to obtain real-time 3D reconstruction results.

[0148] like Figure 27 As shown, in one embodiment, obtaining a current medical image captured by a medical mirror includes:

[0149] S1102: Acquire the reflected light of the tissue collected by the image sensor of the medical mirror.

[0150] Specifically, the image processor acquires reflected light through the image sensor of a medical scope. For example, the image processor acquires reflected light through the image sensor of an endoscope and the endoscope's driver module. The image sensor consists of two sensors, one located at the front end of the endoscope, which are used to receive self-organized reflected light from the blood pool. The endoscope's driver module is responsible for initializing the parameters of the two sensors and configuring the parameters transmitted from the endoscope's back-end control unit in real time.

[0151] MIPI, short for Mobile Industry Processor Interface, is an open standard developed by the MIPI Alliance for mobile application processors. LVDS, short for Low-Voltage Differential Signaling, enables signal transmission at hundreds of Mbps over differential PCB wire pairs or balanced cables. Its low voltage amplitude and low current drive output achieve low noise and low power consumption.

[0152] It should be noted that the image sensor outputs captured images using the MIPI or LVDS protocol. After conversion by the bridge chip, the images are transmitted over long distances as optical signals to the image processor. The driver module communicates with the image processor via an RS485 or RS232 communication interface and sends the acquired information to the image sensor via the IIC protocol, controlling the output format, mode, and frame rate. The driver module collects key signals and communicates with the image processor via RS485.

[0153] The image processor receives data from the endoscope through the bridge chip and sends it to the image analysis module for analysis into fixed and appropriate image data. The image processing module and the endoscope control module are sent to the image processing module and the endoscope control module respectively. The endoscope control module analyzes and processes the image, sends the control information to the endoscope via RS485, and receives the key signals via RS485.

[0154] The endoscope control unit receives the left and right scenes transmitted by the endoscope driving module, processes the scene images respectively, calculates appropriate feedback values, and feeds back to the endoscope driving module.

[0155] S1104: Obtain a current medical image based on the reflected light.

[0156] Specifically, the image processor can obtain a current medical image based on the acquired reflected light. For example, the image processor can obtain a current scene inside the patient's body cavity based on the acquired reflected light, so as to perform operations such as fusion on the current scene in subsequent operations.

[0157] In this embodiment, the image processor obtains the reflected light of the tissue collected by the image sensor of the medical mirror, and then obtains the current medical image based on the reflected light, which can facilitate subsequent operations such as fusion and 3D reconstruction based on the current medical scene.

[0158] like Figure 5 As shown, in one embodiment, the target phenomenon monitoring method further includes: performing exposure control on the current medical image according to the current brightness.

[0159] Specifically, the image processor obtains the brightness of the current medical image, and performs exposure control on the brightness of the current medical image according to a preset target brightness range, so that the current medical image is within the preset target brightness range.

[0160] In this embodiment, after the image processor obtains the brightness of the current medical image, it determines whether the brightness of the current medical image is within a preset brightness range. If it is not within the preset brightness range, the exposure of the current medical image is controlled so that the brightness of the current medical image is within the preset brightness range.

[0161] like Figure 28 As shown, in one embodiment, performing exposure control on a current medical image according to current brightness includes:

[0162] S1202: When the current brightness of the current medical image is lower than the lower limit of the target brightness range, the image brightness is adjusted by increasing the current medical image sensor exposure.

[0163] Specifically, the image processor obtains the brightness of the current medical image. If the brightness is lower than the lower limit of the target brightness range, the image brightness can be adjusted by increasing the exposure of the current medical image sensor. For example, the target brightness range value is (B min , B max ), the image processor obtains the brightness of the current medical image of the left scene image as B. When B is lower than the lower limit value Bmin of the target brightness range, the image brightness is adjusted by increasing the exposure of the visible light image sensor. Priority is given to increasing the exposure time.

[0164] S1204: When the current brightness of the current medical image is higher than the upper limit of the target brightness range, the image brightness is adjusted by reducing the exposure of the current medical image sensor.

[0165] Specifically, the image processor obtains the brightness of the current medical image. If the brightness exceeds the upper limit of the target brightness range, the image brightness is adjusted by reducing the exposure of the current medical image sensor. For example, if the image processor obtains the brightness of the current medical image of the left scene as B, and B exceeds the upper limit of the target brightness range, Bmax, the image brightness is adjusted by reducing the exposure of the visible light image sensor. Prioritizing increasing the exposure time, the right scene image is adjusted in a similar manner, which will not be further described.

[0166] In this embodiment, the image processor obtains the brightness of the current medical image and determines whether the brightness of the current medical image is within a preset target brightness range value. If it is not within the range value, the brightness of the current medical image is adjusted so that the brightness of the current medical image is within the target brightness range, which facilitates subsequent image processing and viewing of the current medical image.

[0167] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0168] Based on the same inventive concept, the present application also provides a target phenomenon monitoring system for implementing the target phenomenon monitoring method described above. The implementation solution provided by this system is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more target phenomenon monitoring system embodiments provided below can be found in the above-mentioned limitations of the target phenomenon monitoring method, and will not be repeated here.

[0169] like Figure 29 、 Figure 30 、 Figure 31 and Figure 32As shown, in one embodiment, a target phenomenon monitoring system includes a medical mirror and an image processing device, the medical mirror communicates with the image processing device, and the medical mirror is used to collect the current medical image; the image processing device is used to execute the target phenomenon monitoring method in any one of the above embodiments.

[0170] Specifically, the user controls the power and color temperature of the light source output via the image processing device control module based on user interface control information. The light source then illuminates the medical mirror via a light guide. Based on the user interface control information, the user transmits this control information to the medical mirror control unit via the image processing device control module to control the mode, format, and frame rate of the image output by the image sensor. The medical mirror control unit controls the mode, format, and frame rate of the image output by the sensor based on the control information transmitted from the image processing device control module. It controls the sensor's photosensitivity based on the information contained in the received image data. The image data is then transmitted to the lens contamination monitoring module and the bleeding monitoring module. The lens contamination monitoring module monitors the left and right scenes for lens contamination. If severe lens contamination is detected, it sends an image fusion termination command to the image fusion and 3D reconstruction module.

[0171] Specifically, the bleeding monitoring module monitors the received left and right scenes for bleeding. Once bleeding is detected, it sends an image fusion start command to the image fusion and 3D reconstruction module. The received left and right scene image data are then sent to the image fusion and 3D reconstruction module. Based on control information from the lens contamination monitoring module and the bleeding monitoring module, the image fusion and 3D reconstruction module controls the left and right scene image fusion modules, respectively. The fusion results of the left and right scenes are then fed into the 3D reconstruction process for reconstructing the current medical image. The user can also select the display mode, namely, target phenomenon monitoring mode or normal display mode, using the mode selection button on the display.

[0172] Based on the same inventive concept, the present application also provides a target phenomenon monitoring device for implementing the target phenomenon monitoring method described above. The solution to the problem provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more target phenomenon monitoring device embodiments provided below can be found in the above-mentioned limitations of the target phenomenon monitoring method and will not be repeated here.

[0173] In one embodiment, a bleeding monitoring device is provided, comprising: an acquisition module, an analysis unit, a processing control information generation unit, and a monitoring information generation unit, wherein:

[0174] An acquisition unit, configured to acquire a current medical image captured by the medical mirror;

[0175] An analysis unit, configured to obtain, based on the current medical image, an identification result corresponding to a target phenomenon and / or a contamination result of the medical mirror;

[0176] a processing control information generating unit, configured to obtain processing control information of the current medical image based on the recognition result and / or the contamination result of the medical mirror;

[0177] The monitoring information generating unit is used to process the current medical image according to the processing control information to obtain monitoring information of the target phenomenon.

[0178] In one embodiment, the method for generating the contamination result of the medical mirror includes:

[0179] An analysis unit, configured to calculate image information entropy of a current medical image;

[0180] The comparison unit is used to compare the image information entropy with the information entropy threshold obtained in advance. When the image information entropy is less than or equal to the information entropy threshold obtained in advance, a contamination result is generated indicating that the medical mirror is contaminated; when the image information entropy is greater than the information entropy threshold obtained in advance, a contamination result is generated indicating that the medical mirror is not contaminated.

[0181] In one embodiment, obtaining a recognition result corresponding to a target phenomenon based on a current medical image includes:

[0182] An object extraction unit, configured to extract an object from a current medical image;

[0183] The judgment unit is used to obtain a judgment result of whether the target phenomenon exists when the target is extracted; and obtain a recognition result that the target phenomenon does not exist when the target is not extracted.

[0184] In one embodiment, before performing target extraction on the current medical image, the method further includes:

[0185] The adjustment unit is used to adjust the brightness and / or contrast of the current medical image.

[0186] In one embodiment, obtaining processing control information of the current medical image based on the recognition result and / or the contamination result of the medical mirror includes:

[0187] a fusion processing control information generating unit, configured to generate processing control information for starting image fusion when the recognition result is yes;

[0188] The fusion termination processing control information generating unit is used to generate processing control information for terminating image fusion when the recognition result is existence and the contamination result of the medical mirror is contamination; when the recognition result is non-existence, output a movement instruction for the medical mirror.

[0189] In one embodiment, processing the current medical image according to the processing control information to obtain monitoring information of the target phenomenon includes:

[0190] A display mode acquisition unit, configured to acquire a display mode of a target phenomenon when the processing control information is to perform fusion processing on the current medical image;

[0191] The monitoring and display unit is used to monitor and display the target phenomenon of the current medical image according to the display mode.

[0192] In one embodiment, monitoring and displaying a target phenomenon in a current medical image according to a display mode includes:

[0193] An extraction unit, configured to extract feature points from a current frame image and a previous frame image of a first channel;

[0194] A registration unit, used to register feature points of the current frame image with those of the previous frame image;

[0195] The superposition unit is used to superimpose the overlapping parts of the current frame image and the previous frame image according to the registration result, and to superimpose and stitch the boundary areas of the non-overlapping parts to obtain a fused image;

[0196] The monitoring and display unit is used to monitor and display the target phenomenon based on the fused image.

[0197] In one embodiment, registering feature points of a current frame image with feature points of a previous frame image includes:

[0198] an acquisition unit, configured to acquire a first preset number of first pixels to be processed in a current frame image and a first preset number of second pixels to be processed in a previous frame image;

[0199] An analysis unit, configured to calculate a second preset number of pixels meeting the requirements in another frame of image, taking one of the pixels to be processed as an origin;

[0200] An acquisition unit, used to screen pixels that meet the requirements to obtain target pixels;

[0201] a disparity generation unit, configured to use the distance between the origin and the target pixel as the disparity;

[0202] A motion processing unit, configured to perform motion processing on the pixels to be processed of the current frame image and / or the previous frame image according to the parallax;

[0203] The registration unit is used to superimpose the current frame image after the pixel point movement processing with the previous frame image to align the feature points of the current frame image with the previous frame image.

[0204] In one embodiment, superimposing and splicing the boundary area of the non-overlapping portion of the current frame image and the previous frame image to obtain a fused image includes:

[0205] An acquiring unit, configured to acquire a first brightness weight of a current frame image and a second brightness weight of a previous frame image;

[0206] an overlay processing unit, configured to perform overlay processing on the brightness of an overlapping area of a current frame image and a previous frame image according to the first brightness weight, the second brightness weight, and the pixel value difference;

[0207] The adjustment unit is used to keep the brightness of the non-overlapping area of the current frame image and the previous frame image unchanged.

[0208] In one embodiment, monitoring and displaying a target phenomenon in a current medical image according to a display mode includes:

[0209] A normal display unit, configured to superimpose the current medical image and the previous frame image and store them when the display mode is normal display;

[0210] The bleeding display unit is used to reconstruct and display the current medical image and the previous frame image after superimposing them when the display mode is bleeding display.

[0211] In one embodiment, when the display mode is bleeding display, the current medical image is reconstructed and the bleeding monitoring and display modes are displayed, including:

[0212] The reconstruction unit is used to perform relative movement on the two-channel current medical images to achieve current medical image reconstruction.

[0213] In one embodiment, obtaining a current medical image captured by a medical mirror includes:

[0214] an acquisition unit, configured to acquire reflected light from tissue captured by an image sensor of the medical mirror;

[0215] The current medical image generating unit is configured to obtain a current medical image according to the reflected light.

[0216] In one embodiment, the target phenomenon monitoring method further includes:

[0217] The exposure control unit is used to control the exposure of the current medical image according to the current brightness.

[0218] In one embodiment, performing exposure control on a current medical image according to current brightness includes:

[0219] a first adjusting unit, configured to adjust the image brightness by increasing the current medical image sensor exposure when the current brightness of the current medical image is lower than a lower limit of the target brightness range;

[0220] The second adjustment unit is configured to adjust the image brightness by reducing the exposure of the current medical image sensor when the current brightness of the current medical image is higher than the upper limit of the target brightness range.

[0221] Each module in the target phenomenon monitoring device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0222] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 33 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store periodic task allocation data, such as configuration files, theoretical operating parameters and theoretical deviation value ranges, task attribute information, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a bleeding monitoring method is implemented. Those skilled in the art will understand that Figure 33 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. Specifically, the computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0223] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0224] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0225] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0226] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0227] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0228] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A target phenomenon monitoring method, characterized in that: The target phenomenon monitoring method includes: Acquire the current medical image captured by the medical mirror; Obtaining, based on the current medical image, a recognition result corresponding to a target phenomenon and / or a contamination result of the medical mirror; Obtaining processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror; processing the current medical image according to the processing control information to obtain monitoring information of the target phenomenon; The obtaining of the processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror includes: When the recognition result is that the image fusion is present, processing control information for starting the image fusion is generated; When the recognition result is existence and the contamination result of the medical mirror is contamination, generating processing control information for terminating image fusion; When the recognition result is no, a movement instruction of the medical mirror is output.

2. The target phenomenon monitoring method according to claim 1, characterized in that: The method for generating the contamination result of the medical mirror includes: Calculating the image information entropy of the current medical image; Comparing the image information entropy with a pre-trained information entropy threshold; When the image information entropy is less than or equal to the information entropy threshold obtained by pre-training, a contamination result is generated indicating that the medical mirror is contaminated; When the image information entropy is greater than the information entropy threshold obtained by pre-training, a contamination result is generated indicating that the medical mirror is not contaminated.

3. The target phenomenon monitoring method according to claim 1, characterized in that: Obtaining a recognition result corresponding to a target phenomenon according to the current medical image includes: performing target extraction on the current medical image; When the target is extracted, a recognition result indicating the existence of the target phenomenon is obtained; When the target is not extracted, a recognition result indicating that the target phenomenon does not exist is obtained.

4. The target phenomenon monitoring method according to claim 3, characterized in that: Before performing target extraction on the current medical image, the method further includes: The brightness and / or contrast of the current medical image is adjusted.

5. The target phenomenon monitoring method according to claim 1, characterized in that: The processing of the current medical image according to the processing control information to obtain the monitoring information of the target phenomenon includes: When the processing control information is to perform fusion processing on the current medical image, obtaining a display mode of the target phenomenon; The target phenomenon is monitored and displayed on the current medical image according to the display mode.

6. The target phenomenon monitoring method according to claim 5, characterized in that: The monitoring and displaying of the target phenomenon on the current medical image according to the display mode includes: Extract feature points from the current frame image and the previous frame image of the first channel; Registering feature points of the current frame image and the previous frame image; Superimposing the overlapping parts of the current frame image and the previous frame image according to the registration result, and superimposing and splicing the boundary areas of the non-overlapping parts to obtain a fused image; The fused image is used to monitor and display the target phenomenon according to the display mode.

7. The target phenomenon monitoring method according to claim 6, characterized in that: The registering the feature points of the current frame image and the previous frame image includes: Acquire a first preset number of first pixels to be processed in the current frame image and a first preset number of second pixels to be processed in the previous frame image; Taking one of the pixels to be processed as an origin, calculating a second preset number of pixels in another frame image that meet the requirements, wherein the second preset number of pixels in another frame image that meet the requirements include a second preset number of pixels with the smallest template pixel difference in a pixel template in the other frame image; Screening the pixels that meet the requirements to obtain target pixels; Taking the distance between the origin and the target pixel as the disparity; performing movement processing on the to-be-processed pixel points of the current frame image and / or the previous frame image according to the disparity; The current frame image after the pixel point movement processing is superimposed on the previous frame image to align the feature points of the current frame image and the previous frame image.

8. The target phenomenon monitoring method according to claim 7, characterized in that: The method of superimposing the overlapping portion of the current frame image and the previous frame image according to the registration result and superimposing and stitching the boundary area of the non-overlapping portion to obtain a fused image includes: Obtaining a first brightness weight of the current frame image and a second brightness weight of the previous frame image; and performing a superposition process on the brightness of an overlapping area of the current frame image and the previous frame image based on the first brightness weight, the second brightness weight, and the pixel value difference; The brightness of the non-overlapping area of the current frame image and the previous frame image is kept unchanged.

9. The target phenomenon monitoring method according to claim 6, characterized in that: The monitoring and displaying of the target phenomenon on the current medical image according to the display mode includes: When the display mode is normal display, the current medical image and the previous frame image are superimposed and stored; When the display mode is bleeding display, the current medical image and the previous frame image are superimposed, reconstructed, and displayed.

10. The target phenomenon monitoring method according to claim 9, characterized in that: When the display mode is bleeding display, the current medical image and the previous frame image are superimposed and then reconstructed, including: When the display mode is bleeding display, the current medical image acquired by each acquisition device and the previous frame image are superimposed, and the superimposed images of each acquisition device are reconstructed.

11. The target phenomenon monitoring method according to claim 1, characterized in that: The obtaining of the current medical image captured by the medical mirror includes: Acquiring reflected light from tissue collected by an image sensor of the medical mirror; A current medical image is obtained according to the reflected light.

12. The target phenomenon monitoring method according to claim 1, characterized in that: The method further comprises: Exposure control is performed on the current medical image according to the current brightness.

13. The target phenomenon monitoring method according to claim 12, characterized in that: The performing exposure control on the current medical image according to the current brightness includes: When the current brightness of the current medical image is lower than a lower limit of a target brightness range, adjusting the image brightness by increasing the current medical image sensor exposure; When the current brightness of the current medical image is higher than the upper limit of the target brightness range, the image brightness is adjusted by reducing the exposure of the current medical image sensor.

14. A target phenomenon monitoring system, characterized in that: It comprises a medical mirror and an image processing device, wherein the medical mirror communicates with the image processing device and is used to collect current medical images; the image processing device is used to execute the target phenomenon monitoring method described in any one of claims 1 to 13.

15. A target phenomenon monitoring device, characterized in that: The target phenomenon monitoring device comprises: An acquisition unit, configured to acquire a current medical image captured by the medical mirror; an analyzing unit, configured to obtain, based on the current medical image, an identification result corresponding to a target phenomenon and / or a contamination result of the medical mirror; a processing control information generating unit, configured to obtain processing control information of the current medical image according to the recognition result and / or the contamination result of the medical mirror; a monitoring information generating unit, configured to process the current medical image according to the processing control information to obtain monitoring information of the target phenomenon; The processing control information generating unit includes: a fusion processing control information generating unit, configured to generate processing control information for starting image fusion when the recognition result is yes; The fusion termination processing control information generating unit is used to generate processing control information for terminating image fusion when the recognition result is existence and the contamination result of the medical mirror is contamination; when the recognition result is non-existence, output a movement instruction of the medical mirror.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images