A real-time laparoscopic identification method for ovarian endometriomas

Abstract

The present invention relates to the technical field of laparoscopic image annotation, and specifically to a laparoscopic real-time identification method for ovarian endometriosis cysts. The present invention first analyzes the possibility of the existence of false boundaries in the suspected cyst area divided by a preset segmentation model in combination with the position and structural characteristics of false boundaries, and determines the boundary correction coefficient of each suspected cyst area in each frame of the image; further, each suspected cyst area in each frame of the image is divided into several local areas, and a detailed analysis of the suspected cyst area is performed in combination with instrument traction and respiratory effects, and the cyst confidence of each local area is evaluated as a microcyst. Finally, the segmentation parameters of the segmentation model are comprehensively adjusted to accurately segment and annotate the real cyst area, thereby improving the recognition effect of ovarian endometriosis cyst lesions.

Description

Technical Field

[0001] The present invention relates to the technical field of laparoscopic image annotation, and in particular to a laparoscopic real-time recognition method for ovarian endometriosis cysts. Background Art

[0002] Ovarian endometriosis refers to the ectopic growth of endometrial tissue outside the uterine cavity, most commonly in the ovaries, resulting in the formation of ovarian endometriomas. Laparoscopy allows medical personnel to directly observe ectopic lesions in the ovaries and other pelvic areas. Therefore, based on the visual images during the laparoscopic procedure, real-time abdominal images of each observation site, such as the rectouterine pouch, ovarian fossa, and broad ligament, can be collected. These images are then analyzed for features to identify and label ovarian endometriomas.

[0003] However, during laparoscopic operation, instrument traction, changes in pneumoperitoneum pressure, or cyst fluid leakage can cause displacement or morphological changes in the ovaries and cysts. At the same time, the complex adhesions and blurred anatomical hierarchical structure in the abdominal cavity may make it difficult to accurately identify some tiny lesions in laparoscopic images, and the normal tissues around the cyst lesions may be misidentified as cyst lesions, that is, there is the possibility of false boundaries, which leads to poor identification of ovarian endometriosis cyst lesions. Summary of the Invention

[0004] In order to solve the technical problem of poor recognition effect of ovarian endometriosis cyst lesions, the purpose of the present invention is to provide a laparoscopic real-time recognition method for ovarian endometriosis cysts. The technical solution adopted is as follows:

[0005] For each observed part of the patient, a laparoscopic image sequence is acquired, along with the respiratory signal, and the suspected cyst region in each frame of the sequence is segmented based on a preset segmentation model. All edges within each suspected cyst region are also obtained.

[0006] In each frame of the image, the boundary correction coefficient of each suspected cyst region is obtained based on the position distribution of each edge in each suspected cyst region and its position change in the laparoscopic image sequence; each suspected cyst region in each frame of the image is divided into several local regions, and the motion feature sequence of each local region is obtained based on the change characteristics of each local region between adjacent frames of the image;

[0007] A respiratory feature sequence is obtained based on the fluctuation of the respiratory signal, and the cyst confidence of each local area is obtained based on the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change features of each local area and the rest of the local areas in the laparoscopic image sequence. In each frame of the image, the segmentation parameters of the preset segmentation model are adjusted according to the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs, and the cyst area is re-labeled.

[0008] Furthermore, the method for obtaining the boundary correction coefficient includes:

[0009] The boundary coefficient of each edge is obtained based on the distance between each pixel on each edge and the center of the suspected cyst area to which it belongs. The edges in the abdominal image sequence are matched based on the position of the geometric center of each edge, and the structural stability coefficient of each edge is obtained based on the position difference between the geometric centers of the matched edges between all adjacent frames.

[0010] The boundary coefficient and the structural stability coefficient are integrated to obtain the boundary attribute of each edge; the boundary attributes of all edges in the cyst area are combined to obtain the boundary correction coefficient of each suspected cyst area.

[0011] Furthermore, the method for obtaining the boundary attribute includes:

[0012] The product of the boundary coefficient and the structural stability coefficient is used as the boundary attribute.

[0013] Furthermore, the method for acquiring the motion feature sequence includes:

[0014] Between each frame and the previous frame, the motion vector of each pixel in each local area is obtained based on the optical flow method, and the relative displacement between the matching corner points in the local areas at the same position between adjacent frames is obtained;

[0015] Based on the motion vectors of all pixels in each local area and the relative displacement between all matching corner points, the motion feature coefficients of each local area between adjacent frames are obtained; in the laparoscopic image sequence, the motion feature coefficients of the local areas at the same position between adjacent frames are sorted in chronological order to construct a motion feature sequence.

[0016] Furthermore, the method for acquiring the respiratory feature sequence includes:

[0017] Based on a preset phase segmentation algorithm, the respiratory phase of each signal point in the respiratory signal is marked, and the signal points under the same respiratory phase are sorted in time sequence to construct a respiratory phase signal under the corresponding respiratory phase;

[0018] In each respiratory phase signal, the respiratory characteristic coefficient at each moment is obtained according to the signal fluctuation intensity in the preset historical time domain of the signal point at each moment; all the respiratory characteristic coefficients are sorted in time sequence to construct a respiratory characteristic sequence.

[0019] Furthermore, the method for obtaining the respiratory characteristic coefficient includes:

[0020] In each respiratory phase signal, within the preset historical time domain of the signal point at each moment, the variance of the signal point is used as the first fluctuation parameter, and the second fluctuation parameter is obtained by combining the negative correlation mapping results of the occurrence frequency of signal points of each amplitude; the first fluctuation parameter and the second fluctuation parameter are fused to obtain the respiratory characteristic coefficient.

[0021] Furthermore, the method for obtaining the cyst confidence includes:

[0022] In a laparoscopic image sequence, any local area is taken as the target area, and all local areas adjacent to the target area are taken as reference areas; the deformation coefficient of the target area is obtained based on the sequence difference between the motion feature sequence of the target area and the motion feature sequence of each reference area; the absolute value of the Pearson correlation coefficient between the respiratory feature sequence and the motion feature sequence is taken and negative correlation mapping is performed to obtain the respiratory weight; the deformation coefficient is weighted using the respiratory weight, and the weighted result is used as the cyst confidence of the target area.

[0023] Furthermore, the method for re-marking the cyst area includes:

[0024] In each frame of the image, the segmentation adjustment weight of each local area is obtained according to the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs; the initial segmentation threshold when the preset segmentation model segments the suspected cyst area is obtained; the initial segmentation threshold is weighted by the segmentation adjustment weight, and the weighted result is used as the segmentation threshold of the corresponding local area, and segmentation is performed to obtain the cyst sub-area in the local area; the cyst sub-areas in all local areas in the suspected tumor area are integrated to obtain the cyst area in each frame of the image and mark it.

[0025] Furthermore, the method for obtaining the segmentation adjustment weight includes:

[0026] In each frame of the image, the product of the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs is used as the adjustment amplitude index, and the sum of the constant 1 and the adjustment amplitude index is used as the segmentation adjustment weight.

[0027] Furthermore, the preset segmentation model is a U-Net model.

[0028] The present invention has the following beneficial effects:

[0029] For each observed part of the patient, the present invention first obtains a laparoscopic image sequence and synchronously obtains a respiratory signal, segments the suspected cyst area in each frame of the sequence based on a preset segmentation model, and obtains all edges in each suspected cyst area to prepare for evaluating the existence of false boundaries in the suspected cyst area and whether boundary correction is needed; then, in each frame of the image, based on the position distribution of each edge in each suspected cyst area and its position change in the laparoscopic image sequence, evaluates the possibility of the existence of false boundaries and obtains the boundary correction coefficient of each suspected cyst area; further, each suspected cyst area in each frame of the image is divided into several local areas for refined analysis, and based on each suspected cyst area, the boundary correction coefficient is obtained. The change characteristics of each local area between adjacent frames of images are used to obtain the motion feature sequence of each local area, in preparation for the subsequent evaluation of the cyst confidence of the local area in combination with instrument traction and respiratory influence; then the respiratory feature sequence is obtained according to the fluctuation of the respiratory signal, and the cyst confidence of each local area is obtained according to the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change characteristics of each local area and the other local areas in the laparoscopic image sequence; finally, in each frame of image, according to the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs, the segmentation parameters of the preset segmentation model are adjusted and the cyst area is re-labeled. The present invention first analyzes the possibility of the existence of false boundaries in the suspected cyst area divided by the preset segmentation model through the position and structural characteristics of the false boundaries, further combines the instrument traction and respiratory influence to perform a detailed analysis of the suspected cyst area, evaluates the possibility of each local area being a microcyst, and finally comprehensively adjusts the segmentation accuracy of the segmentation model, thereby accurately segmenting and labeling the real cyst area, thereby improving the recognition effect of ovarian endometriosis cyst lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 A flowchart of a method for real-time laparoscopic identification of ovarian endometriomas provided by one embodiment of the present invention;

[0032] Figure 2 A flow chart of a method for obtaining cyst confidence provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0033] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effectiveness of a laparoscopic real-time identification method for ovarian endometriomas proposed by the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0034] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0035] The specific scheme of the laparoscopic real-time identification method of ovarian endometrioma provided by the present invention is described in detail below with reference to the accompanying drawings.

[0036] See also Figure 1 , which shows a method flow chart of a laparoscopic real-time identification method for ovarian endometriomas provided by one embodiment of the present invention, specifically comprising:

[0037] Step S1: For each observed part of the patient, a laparoscopic image sequence is obtained, a respiratory signal is obtained synchronously, and the suspected cyst area in each frame of the sequence is segmented based on a preset segmentation model, and all edges in each suspected cyst area are obtained.

[0038] In one embodiment of the present invention, a laparoscope is first used to collect laparoscopic images at each observation site of the patient, and the laparoscopic images collected at each observation site are sorted in chronological order to obtain a laparoscopic image sequence. Collecting laparoscopic images is a conventional technique well known to those skilled in the art, and the operation steps are briefly described here:

[0039] Instruct the patient to adopt a modified lithotomy position, and use an inflatable fixation pad to limit the patient's body position from sliding. After anesthesia, the surgical area is disinfected and draped, and pneumoperitoneum is established at the umbilicus (pressure 12-15 mmHg). Then, a high-definition laparoscopic lens is inserted to ensure that the surgical field is accurately displayed and the details are clear. A 5mm auxiliary puncture device is inserted in the lower abdomen on both sides to assist other operating instruments. A standardized four-quadrant exploration method is used to perform a full-scale scan of the patient's abdominal cavity, covering at least the rectouterine pouch, ovary, and broad ligament areas where cysts may occur. Taking any observation area as an example, after the laparoscope approaches the observation area and ensures that the tissue characteristics of the observation area can be clearly observed, the laparoscopic lens is stopped and high-resolution laparoscopic images are continuously collected using the laparoscopic lens. During the scanning process, non-invasive grasping forceps can be used to gently adjust tissues such as the ovary or adhesions to ensure complete observation. The implementer can also further screen out low-quality images caused by smoke interference, bleeding obscuration, and focus misalignment.

[0040] In one embodiment of the present invention, while collecting the laparoscopic image sequence of each observation site, the patient is provided with a chest-strap piezoelectric sensor to synchronously collect the patient's respiratory signal, wherein the sensor's collection frequency is 100 Hz, which can be set by the implementer according to actual application.

[0041] It should be noted that in one embodiment of the present invention, the acquisition time for each observation site is set to 30 seconds. The acquisition of laparoscopic images and respiratory signals requires a unified clock source, and ensures that each frame of image is aligned with the respiratory signal point in the respiratory signal, that is, maintains synchronization, so that each laparoscopic image sequence corresponds to a respiratory signal point. In another embodiment of the present invention, the implementer can also interpolate and align the respiratory signal and the laparoscopic image to ensure that each frame of image corresponds to the same respiratory phase, such as mid-respiratory phase. This is already existing technology and will not be repeated here.

[0042] It should be noted that the analysis and identification method for ovarian endometriomas at each observation site is the same, and only any one observation site is used as an example for analysis and description.

[0043] After obtaining the laparoscopic image sequence of the observation part and its synchronously collected respiratory signal, all suspected cyst areas in each frame of the laparoscopic image sequence can be further obtained; considering that the deep learning-based segmentation model can perform segmentation by learning image features, the embodiment of the present invention will segment the suspected cyst areas in each frame of the sequence based on the preset segmentation model.

[0044] In a preferred embodiment of the present invention, the preset segmentation model is a U-Net model; historical laparoscopic images with manually labeled cyst areas are used as a training set to train the U-Net model; each frame of the collected laparoscopic image sequence is then input into the trained U-Net model, and the model automatically outputs the segmented cyst area; it should be noted that the training and segmentation applications of the U-Net model are both existing technologies well known to those skilled in the art and will not be described in detail.

[0045] Considering that endometriosis cysts are often accompanied by adhesions with surrounding organs (such as the uterus, intestines, etc.), and may also cause inflammatory reactions or fibrosis in surrounding normal tissues, leading to local hardening of normal tissues, these hardened tissues are visually similar to endometriosis cyst lesions, both appearing dark red, and are false boundaries of the cyst area, which may affect the segmentation accuracy of the U-Net model; therefore, the embodiment of the present invention preliminarily regards these cyst areas obtained using the model as suspected cyst areas, and further obtains all edges in each suspected cyst area, so as to subsequently evaluate the possibility that the edge is the edge corresponding to the hardened tissue, and then prepare for subsequent analysis to optimize the model segmentation parameters, thereby improving the segmentation accuracy.

[0046] In one embodiment of the present invention, the Canny edge detection algorithm is specifically used to obtain all edges in each suspected cyst area; this is already a prior art, and implementers may also use other edge detection algorithms, which will not be described in detail here.

[0047] It should be noted that there may be more than one suspected cyst area segmented in each frame of the image, but since the laparoscopic image frames are continuously collected at the same observation site and the same viewing angle, the suspected cyst areas between consecutive frames in the laparoscopic image sequence are also matched, that is, they correspond to the same tissue site; the implementer can match the suspected tumor areas between consecutive frames based on the positional similarity of the center of the suspected cyst area, or can use the scale-invariant feature transform (SIFT) algorithm or the optical flow method to track and match the suspected cyst areas, which are all existing technologies and will not be repeated here; among them, the segmentation and adjustment method of each suspected cyst area in the laparoscopic image sequence is the same, and the subsequent analysis and description will only take any one of the matched suspected cyst areas as an example.

[0048] Step S2: In each frame of the image, the boundary correction coefficient of each suspected cyst area is obtained according to the position distribution of each edge in each suspected cyst area and its position change in the laparoscopic image sequence; each suspected cyst area in each frame of the image is divided into several local areas, and the motion feature sequence of each local area is obtained according to the change characteristics of each local area between adjacent frames of the image.

[0049] Considering that the preset segmentation model may mistakenly identify the pseudo boundary of local sclerotic tissue as a suspected cyst area, and considering that local sclerotic tissue is less affected by the stretching of instrument operation than cysts, has stronger structural stability, and is located at the boundary of the suspected cyst area, when the edge in the suspected cyst area is structurally stable and the closer its position distribution is to the regional boundary, the greater the possibility that the edge corresponds to the local sclerotic tissue, and the more necessary the boundary correction is;

[0050] Based on this, the embodiment of the present invention will obtain the boundary correction coefficient of each suspected cyst area in each frame of the image according to the position distribution of each edge in each suspected cyst area and its position change in the laparoscopic image sequence; the boundary correction coefficient reflects the possibility of the suspected cyst area containing local sclerotic tissue-like false boundaries, preparing for the subsequent adjustment of the model segmentation parameters.

[0051] Preferably, in one embodiment of the present invention, considering that the farther the distance between the pixel point on the edge and the center of the suspected cyst area is, the greater the possibility that it is localized sclerotic tissue, its boundary coefficient can be obtained first; considering that based on the position information of the edge in the suspected cyst area, the edge can be matched, and then the deformation of the edge affected by instrument traction can be evaluated based on the position change of the matched edge in consecutive frames to obtain its structural stability coefficient; finally, the possibility that the edge is localized sclerotic tissue can be comprehensively evaluated to obtain the boundary correction coefficient; based on this, the method for obtaining the boundary correction coefficient includes:

[0052] The boundary coefficient of each edge is obtained based on the distance between each pixel on each edge and the center of the suspected cyst area to which it belongs. The edges in the abdominal image sequence are matched based on the position of the geometric center of each edge, and the structural stability coefficient of each edge is obtained based on the position difference between the geometric centers of the matched edges between all adjacent frames.

[0053] The boundary coefficient and structural stability coefficient are integrated to obtain the boundary attributes of each edge; the boundary attributes of all edges in the cyst area are integrated to obtain the boundary correction coefficient of each suspected cyst area.

[0054] Among them, in a preferred embodiment of the present invention, considering that the larger the boundary coefficient and the larger the structural stability coefficient, the greater the possibility that it is a pseudo boundary of local hardened tissue, and the greater its boundary attribute; the method for obtaining the boundary attribute includes: taking the product of the boundary coefficient and the structural stability coefficient as the boundary attribute.

[0055] As an example, between the matched suspected cyst regions, a coordinate system is constructed with the center of each suspected cyst region as the origin, and the position coordinates of each pixel in the suspected cyst region are obtained. Then, based on the position coordinates, the Euclidean distance of each pixel on each edge in the suspected cyst region relative to the center of the suspected cyst region to which it belongs can be measured, and the mean of all Euclidean distances is used as the boundary coefficient of the edge.

[0056] Then, in the abdominal image sequence, between the matching suspected cyst areas, the edges in different frames are matched, and the centroid of each edge is obtained. The edge with the smallest centroid coordinate difference between different frames is regarded as the matching edge. Then, between the matching edges, the cumulative sum of the centroid position coordinate differences of every two matching edges can be negatively correlated to obtain the structural stability coefficient of the corresponding edge.

[0057] Then, the boundary attributes of each edge can be obtained. The boundary attributes reflect the possibility that the edge is local sclerotic tissue. Finally, the average of the boundary attributes of all edges is used as the boundary correction coefficient of the suspected cyst area. The larger the boundary attribute coefficient, the more obvious the local sclerotic tissue in the suspected cyst area, and the greater the need for subsequent boundary correction.

[0058] Considering that although laparoscopes can provide a clear field of view, the complex tissue structure in the abdominal cavity and the possibility that instrument traction may change the position or morphology of the cyst make the laparoscopic image inconsistent with the actual anatomical structure, making it difficult for the U-Net model to accurately locate and classify tiny cysts with a diameter of less than 5 mm; therefore, the embodiment of the present invention divides each suspected cyst area in each frame of the image into several local areas, so as to finely analyze the suspected cyst area and identify the possibility of the existence of tiny cysts.

[0059] In one embodiment of the present invention, considering that a suspected cyst region may be an irregular shape, the minimum circumscribed moment of each of the matched suspected cyst regions may be first obtained. Then, a local window, such as a 9*9 window, is used to perform a sliding traversal starting from the upper left corner of the minimum circumscribed rectangle to divide the region into a number of equal local regions. The size of each local region is equal to the size of the local window, and the traversal paths of the local windows do not overlap. Furthermore, between the matched suspected cyst regions, the local subregions at the same position are used as the matched local regions.

[0060] It should be noted that there may be slight differences in the area and shape of the matched suspected cyst areas. When there are differences, the implementer can also adjust the minimum bounding rectangle of each suspected cyst area to make all the bounding rectangles the same. For example, the minimum bounding rectangle with the largest area is used as the standard bounding rectangle of all matched suspected cyst areas for further division. The pixel values ​​of the pixels in the local window that do not belong to the suspected cyst area are defined as 0 to facilitate subsequent analysis.

[0061] In another embodiment of the present invention, the implementer can also define the size and shape of the local window by himself, but the local window should not be too large, so as to divide the local area for detailed analysis of the tiny cysts; then, the SIFT algorithm is used to perform feature matching between the matching suspected cyst areas to obtain the matching local window of each local area.

[0062] Taking into account that the intra-abdominal pressure will change with breathing, which may cause the displacement of intra-abdominal tissues such as ovaries, the movement of the local area should have a strong correlation with the patient's breathing; when the correlation is weak, it means that the local area is more likely to be the area corresponding to the ectopic cyst, because the ectopic cyst may cause the stiffness or elasticity of the ovarian tissue to change, thereby affecting its response to respiratory movement, and thus changing the movement characteristics of the area; therefore, the embodiment of the present invention will further obtain the motion feature sequence of each local area based on the change characteristics of each local area between adjacent frame images, in preparation for the subsequent analysis of the correlation between the movement of the local area and breathing.

[0063] Preferably, in one embodiment of the present invention, considering that between adjacent frames of image, the pixel changes of the matching local area can be analyzed based on the optical flow method to evaluate its motion information; at the same time, the feature points or corner points in the local area may usually represent important tissues, so the displacement of the corner points can also help reflect the motion information of the local area; based on this, the method for obtaining the motion feature sequence includes:

[0064] Between each frame and the previous frame, the motion vector of each pixel in each local area is obtained based on the optical flow method, and the relative displacement between the matching corner points in the local areas at the same position between adjacent frames is obtained;

[0065] Based on the motion vectors of all pixels in each local area and the relative displacement between all matching corner points, the motion feature coefficients of each local area between adjacent frames are obtained; in the laparoscopic image sequence, the motion feature coefficients of local areas at the same position between adjacent frames are sorted in chronological order to construct a motion feature sequence.

[0066] As an example, first, between each frame image and the adjacent previous frame image, between the matching local areas, the motion vector of each pixel in the local area of ​​each frame image is obtained based on the optical flow method, and then the sum of the motion vectors of all pixels is used as the motion vector of the local area in each frame image;

[0067] Further corner point detection is performed on each local area to obtain all corner points, and a coordinate system is constructed with the lower left corner pixel of the local area as the origin to obtain the coordinates of each corner point; then the corner points are matched between the matching local areas, so that based on the coordinates of the corner points, the relative displacement between the matching corner points at the same position between adjacent frames, i.e., between the matching local areas, is obtained, and then the average of the relative displacements between all the matching corner points is used as the relative displacement of the local window in each frame of the image;

[0068] Finally, the product of the motion vector and the relative displacement of each local area is used as the motion feature coefficient of the local window in each frame of the image; the motion feature sequence is further constructed; it should be noted that, in order to facilitate subsequent calculations, the implementer also needs to take the modulus length of the motion vector and then participate in the product operation to obtain the motion feature coefficient; in the laparoscopic image sequence, since the first frame image has no previous adjacent frame, it is impossible to perform change comparison, so the length of the motion feature sequence is 1 less than the length of the laparoscopic image sequence.

[0069] It should be noted that the application of the optical flow method, the motion vector modulus length, the corner point detection and matching, and the acquisition of the relative displacement are all well-known technical means and will not be described in detail.

[0070] Step S3, obtaining a respiratory feature sequence based on the fluctuation of the respiratory signal, and obtaining the cyst confidence of each local area based on the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change features of each local area and the other local areas in the laparoscopic image sequence; in each frame of the image, adjusting the segmentation parameters of the preset segmentation model based on the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs, and re-labeling the cyst area.

[0071] Considering that the patient's breathing may fluctuate during laparoscopic operation, and there are different stages or phases of breathing, including inspiration, expiration and mid-respiration, it is difficult to measure the respiratory fluctuation characteristics; however, the respiratory signal under the same respiratory phase can help evaluate its fluctuation characteristics.

[0072] Based on this, in a preferred embodiment of the present invention, the respiratory phase of each signal point in the respiratory signal is first determined, and then the respiratory signal points at the same phase are sorted and fitted to the respiratory signal at the corresponding phase, thereby evaluating its fluctuation characteristics. Therefore, the method for obtaining the respiratory feature sequence includes:

[0073] Based on a preset phase segmentation algorithm, the respiratory phase of each signal point in the respiratory signal is marked, and the signal points under the same respiratory phase are sorted in time sequence to construct a respiratory phase signal under the corresponding respiratory phase;

[0074] In each respiratory phase signal, the respiratory characteristic coefficient at each moment is obtained according to the signal fluctuation intensity in the preset historical time domain of the signal point at each moment; all the respiratory characteristic coefficients are sorted in time sequence to construct a respiratory characteristic sequence.

[0075] As an example, the preset phase segmentation algorithm is a respiratory signal phase segmentation algorithm based on a temporal convolutional network, which can be used to automatically identify and divide the various respiratory phases in a single respiratory cycle, including inspiration, exhalation and mid-respiration; then, taking any phase as an example, the signal points are sorted in time sequence, and the respiratory phase signal under the corresponding respiratory phase is fitted; the implementer may also adopt other phase labeling models such as those based on deep learning algorithms, which are all existing technologies well known to those skilled in the art and will not be repeated here.

[0076] In a preferred embodiment of the present invention, considering that a longer time series may smooth out the fluctuation characteristics in each respiratory phase signal, the fluctuation characteristics are analyzed in the local historical time domain of each signal point. Furthermore, considering that the variance can reflect the intensity of the fluctuation of the signal point, and that the closer the amplitude levels of the signal points are, the smaller the fluctuation is, the frequency of occurrence of signal points of each amplitude can be used to measure the degree of fluctuation. Based on this, the method for obtaining the respiratory characteristic coefficient includes:

[0077] In each respiratory phase signal, within the preset historical time domain of the signal point at each moment, the variance of the signal point is used as the first fluctuation parameter, and the second fluctuation parameter is obtained by combining the negative correlation mapping results of the occurrence frequency of signal points of each amplitude; the first fluctuation parameter and the second fluctuation parameter are fused to obtain the respiratory characteristic coefficient.

[0078] As an example, first, in each respiratory phase signal, the signal point at each moment is taken as the end point, and a preset number of historical signal points, such as 9, are obtained in the reverse direction of the timing sequence to construct a preset historical time domain, so as to obtain the first fluctuation parameter of the signal segment in the preset historical time domain; then, the amplitude of the signal points in the signal segment in the preset historical time domain is counted, and the frequency of occurrence of signal points of each amplitude is mapped to the exponential function exp(-x) with the natural constant e as the base. When the frequency of occurrence is less and the exponential function value is larger, it means that the amplitude of the signal segment changes more frequently and the possibility of fluctuation is greater. Then, the exponential function values ​​corresponding to the frequency of occurrence of signal points of all amplitudes are multiplied to obtain the second fluctuation parameter; finally, the first fluctuation parameter and the second fluctuation parameter are multiplied and fused to obtain the respiratory characteristic coefficient.

[0079] It should be noted that the implementer may also define a preset number by himself. When the signal points in the reverse direction of the time sequence are less than the preset number of historical signal points, the number less than the preset number will be regarded as a preset historical time domain.

[0080] Considering that the movement of tissues such as the ovaries in the abdominal cavity is affected not only by respiration but also by instrument traction, instrument traction can lead to loss of detail in local areas, and the corresponding edges of the tissues are more likely to deform or overlap. Therefore, the greater the impact of instrument traction on a local area, the greater the need for correction.

[0081] Based on this, after obtaining the respiratory signal feature sequence, the embodiment of the present invention can further obtain the cyst confidence of each local area based on the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change features of each local area and the other local areas in the laparoscopic image sequence; the cyst confidence not only reflects the possibility that the local area is a cyst area, but also reflects the possibility of its boundary blurring caused by instrument traction, preparing for the subsequent adjustment of the segmentation accuracy.

[0082] Preferably, in one embodiment of the present invention, considering that the Pearson correlation coefficient can reflect the correlation between changes in sequences, and considering that the greater the deviation of the corresponding motion feature sequence of the target region relative to the remaining adjacent local regions, the greater the possibility that the target region is relatively deformed by the instrument, and the greater the possibility that the local region is a cyst or has undergone tissue deformation; therefore, the method for obtaining the cyst confidence includes:

[0083] See also Figure 2 , which shows a flow chart of a method for obtaining cyst confidence provided by an embodiment of the present invention, specifically comprising:

[0084] Step S201 , in a laparoscopic image sequence, any local area is taken as a target area, and all local areas adjacent to the target area are taken as reference areas; the deformation coefficient of the target area is obtained based on the sequence difference between the motion feature sequence of the target area and the motion feature sequence of each reference area.

[0085] As an example, a target area is first determined, and then the Manhattan distance is used to measure the sequence difference between the motion feature sequence of the target area and the motion feature sequence of each reference area. The mean of the Manhattan distance is then used as the deformation coefficient of the target area. The greater the sequence difference, the more deviated the motion feature of the target area is relative to the other reference areas, and the larger the deformation coefficient of the target area.

[0086] Step S202 , taking the absolute value of the Pearson correlation coefficient between the respiratory feature sequence and the motion feature sequence and performing negative correlation mapping to obtain a respiratory weight.

[0087] As an example, the absolute value of the Pearson correlation coefficient between the respiratory feature sequence and the motion feature sequence is taken and a preset non-zero positive parameter such as 0.001 is added, and then an inverse operation is performed to perform a negative correlation mapping to obtain the respiratory weight. When the Pearson correlation coefficient approaches 0, the smaller the correlation between the respiratory feature and the motion feature, and the greater the possibility that the local area is the corresponding area of ​​the ectopic cyst.

[0088] Step S203: weighting the deformation coefficient using the respiratory weight, and using the weighted result as the cyst confidence of the target area.

[0089] As an example, the breathing weight is multiplied by the deformation coefficient to obtain the cyst confidence of the target area; by changing the target area, the cyst confidence of each local area can be obtained.

[0090] It should be noted that the Manhattan distance and the Pearson correlation coefficient are both well-known technologies and will not be described in detail.

[0091] After obtaining the cyst confidence of each local area, it can be further combined with the boundary correction coefficient of the suspected cyst area to which it belongs, so as to re-label the cyst area in each frame of the image, improve the model segmentation accuracy, and accurately segment the real cyst area.

[0092] Preferably, in one embodiment of the present invention, considering that the larger the boundary correction coefficient and the cyst confidence, the greater the boundary blur of the local area, the more likely the model is to mistakenly segment more false boundaries, and the U-Net segmentation model usually outputs a cyst probability map for each frame of the image, and then determines the suspected cyst area according to the segmentation threshold. By adjusting the segmentation threshold of the preset segmentation model, the segmentation accuracy can be appropriately improved and false boundaries can be eliminated; therefore, the method of re-marking the cyst area includes:

[0093] In each frame of the image, the segmentation adjustment weight of each local area is obtained according to the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs; the initial segmentation threshold when the preset segmentation model segments the suspected cyst area is obtained; the initial segmentation threshold is weighted by the segmentation adjustment weight, and the weighted result is used as the segmentation threshold of the corresponding local area, and segmentation is performed to obtain the cyst sub-area in the local area; the cyst sub-areas in all local areas in the suspected tumor area are combined to obtain the cyst area in each frame of the image and mark it.

[0094] Among them, in a preferred embodiment of the present invention, considering that the greater the cyst confidence of the local area, the higher the segmentation threshold of the local area should be adjusted to more finely divide the cyst area; considering that the boundary correction coefficient of the suspected cyst area to which it belongs also provides a certain reference for improving the segmentation accuracy, the larger the boundary correction coefficient, the greater the possibility of the existence of a false boundary in the suspected cyst area; therefore, the method for obtaining the segmentation adjustment weight includes:

[0095] In each frame of the image, the product of the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs is used as the adjustment amplitude index, and the sum of the constant 1 and the adjustment amplitude index is used as the segmentation adjustment weight.

[0096] After obtaining the segmentation adjustment weight of each local area, the initial segmentation threshold for segmenting the suspected cyst area in each frame of the image can be further obtained based on the U-Net model. The segmentation adjustment weight is then multiplied by the initial segmentation threshold to obtain the segmentation threshold of the local area in each frame of the image. The cyst sub-area in the local area can then be segmented based on the segmentation threshold. The cyst sub-area of ​​each local area in the suspected cyst area can then be obtained. All cyst sub-areas are then connected, and each connected area is regarded as a cyst area. The real cyst area that has been precisely segmented in each suspected cyst area in each frame of the image is obtained. The cyst area can then be labeled to assist relevant medical personnel in observation and identification.

[0097] It should be noted that obtaining the initial segmentation threshold of the U-Net model and performing regional connectivity are both existing technical means and will not be described in detail.

[0098] In summary, the present invention first obtains a laparoscopic image sequence and a respiratory signal, and segments the suspected cyst area in each frame of the sequence based on a preset segmentation model; then obtains the boundary correction coefficient of each suspected cyst area in each frame of the image; further divides each suspected cyst area in each frame of the image into several local areas, and obtains the cyst confidence of each local area; and then adjusts the segmentation parameters of the preset segmentation model in each frame of the image, and re-labels the cyst area. The present invention first analyzes the possibility of the existence of false boundaries in the suspected cyst area divided by the preset segmentation model through the position and structural characteristics of the false boundaries, and further combines the instrument traction and respiratory effects to perform a detailed analysis of the suspected cyst area, evaluates the possibility of each local area being a microcyst, and finally comprehensively adjusts the segmentation accuracy of the segmentation model, thereby accurately segmenting and labeling the real cyst area, thereby improving the recognition effect of ovarian endometriosis cyst lesions.

[0099] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A laparoscopic real-time identification method for ovarian endometrioma, characterized in that: The method comprises: For each observed part of the patient, a laparoscopic image sequence is acquired, along with the respiratory signal, and the suspected cyst region in each frame of the sequence is segmented based on a preset segmentation model. All edges within each suspected cyst region are also obtained. In each frame of the image, the boundary correction coefficient of each suspected cyst region is obtained based on the position distribution of each edge in each suspected cyst region and its position change in the laparoscopic image sequence; each suspected cyst region in each frame of the image is divided into several local regions, and the motion feature sequence of each local region is obtained based on the change characteristics of each local region between adjacent frames of the image; A respiratory feature sequence is obtained based on the fluctuation of the respiratory signal. The cyst confidence of each local area is obtained based on the correlation between the changes in the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change features of each local area and the rest of the local areas in the laparoscopic image sequence. In each frame of the image, the segmentation parameters of the preset segmentation model are adjusted based on the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs, and the cyst area is re-labeled. The method for re-marking the cyst area includes: In each frame of the image, the segmentation adjustment weight of each local area is obtained according to the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs; the initial segmentation threshold when the preset segmentation model segments the suspected cyst area is obtained; the initial segmentation threshold is weighted by the segmentation adjustment weight, and the weighted result is used as the segmentation threshold of the corresponding local area, and segmentation is performed to obtain the cyst sub-area in the local area; the cyst sub-areas in all local areas in the suspected tumor area are integrated to obtain the cyst area in each frame of the image and mark it.

2. The laparoscopic real-time identification method for ovarian endometrioma according to claim 1, characterized in that: The method for obtaining the boundary correction coefficient includes: The boundary coefficient of each edge is obtained based on the distance between each pixel on each edge and the center of the suspected cyst area to which it belongs. The edges in the abdominal image sequence are matched based on the position of the geometric center of each edge, and the structural stability coefficient of each edge is obtained based on the position difference between the geometric centers of the matched edges between all adjacent frames. The boundary coefficient and the structural stability coefficient are integrated to obtain the boundary attribute of each edge; the boundary attributes of all edges in the cyst area are combined to obtain the boundary correction coefficient of each suspected cyst area.

3. The laparoscopic real-time identification method for ovarian endometrioma according to claim 2, characterized in that: The method for obtaining the boundary attribute includes: The product of the boundary coefficient and the structural stability coefficient is used as the boundary attribute.

4. The laparoscopic real-time identification method for ovarian endometrioma according to claim 1, characterized in that: The method for acquiring the motion feature sequence includes: Between each frame and the previous frame, the motion vector of each pixel in each local area is obtained based on the optical flow method, and the relative displacement between the matching corner points in the local areas at the same position between adjacent frames is obtained; Based on the motion vectors of all pixels in each local area and the relative displacement between all matching corner points, the motion feature coefficients of each local area between adjacent frames are obtained; in the laparoscopic image sequence, the motion feature coefficients of the local areas at the same position between adjacent frames are sorted in chronological order to construct a motion feature sequence.

5. The laparoscopic real-time identification method for ovarian endometrioma according to claim 1, characterized in that: The method for acquiring the respiratory feature sequence includes: Based on a preset phase segmentation algorithm, the respiratory phase of each signal point in the respiratory signal is marked, and the signal points under the same respiratory phase are sorted in time sequence to construct a respiratory phase signal under the corresponding respiratory phase; In each respiratory phase signal, the respiratory characteristic coefficient at each moment is obtained according to the signal fluctuation intensity in the preset historical time domain of the signal point at each moment; all the respiratory characteristic coefficients are sorted in time sequence to construct a respiratory characteristic sequence.

6. The laparoscopic real-time identification method for ovarian endometrioma according to claim 5, characterized in that: The method for obtaining the respiratory characteristic coefficient includes: In each respiratory phase signal, within the preset historical time domain of the signal point at each moment, the variance of the signal point is used as the first fluctuation parameter, and the second fluctuation parameter is obtained by combining the negative correlation mapping results of the occurrence frequency of signal points of each amplitude; the first fluctuation parameter and the second fluctuation parameter are fused to obtain the respiratory characteristic coefficient.

7. The laparoscopic real-time identification method for ovarian endometrioma according to claim 1, characterized in that: The method for obtaining the cyst confidence level includes: In a laparoscopic image sequence, any local area is taken as the target area, and all local areas adjacent to the target area are taken as reference areas; the deformation coefficient of the target area is obtained based on the sequence difference between the motion feature sequence of the target area and the motion feature sequence of each reference area; the absolute value of the Pearson correlation coefficient between the respiratory feature sequence and the motion feature sequence is taken and negative correlation mapping is performed to obtain the respiratory weight; the deformation coefficient is weighted using the respiratory weight, and the weighted result is used as the cyst confidence of the target area.

8. The laparoscopic real-time identification method for ovarian endometrioma according to claim 1, characterized in that: The method for obtaining the segmentation adjustment weight includes: In each frame of the image, the product of the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs is used as the adjustment amplitude index, and the sum of the constant 1 and the adjustment amplitude index is used as the segmentation adjustment weight.

9. The laparoscopic real-time identification method for ovarian endometrioma according to claim 1, characterized in that: The preset segmentation model is a U-Net model.

Images