Laparoscope real-time identification method for ovarian endometriosis cyst

Abstract

The invention relates to the technical field of laparoscope image labeling, in particular to a laparoscope real-time recognition method for ovarian endometriosis cyst. The method comprises the following steps: firstly, analyzing the possibility that suspected cyst areas divided by a preset segmentation model have false boundaries in combination with the position and structural features of the false boundaries, and determining a boundary correction coefficient of each suspected cyst area in each frame of image; further dividing each suspected cyst area in each frame of image into a plurality of local areas, performing fine analysis on the suspected cyst areas in combination with instrument traction and breathing influence, evaluating the cyst confidence coefficient of each local area as a micro cyst, and finally comprehensively adjusting segmentation parameters of the segmentation model to obtain a micro cyst segmentation result. Therefore, the real cyst area is accurately segmented and marked, and the recognition effect of the ovarian endometriosis cyst focus is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of laparoscopic image annotation, and particularly to a real-time laparoscopic recognition method for ovarian endometriotic cysts. Background Art

[0002] Ovarian endometriosis refers to the ectopic growth of endometrial tissue outside the uterine cavity, commonly in the ovary, thus forming ovarian endometriotic cysts. Laparoscopy can help medical staff directly observe the ectopic lesions in the ovary and other pelvic parts. Therefore, based on the visual images during the laparoscopic operation, the abdominal cavity images of each observation site, such as the rectouterine pouch, ovarian fossa, and broad ligament, can be collected in real time, and then the abdominal cavity images are analyzed for characteristics to identify and label ovarian endometriotic cysts.

[0003] However, during the laparoscopic operation, factors such as instrument traction, changes in pneumoperitoneum pressure, or cyst fluid leakage can all cause displacement or morphological changes of the ovary and cysts. At the same time, the complex adhesions and blurred anatomical hierarchical structures in the abdominal cavity may make it difficult to accurately identify some small lesions in the laparoscopic images, and the normal tissues around the cyst lesions may be misidentified as cyst lesions, that is, there is a possibility of false boundaries, resulting in poor recognition effect of ovarian endometriotic cyst lesions. Summary of the Invention

[0004] In order to solve the technical problem of poor recognition effect of ovarian endometriotic cyst lesions, the purpose of the present invention is to provide a real-time laparoscopic recognition method for ovarian endometriotic cysts. The specific technical solution adopted is as follows: For each observation site of the patient, obtain a laparoscopic image sequence, synchronously obtain the respiratory signal, segment the suspected cyst regions in each frame of the sequence based on a preset segmentation model, and obtain all the edges within each suspected cyst region; In each frame of the image, according to the position distribution of each edge within each suspected cyst region and its position change in the laparoscopic image sequence, obtain the boundary correction coefficient of each suspected cyst region; divide each suspected cyst region in each frame of the image into several local regions, and obtain the motion feature sequence of each local region according to the change characteristics between adjacent frames of the image. Obtain the respiratory feature sequence according to the fluctuation of the respiratory signal, and obtain the cyst confidence of each local region according to the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change characteristics of each local region and the other local regions in the laparoscopic image sequence; in each frame of the image, adjust the segmentation parameters of the preset segmentation model according to the cyst confidence of each local region and the boundary correction coefficient of the suspected cyst region to which it belongs, and re-label the cyst region.

[0005] Furthermore, the method for obtaining the boundary correction coefficient includes: According to the distance of each pixel point on each edge from the center of the suspected cyst region to which it belongs, obtain the boundary coefficient of each edge; according to the position of the geometric center of each edge, match the edges in the abdominal cavity image sequence, and between all adjacent frames, obtain the structural stability coefficient of each edge according to the position difference between the geometric centers of the matched edges; Fuse the boundary coefficient and the structural stability coefficient to obtain the boundary attribute of each edge; comprehensively obtain the boundary correction coefficient of each suspected cyst region based on the boundary attributes of all edges within the cyst region.

[0006] Furthermore, the method for obtaining the boundary attribute includes: Take the product of the boundary coefficient and the structural stability coefficient as the boundary attribute.

[0007] Furthermore, the method for obtaining the motion feature sequence includes: Between each frame of image and the adjacent previous frame of image, based on the optical flow method, obtain the motion vector of each pixel point in each local region, and obtain the relative displacement between the matching corner points between the local regions at the same position in adjacent frames; According to the motion vectors of all pixel points in each local region and the relative displacement between all matching corner points, obtain the motion feature coefficient of each local region between adjacent frames; in the laparoscopic image sequence, sort the motion feature coefficients of the local regions at the same position between adjacent frames in chronological order to construct a motion feature sequence.

[0008] Furthermore, the method for obtaining the respiration feature sequence includes: Based on a preset phase segmentation algorithm, label the respiration phase of each signal point in the respiration signal, and sort the signal points under the same respiration phase in chronological order to construct a respiration phase signal corresponding to the respiration phase; In each respiration phase signal, according to the signal fluctuation intensity within the preset historical time domain of the signal point at each moment, obtain the respiration feature coefficient at each moment; sort all the respiration feature coefficients in chronological order to construct a respiration feature sequence.

[0009] Furthermore, the method for obtaining the respiration feature coefficient includes: In each respiration phase signal, within the preset historical time domain of the signal point at each moment, take the variance of the signal point as the first fluctuation parameter, and comprehensively obtain the second fluctuation parameter by the negative correlation mapping result of the occurrence frequency of signal points of each amplitude; fuse the first fluctuation parameter and the second fluctuation parameter to obtain the respiration feature coefficient.

[0010] Further, the method for obtaining the cyst confidence includes: In the laparoscopic image sequence, taking any local area as the target area, and taking all local areas adjacent to the target area as the reference areas; obtaining the deformation coefficient of the target area according to the sequence difference between the motion feature sequence of the target area and the motion feature sequences of each reference area; taking the absolute value of the Pearson correlation coefficient between the respiratory feature sequence and the motion feature sequence and performing negative correlation mapping to obtain the respiratory weight; weighting the deformation coefficient with the respiratory weight, and taking the weighted result as the cyst confidence of the target area.

[0011] Further, the method for relabeling the cyst area includes: In each frame of image, obtaining the segmentation adjustment weight value of each local area according to the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs; obtaining the initial segmentation threshold when the preset segmentation model segments the suspected cyst area; weighting the initial segmentation threshold with the segmentation adjustment weight value, taking the weighted result as the segmentation threshold of the corresponding local area, and performing segmentation to obtain the cyst sub-area in the local area; integrating the cyst sub-areas in all local areas within the suspected tumor area to obtain the cyst area in each frame of image and label it.

[0012] Further, the method for obtaining the segmentation adjustment weight value includes: In each frame of image, taking the product of the cyst confidence of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs as the adjustment amplitude index, and taking the sum value of the constant 1 and the adjustment amplitude index as the segmentation adjustment weight value.

[0013] Further, the preset segmentation model is a U-Net model.

[0014] The present invention has the following beneficial effects: For each observed part of a patient, the present invention first obtains a laparoscopic image sequence, synchronously obtains a respiratory signal, segments the suspected cyst regions in each frame of the sequence based on a preset segmentation model, and obtains all the edges within each suspected cyst region to prepare for evaluating the existence of pseudo-boundaries within the suspected cyst regions and whether boundary correction is required; then, in each frame of the image, according to the position distribution of each edge within each suspected cyst region and its position change in the laparoscopic image sequence, it evaluates the possibility of the existence of pseudo-boundaries and obtains the boundary correction coefficient for each suspected cyst region; further, each suspected cyst region in each frame of the image is divided into several local regions for refined analysis, and according to the change characteristics of each local region between adjacent frames of images, it obtains the motion feature sequence of each local region to prepare for evaluating the cyst confidence of the local region by combining instrument traction and respiratory influence; then, it obtains the respiratory feature sequence according to the fluctuation of the respiratory signal, and according to the correlation between the change of the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change characteristics of each local region and the other local regions in the laparoscopic image sequence, it obtains the cyst confidence of each local region; finally, in each frame of the image, according to the cyst confidence of each local region and the boundary correction coefficient of the suspected cyst region to which it belongs, it adjusts the segmentation parameters of the preset segmentation model and relabels the cyst region. The present invention first analyzes the possibility of the existence of pseudo-boundaries in the suspected cyst regions divided by the preset segmentation model through the position and structural characteristics of the pseudo-boundaries, further conducts a refined analysis of the suspected cyst regions by combining instrument traction and respiratory influence, evaluates the possibility of each local region being a micro-cyst, and finally comprehensively adjusts the segmentation accuracy of the segmentation model, thereby accurately segmenting and labeling the real cyst region, improving the recognition effect of ovarian endometriosis cyst lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of a method for real-time laparoscopic recognition of ovarian endometriosis cysts provided by an embodiment of the present invention; Figure 2 It is a flowchart of a method for obtaining cyst confidence provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following specifically describes, in conjunction with the accompanying drawings and preferred embodiments, a laparoscopic real-time identification method for ovarian endometriotic cysts according to the present invention, including its specific implementation manner, structure, features and effects, as follows. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0019] The following specifically describes, in conjunction with the accompanying drawings, the specific solution of a laparoscopic real-time identification method for ovarian endometriotic cysts provided by the present invention.

[0020] Please refer to Figure 1 , which shows the method flow chart of a laparoscopic real-time identification method for ovarian endometriotic cysts provided by an embodiment of the present invention, specifically including: Step S1, for each observation site of the patient, obtain a laparoscopic image sequence, synchronously obtain a respiratory signal, segment the suspected cyst regions in each frame of the image in the sequence based on a preset segmentation model, and obtain all the edges within each suspected cyst region.

[0021] In an embodiment of the present invention, first, laparoscopic images are collected at each observation site of the patient, and the laparoscopic images collected at each observation site are sorted in chronological order to obtain a laparoscopic image sequence; collecting laparoscopic images is a well-known prior art means for those skilled in the art, and only the operation steps are briefly described here: Instruct the patient to adopt the modified lithotomy position and use an inflatable fixing pad to limit the patient's body position from sliding; after anesthesia, disinfect and drape the surgical area, establish pneumoperitoneum (pressure 12 - 15 mmHg) at the umbilicus, and then insert a high-definition laparoscopic lens to ensure accurate color display and clear details of the surgical field; insert 5 mm auxiliary trocars bilaterally in the lower abdomen to assist other operating instruments; use the standardized four-quadrant exploration method to perform a full-range scan of the patient's abdominal cavity, covering at least the observation sites where cysts may appear, such as the rectouterine pouch, ovaries, and broad ligament regions; taking any observation site as an example, after the laparoscopic lens approaches the observation site and ensures that the tissue characteristics of the observation site can be clearly observed, stop moving the laparoscopic lens, and continuously collect high-resolution laparoscopic images using the laparoscopic lens; during the scanning process, a non-invasive grasping forceps can be used to gently adjust tissues such as the ovaries or adhesions to ensure complete observation; the operator can also further screen out low-quality images with smoke interference, bleeding occlusion, and out-of-focus.

[0022] In one embodiment of the present invention, while collecting the laparoscopic image sequence of each observation site, a chest strap piezoelectric sensor is worn on the patient to synchronously collect the patient's respiratory signal, where the acquisition frequency of the sensor is 100Hz, and the implementer can also set it according to actual applications.

[0023] It should be noted that, in one embodiment of the present invention, the acquisition duration of each observation site is set to 30s. The acquisition of laparoscopic images and the acquisition of respiratory signals require a unified clock source, and it is necessary to ensure that each frame of image can be aligned with the respiratory signal points in the respiratory signal, that is, to maintain synchronization, so that each laparoscopic image sequence corresponds to a respiratory signal point. In another embodiment of the present invention, the implementer can also perform interpolation alignment on the respiratory signal and the laparoscopic image to ensure that each frame of image corresponds to the same respiratory phase, such as the mid-respiratory phase, which is already a prior art and will not be elaborated here.

[0024] It should be noted that the analysis and recognition methods for ovarian endometriosis cysts at each observation site are the same. Here, only one observation site is taken as an example for analysis and description.

[0025] After obtaining the laparoscopic image sequence of the observation site and its synchronously collected respiratory signal, all suspected cyst regions in each frame of the laparoscopic image sequence can be further obtained; considering that the segmentation model based on deep learning can perform segmentation by learning image features, the embodiments of the present invention will segment the suspected cyst regions in each frame of the sequence based on a preset segmentation model.

[0026] In a preferred embodiment of the present invention, the preset segmentation model is a U-Net model; the historical laparoscopic images with manually marked cyst regions are used as the training set, and then the U-Net model is trained; then each frame of the collected laparoscopic image sequence is input into the trained U-Net model, and the model will automatically output the segmented cyst regions; it should be noted that the training and segmentation application of the U-Net model are both well-known prior arts to those skilled in the art and will not be elaborated here.

[0027] Also considering that endometriosis cysts are often accompanied by adhesions with surrounding organs (such as the uterus, intestines, etc.), and will also cause inflammatory reactions or fibrosis in the surrounding normal tissues, resulting in sclerosis of the local normal tissues. These sclerotic tissues are visually similar to the ectopic cyst lesions, both showing dark red, which are the pseudo-boundaries of the cyst regions and may affect the segmentation accuracy of the U-Net model; therefore, the cyst regions obtained by the model in the embodiments of the present invention are initially used as suspected cyst regions, and all edges within each suspected cyst region are further obtained to evaluate the possibility that the edge corresponds to the edge of the sclerotic tissue, so as to prepare for subsequent analysis to optimize the model segmentation parameters and improve the segmentation accuracy.

[0028] In an embodiment of the present invention, the Canny edge detection algorithm is specifically used to obtain all the edges in each suspected cyst region; this is already a prior art, and the implementer can also use other edge detection algorithms, which will not be elaborated here.

[0029] It should be noted that there may be more than one suspected cyst region segmented from each frame of the image. However, since the laparoscopic image frames are continuously acquired at the same observation site from the same perspective, the suspected cyst regions between consecutive frames in the laparoscopic image sequence are also matched, that is, corresponding to the same tissue site. The implementer can match the suspected tumor regions between consecutive frames based on the similarity of the positions of the centers of the suspected cyst regions, or can also use the Scale-invariant feature transform (SIFT) algorithm or the optical flow method to track and match the suspected cyst regions. These are all prior arts and will not be elaborated here. Among them, the segmentation and adjustment methods for each suspected cyst region in the laparoscopic image sequence are the same. Subsequently, only one of the matched suspected cyst regions will be taken as an example for analysis and description.

[0030] Step S2: In each frame of the image, according to the position distribution of each edge in each suspected cyst region and its position change in the laparoscopic image sequence, obtain the boundary correction coefficient for each suspected cyst region; divide each suspected cyst region in each frame of the image into several local regions, and according to the change characteristics of each local region between adjacent frame images, obtain the motion feature sequence for each local region.

[0031] Considering that the preset segmentation model may misidentify the pseudo-boundaries of local sclerosed tissue as suspected cyst regions, and considering that the local sclerosed tissue is less affected by the stretching of the instrument operation compared to the cyst, has strong structural stability, and is located at the boundary position of the suspected cyst region, so when the edge in the suspected cyst region has structural stability and the position distribution tends to the region boundary more, the greater the possibility that this edge corresponds to the edge of the local sclerosed tissue, and the more the boundary needs to be corrected; Based on this, the embodiment of the present invention will, in each frame of the image, according to the position distribution of each edge in each suspected cyst region and its position change in the laparoscopic image sequence, obtain the boundary correction coefficient for each suspected cyst region; the boundary correction coefficient reflects the possibility of including pseudo-boundaries of local sclerosed tissue in the suspected cyst region, and prepares for adjusting the model segmentation parameters in the subsequent steps.

[0032] Preferably, in an embodiment of the present invention, considering that the farther the pixel points on the edge are from the center of the suspected cyst area, the greater the possibility that they are local sclerotic tissues, so the boundary coefficient can be obtained first; also considering that based on the position information of the edge within the suspected cyst area, the edges can be matched, and then the deformation situation affected by the instrument traction can be evaluated according to the position change of the matched edges in consecutive frames, and the structural stability coefficient can be obtained; finally, the possibility that the edge is a local sclerotic tissue can be comprehensively evaluated to obtain the boundary correction coefficient; based on this, the method for obtaining the boundary correction coefficient includes: According to the distance of each pixel point on each edge from the center of the suspected cyst area to which it belongs, obtain the boundary coefficient of each edge; according to the position of the geometric center of each edge, match the edges in the abdominal cavity image sequence, and between all adjacent frames, obtain the structural stability coefficient of each edge according to the position difference between the geometric centers of the matched edges; Fuse the boundary coefficient and the structural stability coefficient to obtain the boundary attribute of each edge; comprehensively obtain the boundary correction coefficient of each suspected cyst area based on the boundary attributes of all edges within the cyst area.

[0033] Among them, in a preferred embodiment of the present invention, considering that the larger the boundary coefficient and the larger the structural stability coefficient, the greater the possibility that it is a pseudo-boundary of local sclerotic tissue, and the greater its boundary attribute; then the method for obtaining the boundary attribute includes: taking the product of the boundary coefficient and the structural stability coefficient as the boundary attribute.

[0034] As an example, between the matched suspected cyst areas, a coordinate system is constructed with the center of each suspected cyst area as the origin, and the position coordinates of each pixel point within the suspected cyst area are obtained. Then, based on the position coordinates, the Euclidean distance of each pixel point on each edge within the suspected cyst area from the center of the suspected cyst area to which it belongs can be measured, and the mean value of all Euclidean distances is used as the boundary coefficient of the edge; Then, in the abdominal cavity image sequence, between the matched suspected cyst areas, the edges in different frame images are matched. Specifically, the centroid of each edge is obtained, and the edge with the smallest difference in the corresponding coordinates of the centroids between different frames is used as the matched edge. Then, between the matched edges, the cumulative sum of the position coordinate differences of the centroids of every two matched edges is subjected to a negative correlation mapping to obtain the structural stability coefficient of the corresponding edge; Furthermore, the boundary attribute of each edge can be obtained. The boundary attribute reflects the possibility that the edge is a local sclerotic tissue. Finally, the mean value of the boundary attributes of all edges is used as the boundary correction coefficient of the suspected cyst area; the larger the boundary attribute coefficient, the more obvious the local sclerotic tissue within the suspected cyst area, and the greater the necessity for subsequent boundary correction.

[0035] Considering that although laparoscopy can provide a clear view, due to the complex intra-abdominal tissue structure and the fact that instrument traction may change the position or shape of the cyst, the laparoscopic image does not match the actual anatomical structure, making it difficult for the U-Net model to accurately locate and divide micro-cysts with a diameter less than 5 mm. Therefore, in the embodiments of the present invention, each suspected cyst area in each frame of the image is divided into several local areas to finely analyze the suspected cyst area and identify the possibility of the existence of micro-cysts.

[0036] In an embodiment of the present invention, considering that the suspected cyst area may be an irregular shape, first, in the matching suspected cyst area, the minimum bounding rectangle can be taken for each suspected cyst area, and then a local window such as 9*9 is used to slide and traverse from the upper left corner of the minimum bounding rectangle to divide several equal local areas. The size of each local area is equal to the size of the local window, and the traversal path of the local window does not overlap. Further, between the matching suspected cyst areas, the local sub-areas at the same position are used as the matching local areas. It should be noted that there may be slight differences in the area and shape of the matching suspected cyst areas. When there are differences, the implementer can also adjust the minimum bounding rectangle of each suspected cyst area by himself / herself to make all the bounding rectangles the same. For example, the minimum bounding rectangle with the largest area is used as the standard bounding rectangle for all matching suspected cyst areas for further division. Among them, the pixel values of the pixel points outside the suspected cyst area in the local window are defined as 0 for subsequent analysis.

[0037] In another embodiment of the present invention, the implementer can also define the size and shape of the local window by himself / herself, but the local window should not be too large to finely analyze the micro-cysts by dividing the local areas. Then, between the matching suspected cyst areas, the SIFT algorithm is used for feature matching to obtain the matching local window of each local area.

[0038] Also considering that the intra-abdominal pressure will change with breathing, which may cause the displacement of intra-abdominal tissues such as the ovaries, the movement of the local area should have a strong correlation with the patient's breathing. When the correlation is weak, it indicates that the local area is more likely to be the corresponding area of the ectopic cyst because the ectopic cyst may cause changes in the stiffness or elasticity of the ovarian tissue, thus affecting its response to the breathing movement and changing the movement characteristics of this area. Therefore, in the embodiments of the present invention, the movement feature sequence of each local area will be further obtained according to the change characteristics of each local area between adjacent frames of images to prepare for subsequent analysis of the correlation between the movement of the local area and breathing.

[0039] Preferably, in an embodiment of the present invention, considering that between adjacent frame images, the change of pixel points in the locally matching regions can be analyzed based on the optical flow method to evaluate their motion information; at the same time, the feature points or corner points in the local region usually may represent important tissues, so the displacement of the corner points can also help to reflect the motion information of the local region; based on this, the method for obtaining the motion feature sequence includes: Between each frame image and the adjacent previous frame image, based on the optical flow method, obtain the motion vector of each pixel point in each local region, and obtain the relative displacement between the matching corner points between the local regions at the same position between adjacent frames; According to the motion vectors of all pixel points in each local region and the relative displacements between all matching corner points, obtain the motion feature coefficient of each local region between adjacent frames; in the laparoscopic image sequence, sort the motion feature coefficients of the local regions at the same position between adjacent frames in chronological order to construct a motion feature sequence.

[0040] As an example, first, between each frame image and the adjacent previous frame image, between the matching local regions, based on the optical flow method, obtain the motion vector of each pixel point in the local region of each frame image, and then take the sum of the motion vectors of all pixel points as the motion vector of the local region in each frame image; Further, perform corner detection on each local region to obtain all corner points, and construct a coordinate system with the lower left corner pixel point of the local region as the origin to obtain the coordinates of each corner point; then, between the matching local regions, match the corner points, and thus based on the coordinates of the corner points, obtain the relative displacement between the matching corner points between the local regions at the same position between adjacent frames, that is, the matching local regions, and then take the average value of the relative displacements between all matching corner points as the relative displacement of the local window in each frame image; Finally, take the product of the motion vector and the relative displacement of each local region as the motion feature coefficient of the local window in each frame image; further construct a motion feature sequence; it should be noted that for the convenience of subsequent operations, the implementer also needs to take the modulus length of the motion vector and then participate in the product operation to obtain the motion feature coefficient; in the laparoscopic image sequence, since the first frame image has no previous adjacent frame, it cannot be compared for changes, so the length of the motion feature sequence is 1 less than the length of the laparoscopic image sequence.

[0041] It should be noted that the application of the optical flow method, taking the modulus length of the motion vector, corner detection and matching, and obtaining the relative displacement are all well-known technical means and will not be elaborated.

[0042] Step S3: Obtain a respiratory feature sequence based on the fluctuation of the respiratory signal, and obtain the cyst confidence of each local region according to the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change features of each local region and the other local regions in the laparoscopic image sequence; in each frame of the image, adjust the segmentation parameters of the preset segmentation model according to the cyst confidence of each local region and the boundary correction coefficient of its suspected cyst region, and re-label the cyst region.

[0043] Considering that during laparoscopic operation, the patient's breathing may have certain fluctuations, and breathing has different stages or phases, including inhalation, exhalation, and mid-respiration, it is difficult to measure the breathing fluctuation characteristics; however, the breathing signals in the same breathing phase can help evaluate its fluctuation characteristics.

[0044] Based on this, in a preferred embodiment of the present invention, first determine the breathing phase of each signal point in the respiratory signal, and further sort and fit the breathing signal points in the same phase to obtain the breathing signal corresponding to the phase, so as to evaluate its fluctuation characteristics; therefore, the method for obtaining the respiratory feature sequence includes: Label the breathing phase of each signal point in the respiratory signal based on a preset phase segmentation algorithm, and sort the signal points in the same breathing phase in chronological order to construct a breathing phase signal corresponding to the breathing phase; In each breathing phase signal, obtain the respiratory feature coefficient at each moment according to the signal fluctuation intensity in the preset historical time domain of the signal point at each moment; sort all the respiratory feature coefficients in chronological order to construct a respiratory feature sequence.

[0045] As an example, the preset phase segmentation algorithm is a respiratory signal phase segmentation algorithm based on a temporal convolutional network, which can be used to automatically identify and divide each breathing phase in a single breathing cycle, including inhalation, exhalation, and mid-respiration; then, taking any phase as an example, sort the signal points in chronological order to fit the breathing phase signal corresponding to the breathing phase; the implementer can also adopt other phase annotation models based on deep learning algorithms, which are all well-known prior arts in the art and will not be elaborated here.

[0046] Among them, in a preferred embodiment of the present invention, considering that in each breathing phase signal, a longer time series may smooth out the fluctuation characteristics, the fluctuation characteristics are analyzed in the local historical time domain of each signal point; and considering that variance can reflect the intensity of signal point fluctuations, and at the same time, the closer the amplitude levels of signal points are, the smaller the fluctuations are, the fluctuation degree can be measured by the occurrence frequency of signal points of each amplitude; based on this, the method for obtaining the respiratory feature coefficient includes: In each respiratory phase signal, within the preset historical time domain of the signal points at each moment, the variance of the signal points is used as the first fluctuation parameter, and the second fluctuation parameter is obtained by comprehensively obtaining the negative correlation mapping result of the occurrence frequency of the signal points of each amplitude; the first fluctuation parameter and the second fluctuation parameter are fused to obtain the respiratory characteristic coefficient.

[0047] As an example, first, in each respiratory phase signal, taking the signal points at each moment as the end points, a preset number of historical signal points, such as 9, are obtained in the reverse time sequence direction to construct a preset historical time domain, so that the first fluctuation parameter of the signal segment within the preset historical time domain can be obtained; then, the amplitudes of the signal points in the signal segment within the preset historical time domain are counted, and the occurrence frequency of the signal points of each amplitude is mapped into the exponential function exp(-x) with the natural constant e as the base. The less the occurrence frequency, the larger the exponential function value, indicating that the amplitude change of the signal segment is more frequent and the fluctuation possibility is greater. Furthermore, the exponential function values corresponding to the occurrence frequencies of the signal points of all amplitudes are multiplied to obtain the second fluctuation parameter; finally, the first fluctuation parameter and the second fluctuation parameter are multiplied and fused to obtain the respiratory characteristic coefficient.

[0048] It should be noted that the implementer can also define the preset number by himself / herself. When the number of historical signal points in the reverse time sequence direction of the signal points is less than the preset number, the insufficient number is regarded as a preset historical time domain.

[0049] Considering that the movement of tissues such as the ovaries in the abdominal cavity is not only affected by respiration but also by the traction of the instrument, the traction of the instrument will cause detail loss in the local area, and the corresponding edges of the tissues inside are more likely to be deformed or overlapped, etc. Therefore, the greater the influence of the instrument traction on the local area, the greater the necessity of correcting it; Based on this, after obtaining the respiratory signal feature sequence in the embodiment of the present invention, the cyst confidence of each local area can be further obtained according to the change correlation between the respiratory feature sequence and the movement feature sequence, and the deviation between the corresponding change features of each local area and the other local areas in the laparoscopic image sequence; the cyst confidence not only reflects the possibility that the local area is a cyst area, but also reflects the possibility of boundary blur caused by the traction of the instrument, preparing for subsequent adjustment of the segmentation accuracy.

[0050] Preferably, in an embodiment of the present invention, considering that the Pearson correlation coefficient can reflect the change correlation between sequences, and considering that the more the corresponding movement feature sequence of the target area deviates from the other local areas adjacent to it, the greater the possibility that it is deformed relatively due to the traction of the instrument, and the greater the possibility that the local area is a cyst or tissue deformation occurs; therefore, the method for obtaining the cyst confidence includes: Please refer to Figure 2, which shows a flowchart of a method for obtaining cyst confidence provided by an embodiment of the present invention, specifically including: Step S201, in the laparoscopic image sequence, taking any local area as the target area, and taking all local areas adjacent to the target area as the reference areas; according to the sequence difference between the motion feature sequence of the target area and the motion feature sequence of each reference area, obtain the deformation coefficient of the target area.

[0051] As an example, first determine a target area, and then use the Manhattan distance to measure the sequence difference between the motion feature sequence of the target area and the motion feature sequence of each reference area, and then take the mean value of the Manhattan distance as the deformation coefficient of the target area; the greater the sequence difference, the more deviated the motion features of the target area are relative to the other reference areas, and the greater the deformation coefficient of the target area.

[0052] Step S202, take the absolute value of the Pearson correlation coefficient between the respiration feature sequence and the motion feature sequence and perform a negative correlation mapping to obtain the respiration weight.

[0053] As an example, take the absolute value of the Pearson correlation coefficient between the respiration feature sequence and the motion feature sequence, add a preset non-zero positive parameter such as 0.001, and then perform a reciprocal operation for negative correlation mapping to obtain the respiration weight. The closer the Pearson correlation coefficient is to 0, the smaller the correlation between the respiration feature and the motion feature, and the greater the possibility that the local area is the corresponding area of the ectopic cyst.

[0054] Step S203, weight the deformation coefficient with the respiration weight, and take the weighted result as the cyst confidence of the target area.

[0055] As an example, multiply the respiration weight by the deformation coefficient to obtain the cyst confidence of the target area; by changing the target area, the cyst confidence of each local area can be obtained.

[0056] It should be noted that both the Manhattan distance and the Pearson correlation coefficient are well-known technologies and will not be elaborated here.

[0057] After obtaining the cyst confidence of each local area, it is possible to further combine the boundary correction coefficient of its suspected cyst area, so as to re-label the cyst area in each frame of image, improve the model segmentation accuracy, and accurately segment the real cyst area.

[0058] Preferably, in an embodiment of the present invention, considering that the larger the boundary correction coefficient and the cyst confidence degree are, the greater the degree of boundary blur in the local area is, and the more likely the model is to mis-segment more pseudo-boundaries. The U-Net segmentation model usually outputs the cyst probability map of each frame of image, and then determines the suspected cyst area according to the segmentation threshold. Therefore, by adjusting the segmentation threshold of the preset segmentation model, the segmentation accuracy can be appropriately improved and the pseudo-boundaries can be excluded. Therefore, the method for re-labeling the cyst area includes: In each frame of image, according to the cyst confidence degree of each local area and the boundary correction coefficient of its suspected cyst area, obtain the segmentation adjustment weight of each local area; obtain the initial segmentation threshold when the preset segmentation model segments the suspected cyst area; use the segmentation adjustment weight to weight the initial segmentation threshold, take the weighted result as the segmentation threshold of the corresponding local area, and perform segmentation to obtain the cyst sub-area in the local area; integrate the cyst sub-areas in all local areas within the suspected tumor area to obtain the cyst area in each frame of image and label it.

[0059] Among them, in a preferred embodiment of the present invention, considering that the larger the cyst confidence degree of the local area is, in order to more finely divide the cyst area, the segmentation threshold of the local area should be increased; also considering that the boundary correction coefficient of its suspected cyst area also provides a certain reference for improving the segmentation accuracy. When the boundary correction coefficient is larger, it means that the possibility of pseudo-boundaries existing in the suspected cyst area is greater. Therefore, the method for obtaining the segmentation adjustment weight includes: In each frame of image, take the product of the cyst confidence degree of each local area and the boundary correction coefficient of its suspected cyst area as the adjustment amplitude index, and take the sum of the constant 1 and the adjustment amplitude index as the segmentation adjustment weight.

[0060] After obtaining the segmentation adjustment weight of each local area, the initial segmentation threshold when segmenting the suspected cyst area in each frame of image can be further obtained based on the U-Net model. Then, multiply the segmentation adjustment weight by the initial segmentation threshold to obtain the segmentation threshold of this local area in each frame of image. Furthermore, the cyst sub-area in the local area can be segmented based on the segmentation threshold, and the cyst sub-areas of each local area within the suspected cyst area can be further obtained. Then, perform region connectivity on all cyst sub-areas, so that each connected area is used as a cyst area, and the truly segmented real cyst area in each suspected cyst area in each frame of image can be obtained. Furthermore, the cyst area can be labeled to assist relevant medical staff in observing and identifying.

[0061] It should be noted that obtaining the initial segmentation threshold of the U-Net model and performing region connectivity are already existing technical means and will not be elaborated here.

[0062] In summary, the present invention first obtains a laparoscopic image sequence and a respiratory signal, and based on a preset segmentation model, segments the suspected cyst regions in each frame of the sequence; then obtains the boundary correction coefficient of each suspected cyst region in each frame of the image; further divides each suspected cyst region in each frame of the image into several local regions, and obtains the cyst confidence of each local region; and then in each frame of the image, adjusts the segmentation parameters of the preset segmentation model and relabels the cyst regions. The present invention first analyzes the possibility of the existence of a pseudo-boundary in the suspected cyst region divided by the preset segmentation model through the position and structural characteristics of the pseudo-boundary, further combines the instrument traction and respiratory influence to finely analyze the suspected cyst region, evaluates the possibility of each local region being a micro-cyst, and finally comprehensively adjusts the segmentation accuracy of the segmentation model, so as to accurately segment and label the real cyst region, improving the recognition effect of ovarian endometriosis cyst lesions.

[0063] It should be noted that the above sequence of the embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0064] Each embodiment in this specification is described in a progressive manner, and the same or similar parts among the embodiments can be referred to each other. The key points of each embodiment are the differences from other embodiments.

Claims

1. A laparoscopic real-time identification method for ovarian endometriotic cysts, characterized in that, The method includes: For each observed part of the patient, obtain a laparoscopic image sequence, synchronously obtain a respiratory signal, segment the suspected cyst regions in each frame of the sequence based on a preset segmentation model, and obtain all the edges within each suspected cyst region; In each frame of the image, according to the position distribution of each edge within each suspected cyst region and its position change in the laparoscopic image sequence, obtain the boundary correction coefficient of each suspected cyst region; divide each suspected cyst region in each frame of the image into several local regions, and according to the change characteristics of each local region between adjacent frames of the image, obtain the motion feature sequence of each local region; Obtain the respiratory feature sequence according to the fluctuation of the respiratory signal, and according to the change correlation between the respiratory feature sequence and the motion feature sequence, as well as the deviation between the corresponding change characteristics of each local region and the other local regions in the laparoscopic image sequence, obtain the cyst confidence of each local region; in each frame of the image, according to the cyst confidence of each local region and the boundary correction coefficient of the suspected cyst region to which it belongs, adjust the segmentation parameters of the preset segmentation model and re-label the cyst region.

2. The laparoscopic real-time recognition method of ovarian endometriotic cysts according to claim 1, characterized in that, The method for obtaining the boundary correction coefficient includes: According to the distance of each pixel point on each edge relative to the center of the suspected cyst region to which it belongs, obtain the boundary coefficient of each edge; match the edges in the abdominal cavity image sequence according to the position of the geometric center of each edge, and between all adjacent frames, according to the position difference between the geometric centers of the matched edges, obtain the structural stability coefficient of each edge; Fuse the boundary coefficient and the structural stability coefficient to obtain the boundary attribute of each edge; comprehensively obtain the boundary correction coefficient of each suspected cyst region based on the boundary attributes of all the edges within the cyst region.

3. A laparoscopic real-time identification method for ovarian endometriotic cysts according to claim 2, characterized in that, The method for obtaining the boundary attribute includes: Take the product of the boundary coefficient and the structural stability coefficient as the boundary attribute.

4. A laparoscopic real-time identification method for ovarian endometriotic cysts according to claim 1, characterized in that, The method for obtaining the motion feature sequence includes: Between each frame of the image and the previous adjacent frame, based on the optical flow method, obtain the motion vector of each pixel point within each local region, and obtain the relative displacement between the matching corner points between the local regions at the same position between adjacent frames; According to the motion vectors of all the pixel points within each local region and the relative displacement between all the matching corner points, obtain the motion feature coefficient of each local region between adjacent frames; in the laparoscopic image sequence, sort the motion feature coefficients of the local regions at the same position between adjacent frames in chronological order to construct a motion feature sequence.

5. A laparoscopic real-time recognition method for ovarian endometriotic cysts according to claim 1, characterized in that, The method for obtaining the respiratory feature sequence includes: Based on a preset phase segmentation algorithm, label the respiratory phase of each signal point in the respiratory signal, and sort the signal points under the same respiratory phase in chronological order to construct a respiratory phase signal corresponding to the respiratory phase; In each respiratory phase signal, according to the signal fluctuation intensity within the preset historical time domain of the signal point at each moment, obtain the respiratory feature coefficient at each moment; sort all the respiratory feature coefficients in chronological order to construct a respiratory feature sequence.

6. The laparoscopic real-time recognition method of ovarian endometriotic cysts according to claim 5, characterized in that, The method for obtaining the respiratory feature coefficient includes: In each respiratory phase signal, within the preset historical time domain of signal points at each moment, the variance of the signal points is used as the first fluctuation parameter, and the second fluctuation parameter is obtained by comprehensively acquiring the negative correlation mapping results of the occurrence frequencies of signal points of each amplitude; the first fluctuation parameter and the second fluctuation parameter are fused to obtain the respiratory feature coefficient.

7. A laparoscopic real-time identification method for ovarian endometriotic cysts according to claim 1, characterized in that, The method for obtaining the cyst confidence level includes: In the laparoscopic image sequence, taking any local area as the target area, and taking all local areas adjacent to the target area as the reference areas; obtaining the deformation coefficient of the target area according to the sequence difference between the motion feature sequence of the target area and the motion feature sequences of each reference area; taking the absolute value of the Pearson correlation coefficient between the respiratory feature sequence and the motion feature sequence and performing negative correlation mapping to obtain the respiratory weight; weighting the deformation coefficient with the respiratory weight, and taking the weighted result as the cyst confidence level of the target area.

8. A laparoscopic real-time recognition method for ovarian endometriotic cysts according to claim 1, characterized in that, The method for re-labeling the cyst area includes: In each frame of image, according to the cyst confidence level of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs, obtaining the segmentation adjustment weight value of each local area; obtaining the initial segmentation threshold when the preset segmentation model segments the suspected cyst area; weighting the initial segmentation threshold with the segmentation adjustment weight value, taking the weighted result as the segmentation threshold of the corresponding local area, and performing segmentation to obtain the cyst sub-area in the local area; integrating the cyst sub-areas in all local areas within the suspected tumor area to obtain the cyst area in each frame of image and labeling it.

9. A laparoscopic real-time recognition method for ovarian endometriotic cysts according to claim 8, characterized in that, The method for obtaining the segmentation adjustment weight value includes: In each frame of image, taking the product of the cyst confidence level of each local area and the boundary correction coefficient of the suspected cyst area to which it belongs as the adjustment amplitude index, and taking the sum value of the constant 1 and the adjustment amplitude index as the segmentation adjustment weight value.

10. A laparoscopic real-time recognition method for ovarian endometriotic cysts according to claim 1, characterized in that, The preset segmentation model is a U-Net model.

Images