Method and apparatus for image segmentation of weak contrast gas / water regions in pore seepage

Abstract

The present invention discloses a method and device for image segmentation of weak contrast gas / water regions in pore seepage, which is applicable to the common pore gas / liquid two-phase seepage problems in fields such as CO2 geological storage, oil and gas exploitation, and underground pollutant treatment. The method disclosed by the present invention adopts a "dual-route parallel" strategy of unsupervised learning with the purpose of contour filling and in the "partitioning and grading, stage activation" manner, and strong supervised learning of deep convolutional network-semantic segmentation. A defect repair algorithm of breakpoint search-judgment-connection is adopted to overcome the contour defect problem obtained by unsupervised learning. Finally, a contour-constrained cooperative game method is adopted in the filled closed contour and the semantic segmentation region to segment a gas region with high accuracy. The present invention reduces the accuracy requirement and dependence on deep learning-semantic segmentation, avoids the time-consuming and laborious problem of a large number of sample annotations, and realizes high-precision segmentation of colorless and weak contrast gas / water regions.

Description

Technical Field

[0001] This invention belongs to the technical fields of CO2 geological sequestration and oil and gas field development. More specifically, it relates to a method for image segmentation of weakly contrasting gas / water regions in pore seepage, and also relates to a device for image segmentation of weakly contrasting gas / water regions in pore seepage. Background Art

[0002] Gas / water multiphase seepage in porous media is a common problem in CO2 geological sequestration and oil and gas field development. The occurrence and contact relationship of gas / water in pore-confined spaces are key information in multiphase seepage transmission. Therefore, the characterization of the occurrence characteristics of gas / water in pores is the basis for establishing a multiphase seepage mathematical model.

[0003] In pore seepage, gases (such as CO2, CH4, N2, or air, etc.) are colorless fluid media, and only the contour can be distinguished when in contact with water. In optical images, traditional image segmentation methods, such as threshold segmentation, region growing, clustering, and edge detection, etc., are extremely difficult to find high-resolution features, which poses a great challenge to image segmentation. The problem of image segmentation of the gas / water interface is difficult to solve, which directly leads to difficulties in the subsequent characterization and information mining of gas / water seepage characteristics (phase interface contact lines, occurrence characteristics, saturation, etc.) in porous media.

[0004] In similar prior art solutions, the seepage detection of gas / water two-phase in pores often relies on CT technical means. Patent CN114897923A provides a threshold segmentation method for natural gas / hydrate in CT images. Due to the obvious gray-scale difference between hydrate and water under CT monitoring, a fine threshold segmentation method using histogram adjustment can achieve the segmentation of hydrate / water. In images of gas / water two-phase with no color and weak contrast, no patent disclosure has been found for solving the problem of image segmentation of the gas or water phase.

[0005] In recent years, the rapidly developing convolutional network - deep learning technology can achieve the leap from pixel information to semantic information, and can solve the problem that traditional segmentation methods are difficult to characterize the weakly contrasting gas / water interface feature engineering. It provides a new perspective for solving the difficult segmentation problem of traditional segmentation methods by means of a deep convolutional network.

[0006] This invention uses a microfluidic chip displacement experiment device to monitor the dynamics of pore gas / water multiphase seepage in real time with optical images. Facing the problem of difficult image segmentation caused by the colorless and weakly contrasting gas / water interface, drawing on the path of constructing semantic information of the gas / water interface features of the deep convolutional network, a method and device for image segmentation of weakly contrasting gas / water regions in pore seepage are proposed to solve the problems existing in the prior art. Summary of the Invention

[0007] The object of the present invention is to provide a method for image segmentation of weak contrast gas / water regions in pore seepage. The method is easy to implement and operate simply. It not only realizes high-precision segmentation of gas, but also greatly liberates human labor. This method is a key step in characterizing the phenomenological characteristics of pore seepage, ensuring the promotion of the work of mining and characterizing pore multiphase seepage information, and facilitating the understanding and establishment of subsequent key dynamic mechanisms.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for image segmentation of weak contrast gas / water regions in pore seepage, which adopts the idea of parallel cooperation and game of two branches of unsupervised learning contour filling and deep convolutional network semantic segmentation of strong supervised learning, including:

[0010] Image enhancement method, contour recognition method, alternative method for contour defect repair, deep convolutional network semantic segmentation method, cooperation game method;

[0011] The image enhancement method includes:

[0012] Step E001: Perform MSR (Multi-scale Retinex) enhancement correction on the imported picture, aiming to reduce the interference caused by uneven illumination when recorded by the experimental CCD camera;

[0013] Step E002: Then use bilateral filtering to filter the image to remove the noise interference caused by the difference of adjacent pixels in the image. The module of the image enhancement method can be implemented by using the self-written MSR function and imbilatfilt function in Matlab respectively;

[0014] The contour recognition method includes:

[0015] Step D001: Filter the enhanced image by using variance filtering to enhance the contour information of particle boundaries and gas / water interfaces. The variance filtering can be implemented by using the stdfilt function in Matlab;

[0016] Step D002: Suppress the noise of the image filtered by variance filtering, and uniformly assign the value of 0 to the area where the pixel is less than the threshold range by using the Relu layer method in the deep convolutional network. At this time, the processed image is hereinafter referred to as the denoised variance map;

[0017] Step D003: In the denoised variance map after the noise suppression processing by the Relu layer, the pixels in the non-0 area still have large numerical differences. To reduce the segmentation deviation caused by the wide pixel domain, the denoised variance map is assigned by using the method of regional assignment;

[0018] Step D004: The top-bottom boundary and the pore-particle boundary need to be confirmed through the binary image of particles and pores. The binary image of particles / pores is obtained by contour detection and filling of the microfluidic image completely filled with water or gas.

[0019] Step D005: Edge detection of the particle / pore binary image can confirm the regions of the top-bottom boundary and the particle boundary. The maximum edge detected is the top-bottom boundary, and all other closed boundaries are particle boundaries. This step can be achieved through bwboundaries in Matlab.

[0020] Step D006: Perform a convolution operation on the 1st variance map in Step D005. The convolution kernel is a 3-order all-ones matrix, which enhances the pixel features of the boundary by statistically analyzing the edge points in the adjacent space. The larger the pixel value after convolution, the higher the probability that the position is a boundary.

[0021] Step D007: Filter the 2nd variance map in Step D006 through the Relu layer, which is equivalent to threshold segmentation, that is, select a threshold. Values greater than the threshold are segmented to form a binary image. In this embodiment, the threshold in the 2nd variance map is 0, that is, all non-zero pixel regions are contour regions.

[0022] Step D008: Perform medial axis thinning on the contour region segmented in Step D007, which can be achieved by using the bwmorph function in Matlab.

[0023] Preferably, the alternative methods for repairing contour defects include:

[0024] Step R001: Search for breakpoints in the thinned contour region in Step D008. The strategy for breakpoint search is to perform a 3-order neighborhood convolution operation on the binary region of the thinned contour. If the pixel value after convolution is 2 and it is on the thinned contour, it is determined as a breakpoint.

[0025] Step R002: Traverse the searched breakpoints in the manner described in Figure 2 and respectively count the shortest path length P and the straight-line distance D between the traversed breakpoints j and k.

[0026] Step R003: Judge the shortest path length P and the straight-line distance D in Step R002. If the ratio of P / D is higher than a set threshold η, then the breakpoints j and k are connected.

[0027] Step R004: Traverse the next breakpoint. If not all breakpoints have been traversed, repeat Step R002. If all breakpoints have been traversed, output the repaired thinned contour region.

[0028] Preferably, the deep convolutional network semantic segmentation method includes:

[0029] Step S001: Label the image sets of gas / water two-phase seepage at different times for no less than 3 chips, mainly calibrate the gas invasion areas, and produce the corresponding Ground Truth set, which is jointly produced by using Photoshop and the imageLabeler application in Matlab in this embodiment;

[0030] Step S002: Train the produced Ground Truth set and the corresponding pictures with a deep convolutional neural network. You can choose the deeplabv3+, U-Net or RefineNet network for deep learning;

[0031] Step S003: Use the trained convolutional neural network to perform semantic segmentation on the image to be segmented. The image to be segmented has the same size and preprocessing process as the trained image, that is, the trained network is the enhanced image processed in step E001, and the image to be segmented is also the enhanced image processed in step E001, and the output is the semantic segmentation area.

[0032] Preferably, the cooperative game method includes:

[0033] Step C001: Fill the closed contour of the refined contour output in step R004 and number the filled area, which is implemented by using the bwlabel function in Matlab;

[0034] Step C002: Traverse the filled areas. Assume that the currently traversed filled area is area i, and obtain the intersection area N between area i and the semantic segmentation area obtained in step S003 through the union operation;

[0035] Step C003: Judge the area ratio between the intersection area N and area i. If the area ratio is greater than the set threshold ε, fill area i and confirm that the filled area i is the gas invasion area. Otherwise, do not fill and consider that this area is not the gas invasion area;

[0036] Step C004: Judge whether area i has been completely traversed. If not, repeat step C002. If it has been completely traversed, output the gas invasion area α of the cooperative game;

[0037] Step C005: Perform the union operation on the semantic segmentation area obtained in step S003 and the high-probability area obtained in step D006 to obtain the common area M, which is the confirmed gas invasion area;

[0038] Step C006: Perform the OR operation on the gas invasion area α of the cooperative game obtained in step C004 and the common area M obtained in step C005, and then the final gas segmentation area can be obtained.

[0039] Preferably, in the contour recognition method, the denoised variance map is assigned values in a region-by-region assignment manner, which can be specifically divided into three assignment regions, namely the top and bottom boundary regions of the microfluidic chip, the particle boundary region, and the high-probability region.

[0040] Preferably, in the contour recognition method;

[0041] In step D005, this step can be implemented by bwboundaries in Matlab. The top and bottom boundaries and particle boundaries in the denoised variance map of step D002 are respectively assigned values, and the variance map after assignment is called the 1st variance map;

[0042] In step D006, the variance map after convolution operation is converted into a grayscale map with a value of 255. The frequency statistics of the grayscale values from 0 to 255 are performed, and the cumulative probability percentage is calculated in the way of cumulative addition from small to large of the grayscale values. The grayscale value corresponding to a certain cumulative probability value is set as the segmentation threshold. The intercepted cumulative probability value in this embodiment is 98%. The binary map after segmentation is the high-probability region, and higher pixel values are assigned to the high-probability region in the 1st variance map in step D005 than the top and bottom boundaries and the particle boundary. The variance map after assignment becomes the 2nd variance map.

[0043] The present invention also provides an apparatus for image segmentation of a weak contrast gas / water region in pore seepage, including the method for image segmentation of a weak contrast gas / water region in pore seepage; it also includes an image enhancement module E (image Enhancement module), a contour recognition module D (detection module), an alternative module R (boundary Remediation module) for contour defect repair, a deep convolutional network semantic segmentation module S (Semantic Segment module), and a cooperative game module C (Cooperative game module).

[0044] The image enhancement module is respectively connected to the pictures taken by the experimental device, the deep convolution network semantic segmentation module, and the contour recognition module. The cooperative game module is respectively connected to the deep convolution network semantic segmentation module, the alternative module for contour defect repair, and the output segmented image. The contour recognition module and the deep convolution network semantic segmentation module are connected after the image enhancement module, and the image processed by the image enhancement module is input into the contour recognition module and the deep convolution network semantic segmentation module respectively. Among them, the contour recognition module and the deep convolution network semantic segmentation module are in a parallel relationship. The contour recognition module uses an unsupervised learning method to realize the contour recognition and filling of the air / water interface; the deep convolution network semantic segmentation module trains the annotated image through a deep convolution network, and uses the trained convolution network to perform semantic segmentation on the image. The contour filling area and semantic segmentation results obtained by the parallel route are important input information for the cooperative game module executed last.

[0045] The present invention adopts a parallel approach of unsupervised learning and deep learning, which significantly reduces the requirements and reliance on deep learning semantic segmentation and avoids the time-consuming and laborious problem of labeling a large number of samples. The method requires little preliminary preparation and is highly operable, while achieving high-precision segmentation of colorless and weak-contrast gas / water areas in pores.

[0046] Technical effects and advantages of the present invention: Compared with the prior art, the method and device for image segmentation of weak-contrast gas / water regions in pore seepage proposed by the present invention have the following advantages:

[0047] In the face of the problem of difficult segmentation of weak-contrast gas / water images in microfluidic chips, the present invention uses an algorithm flow that integrates a deep learning convolutional neural network supported by a small number of samples with contour recognition by unsupervised learning to achieve fine segmentation of gas areas in pores. The present invention not only solves the problem of limited effect and inaccurate contour boundaries of relying solely on deep learning semantic segmentation, but also overcomes the difficulty that deep learning convolutional networks require a large amount of learning sample annotation and a large workload;

[0048] (1) In the face of the problem of difficult segmentation of gas / water areas with no color and weak contrast, high-precision segmentation of gas is achieved. On the one hand, the problem of inaccurate edge segmentation in deep learning semantic segmentation is overcome. Compared with the MIoU accuracy of 0.67-0.87 of semantic segmentation, the MIoU of this method can reach 0.93-0.97, achieving a significant improvement in segmentation accuracy.

[0049] (2). This method greatly reduces the requirements and dependence on deep learning. Relying solely on deep learning methods, to ensure extremely high or even demanding MIoU segmentation accuracy, often requires a suitable network structure and a large number of high-precision labeled samples. The preparation of labeled samples is time-consuming and laborious, which greatly hinders the industrial application of strong supervised learning. By reducing the requirements and dependence on deep learning, in such difficult-to-segment images, only a few to a dozen training labeled samples are needed to achieve high-precision segmentation.

[0050] (3). Automatically realizes the work of finding and repairing contour defects. Through the operation of the algorithm, the contour defect repair and filling work can be achieved without manual intervention, greatly liberating the manpower.

[0051] Other features and advantages of the present invention will be described in the following specification, and partly will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the structure pointed out in the specification and the drawings. Brief Description of the Drawings

[0052] Figure 1 It is a flowchart of the method for image segmentation of weak contrast gas / water regions in pore seepage;

[0053] Figure 2 It is a schematic diagram of the assignment of the top and bottom boundaries in the embodiment of the invention;

[0054] Figure 3 It is a schematic diagram of the assignment of the particle boundary in the embodiment of the invention;

[0055] Figure 4 It is a schematic diagram of the assignment of the high-probability region in the embodiment of the invention;

[0056] Figure 5 It is a schematic diagram of the breakpoint connection diagram in the embodiment of the invention;

[0057] Figure 6 It is a schematic diagram of the enhanced image and semantic segmentation region in the embodiment of the invention;

[0058] Figure 7 It is a schematic diagram of the final gas segmentation region in the embodiment of the invention;

[0059] Figure 8 It is a flowchart of the microfluidic experiment in the prior art of the present invention.

[0060] Figure 9 It is a connection relationship diagram between each module in the present invention. Detailed Embodiment

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0062] Embodiment 1:

[0063] According to Figure 1-9 it can be known that: a device for image segmentation of weak contrast gas / water regions in pore seepage includes 5 major modules, namely: ① image enhancement module E; ② contour recognition module D; ③ alternative module R for contour defect repair; ④ deep convolutional network semantic segmentation module S; ⑤ cooperative game module C.

[0064] Figure 1 and Figure 9 shows the connection relationship between the 5 major modules. Using Figure 8 the experimental device and process shown to record the experimental images of pore multiphase seepage, and importing the captured experimental images into the image enhancement module E to enhance the image information. Among them, the image enhancement module E belongs to the preparatory work before image segmentation, aiming to remove the noise generated by uneven illumination during image shooting to enhance the segmentation information of gas / water interface pixels.

[0065] The described image enhancement module E is respectively connected to the pictures taken by the experimental device, the deep convolutional network semantic segmentation module S, and the contour recognition module D. The cooperative game module C is respectively connected to the deep convolutional network semantic segmentation module S, the alternative module R for contour defect repair, and the output segmentation image. The contour recognition module D and the deep convolutional network semantic segmentation module S are connected after the image enhancement module E, and the image processed by the image enhancement module E is respectively input into the contour recognition module D and the deep convolutional network semantic segmentation module S. Among them, the contour recognition module D and the deep convolutional network semantic segmentation module S are in a parallel relationship. The contour recognition module D realizes the contour recognition and filling of the gas / water interface in an unsupervised learning manner; the deep convolutional network semantic segmentation module S trains the annotated images through the deep convolutional network and performs semantic segmentation on the images using the trained convolutional network. The contour filling area and the semantic segmentation result obtained from the parallel route are important input information for the finally executed cooperative game module C.

[0066] The alternative module R for contour defect repair is between the contour recognition module D and the cooperative game module C. This module detects and intelligently repairs the contour boundary defects output by the contour recognition module D, avoiding the problem of over-large filled connected regions caused by similar but unclosed contour boundaries. The alternative module R for contour defect repair ensures the filling accuracy of the closed contour and also guarantees the accuracy of subsequent gas / water interface segmentation.

[0067] Finally, the cooperative game module C is executed. It summarizes the output results of the contour recognition module D and the deep convolutional network semantic segmentation module S, and outputs the final segmented binary image by means of cooperative game.

[0068] Example 2:

[0069] As Figure 2 shown, a method for image segmentation of weak contrast gas / water regions in pore seepage. Its steps are as follows: This method adopts the idea of two parallel branches of unsupervised learning contour filling and strong supervised learning deep convolutional network semantic segmentation and then cooperative game, including five major modules, namely: ① Image enhancement module; ② Contour recognition module; ③ Alternative module for contour defect repair; ④ Deep convolutional network semantic segmentation module; ⑤ Cooperative game module.

[0070] ① Image enhancement module E (image Enhancement module)

[0071] Step E001: Perform MSR (Multi-scale Retinex) enhancement correction on the imported picture, aiming to reduce the interference caused by uneven illumination when recorded by the experimental CCD camera.

[0072] Step E002: Then filter the image using bilateral filtering to remove the noise interference caused by the difference in adjacent pixels of the image. This module can be implemented in Matlab using the self-written MSR function and the imbilatfilt function respectively.

[0073] ② Contour recognition module D (Detection module)

[0074] Step D001: Filter the enhanced image using variance filtering to enhance the contour information of particle boundaries and gas / water interfaces. Variance filtering can be implemented using the stdfilt function in Matlab.

[0075] Step D002: Suppress the noise of the image filtered by variance filtering. Use the Relu layer method in the deep convolutional network to uniformly assign the regions where the pixels are less than the threshold range to 0. At this time, the processed image is hereinafter referred to as the denoised variance map.

[0076] Step D003: In the denoised variance map after the Relu layer noise suppression process, the pixels in the non-zero regions still have large numerical differences. To reduce the segmentation deviation caused by the wide pixel range, the denoised variance map is assigned values in a region-by-region manner. Specifically, it can be divided into three major assignment regions, namely ① the top and bottom boundary regions of the microfluidic chip; ② the particle boundary regions; ③ the high-probability regions.

[0077] Step D004: The top and bottom boundaries and the pore-particle boundaries need to be confirmed through the binary image of particles and pores (hereinafter referred to as the particle / pore binary map). The particle / pore binary map is obtained by using the method of contour detection and then filling in the microfluidic image completely filled with water or gas (abbreviated as the single-phase saturated image).

[0078] Step D005: Edge detection of the particle / pore binary map can confirm the regions of the top and bottom boundaries and the particle boundaries. Among them, the largest edge detected is the top and bottom boundary ( Figure 3 ), and all other closed boundaries are particle boundaries ( Figure 4 ). This step can be implemented by bwboundaries in Matlab. The top and bottom boundaries and the particle boundaries in the denoised variance map of Step D002 are assigned values respectively. The variance map after assignment is called the 1st variance map.

[0079] Step D006: Perform a convolution operation on the 1st variance map in Step D005. The convolution kernel is a 3-order all-1 matrix. This convolution kernel strengthens the pixel features of the boundary by statistically analyzing the edge points in the adjacent space. The larger the pixel value after convolution, the higher the probability that the position is a boundary. The variance map after convolution is converted into a grayscale map with a value of 255. The frequency of the grayscale values from 0 to 255 is statistically analyzed, and the cumulative probability percentage is calculated in the way of accumulating from small to large according to the grayscale values. Set the grayscale value corresponding to a certain cumulative probability value as the segmentation threshold. The intercepted cumulative probability value in this embodiment is 98%, and the binary map after segmentation is the high-probability region ( Figure 5 ). A higher pixel value is assigned to the high-probability region in the 1st variance map in Step D005 than the top and bottom boundaries and the particle boundaries. The variance map after assignment becomes the 2nd variance map.

[0080] Step D007: Perform Relu layer filtering on the 2nd variance map in Step D006. Its effect is equivalent to threshold segmentation, that is, select a threshold, and a binary map is formed when it is greater than this threshold. The threshold in the 2nd variance map in this embodiment is 0, that is, all non-zero pixel regions are contour regions.

[0081] Step D008: Perform medial axis thinning on the contour region segmented in Step D007, which can be achieved by using the bwmorph function in Matlab.

[0082] ③Alternative module R for contour defect repair (boundary Remediation module)

[0083] Step R001: Find breakpoints in the refined contour area in step D008. The strategy for breakpoint search is to perform a 3 - order neighborhood convolution operation on the binary area of the refined contour. A pixel with a value of 2 after convolution and on the refined contour is determined as a breakpoint.

[0084] Step R002: Traverse the found breakpoints in the way described by Figure 2 . Respectively, count the shortest path length P and the straight - line distance D between the traversed breakpoints j and k. The straight - line distance D can be calculated through the coordinates of breakpoints j and k. The shortest path length P is searched for the shortest path between two points on the refined contour area obtained in step D008, and is implemented using a self - written function PoreMinDist. Functions such as bwdistgeodesic and imregionalmin are called in this self - written function.

[0085] Step R003: Judge the shortest path length P and the straight - line distance D in step R002. If the ratio of P / D is higher than a set threshold η, then breakpoints j and k are connected ( Figure 6 ). In this embodiment, the threshold η is selected as 30.

[0086] Step R004: Traverse the next breakpoint. If not all breakpoints have been traversed, repeat step R002. If all breakpoints have been traversed, output the repaired refined contour area.

[0087] ④Deep convolutional network semantic segmentation module S (Semantic Segment module)

[0088] Step S001: Label an image set of gas / water two - phase seepage at different times in no less than 3 chips. Mainly calibrate the gas intrusion area to produce a corresponding Ground Truth (background) set. In this embodiment, it is jointly produced using the imageLabeler applications in Photoshop and Matlab.

[0089] Step S002: Train the generated Ground Truth set and the corresponding images using a deep convolutional network. You can choose deeplabv3+, U-Net, or RefineNet for deep learning. In this embodiment, deeplabv3+ is used for deep learning, and the backbone network is selected as xception. The imported images can be the original images without enhancement or the enhanced images processed in step E001. Since the deep convolutional network requires a large amount of video memory, training is performed in the form of reduced image size. In this embodiment, the image size used for training is 393×537.

[0090] Step S003: Use the trained convolutional neural network to perform semantic segmentation on the image to be segmented. The image to be segmented has the same size and preprocessing process as the training image, that is, the training network is the enhanced image processed in step E001, and the image to be segmented is also the enhanced image processed in step E001. The output is the semantic segmentation region ( Figure 7 , with the image output in step E001 on the left and the semantic segmentation region (black region) of the deeplabv3+ network on the right).

[0091] ⑤ Cooperative game module C(Cooperative game module)

[0092] Step C001: Fill the closed contour of the refined contour output in step R004 and number the filled regions, which is implemented using the bwlabel function in Matlab.

[0093] Step C002: Traverse the filled regions. Assume that the currently traversed filled region is region i, and obtain the intersection region N between region i and the semantic segmentation region obtained in step S003 through the union operation.

[0094] Step C003: Judge the area ratio of the intersection region N to region i. If the area ratio is greater than the set threshold ε, fill region i and confirm that the filled region i is the gas intrusion region. Otherwise, do not fill and consider that this region is not the gas intrusion region.

[0095] Step C004: Judge whether region i has been completely traversed. If not, repeat step C002. If it has been completely traversed, output the gas intrusion region α of the cooperative game.

[0096] Step C005: Perform the union operation on the semantic segmentation region obtained in step S003 and the high-probability region obtained in step D006 to obtain the common region M, which is the confirmed gas intrusion region.

[0097] Step C006: Perform an OR operation on the gas intrusion region α of the cooperative game obtained in Step C004 and the common region M obtained in Step C005, and the final gas segmentation region can be obtained. Figure 8 , the left is the image output by Step E001, and the right is the final output gas segmentation region (black region).

[0098] (1). In the face of the difficult segmentation problem of weak contrast gas / water regions, an "ambulatory on two legs" strategic route of parallel unsupervised learning (contour filling) and strong supervised learning (deep convolutional network) is adopted. This route breaks through the dilemma of relying too much on deep learning-semantic segmentation for "walking on one leg". On the one hand, the accuracy of deep learning-semantic segmentation does not need to reach a very high requirement (the mean intersection over union MIoU should be at least above 0.94. Note: an MIoU of 0.94 is almost demanding and difficult to achieve. In this patent, an MIoU of only 0.60 - 0.70 is required). On the other hand, due to the reduced accuracy requirement, the number of training images does not need to be large, which greatly avoids the time-consuming and laborious problem of preparing labeled training samples.

[0099] (2). The cooperative game mechanism under the parallel route of unsupervised learning (contour filling) and strong supervised learning (deep convolutional network), that is, with contour filling as the constraint, determines whether the contour region is filled through the game result of convolutional network-semantic segmentation, and realizes the segmentation and recognition of the gas intrusion region.

[0100] (3). In unsupervised learning (contour filling), the method of "partitioning and grading, suppressing noise in stages" is adopted to extract the contour region. In the variance pixel map under variance filtering, the pixel value differences are extremely large, and it is necessary to partition (top and bottom boundaries, particle regions, particle boundary regions, and high-probability regions) for hierarchical assignment. Before and after the assignment, the Relu activation or noise suppression method still needs to be used to ensure the signal-to-noise ratio of the contour information, so that all contour boundaries (gas / water interface, particle boundary, top and bottom boundaries of the chip) can be segmented by subsequent simple threshold segmentation.

[0101] (4). In the real-time recorded images, the gas / water interface may have many problems such as dissolution / phase pressure interference, resulting in blurred interfaces and incomplete contour filling in unsupervised learning (there may be just a small segment of line missing to close, and the small unclosed segment directly leads to an open boundary for the filled region, resulting in an extremely large filled region). Specifically, an alternative contour repair module is adopted, that is, the method of breakpoint search - judgment - connection repair, which can overcome the problem of an overly large filled region caused by a small unclosed segment of line.

[0102] In summary, although deep convolutional networks can quickly solve the problem of feature engineering representation that is difficult to achieve with traditional segmentation methods, the semantic segmentation efficiency of deep convolutional neural networks is still not as ideal as expected. There are two problems in semantic segmentation based on deep learning that limit the segmentation effect. On the one hand, there is a problem of inaccurate segmentation edges, resulting in unsatisfactory semantic segmentation effects. On the other hand, deep convolutional networks with strong supervised learning require a large number of labeled samples. The samples painstakingly labeled are likely to be mixed with labeling noise, posing challenges to semantic segmentation with demanding segmentation requirements.

[0103] Facing the problem of difficult segmentation of gas / water images with weak contrast in microfluidic chips, the present invention uses an algorithm process that combines a deep learning convolutional neural network supported by few samples with unsupervised contour recognition to achieve fine segmentation of the gas region in the pores. The present invention not only solves the problems of limited semantic segmentation effect and inaccurate contour boundaries relying solely on deep learning, but also overcomes the difficulty of large workload in labeling a large number of learning samples required by deep learning convolutional networks.

[0104] Currently in the field of deep learning, the most cutting-edge multi-modal research (starting in 2019, hot research issues from 2020 to 2022) and the unsupervised clustering of point cloud segmentation (Fast Euclidean FEC clustering) newly published this year provide potential new perspectives for segmentation. In short, the development potential in the field of deep learning is extremely large. So far, in the face of the problem of insufficient contour accuracy, no models and methods that can be applied on a large scale in the industrial application field with excellent effects have emerged.

[0105] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0106] Please note: For the above English to Chinese translation, such as: Chinese, the following English is in parentheses (Matlab). If it cannot be translated into Chinese, the source is indicated.

Claims

1. A method for image segmentation of weak contrast gas / water regions in pore seepage, characterized in that, This method adopts the idea of parallel operation and cooperative game of two branches: unsupervised learning-based contour filling and strongly supervised learning-based deep convolutional network semantic segmentation, including: Image enhancement method, contour recognition method, alternative methods for contour defect repair, deep convolutional network semantic segmentation method, cooperative game method; The image enhancement method includes: Step E001: Perform MSR Multi-scale Retinex enhancement correction on the imported picture to reduce the interference caused by uneven illumination during the recording of the experimental CCD camera; Step E002: Then use bilateral filtering to filter the image to remove the noise interference caused by the difference in adjacent pixels of the image; The contour recognition method includes: Step D001: Filter the enhanced image using variance filtering to enhance the contour information of particle boundaries and gas / water interfaces; Step D002: Suppress the noise of the image filtered by variance filtering. Use the Relu layer method in the deep convolutional network to uniformly assign the value of 0 to the area where the pixel is less than the threshold range. The processed image is hereinafter referred to as the denoised variance map; Step D003: In the denoised variance map after the noise suppression process of the Relu layer, the pixels in the non-zero area still have numerical differences. To reduce the segmentation deviation caused by the pixel wide domain, the denoised variance map is assigned values in a region-by-region manner; Step D004: The top and bottom boundaries and the pore particle boundaries are confirmed through the binary image of particles and pores. The binary map of particles / pores is obtained by using the method of contour detection and filling on the microfluidic image completely filled with water or gas; Step D005: Perform edge detection on the particle / pore binary map to confirm the regions of the top and bottom boundaries and the particle boundaries. The largest edge detected by the edge detection is the top and bottom boundary, and all other closed boundaries are particle boundaries; Step D006: Perform convolution operation on the 1st variance map in Step D005. The convolution kernel is a 3-order all-1 matrix. This convolution kernel strengthens the pixel features of the boundary by statistically analyzing the edge points in the adjacent space. The convolved pixel represents the probability that this position is the boundary; Step D007: Perform Relu layer filtering on the 2nd variance map in Step D006. Select a threshold. If it is greater than this threshold, a binary map is formed by segmentation. The threshold of the 2nd variance is 0, and all non-zero pixel regions are contour regions; Step D008: Perform medial axis thinning on the contour region segmented in Step D007.

2. A method for image segmentation of weak contrast gas / water regions in pore seepage, characterized in that: The alternative methods for contour defect repair include: Step R001: Search for breakpoints in the thinned contour region in Step D008. The strategy for breakpoint search is to perform a 3-order neighborhood convolution operation on the binary region of the thinned contour. If the pixel value after convolution is 2 and it is also on the thinned contour, it is determined as a breakpoint; Step R002: Traverse the searched breakpoints in the above-mentioned manner, and respectively count the shortest path length P and the straight-line distance D between the traversed breakpoints j and k; Step R003: Judge the shortest path length P and the straight-line distance D in Step R002. If the ratio of P / D is higher than a set threshold η, then the breakpoints j and k are connected; Step R004: Traverse the next breakpoint. If not all breakpoints have been traversed, repeat Step R002. If all breakpoints have been traversed, output the refined contour region after patching.

3. A method for image segmentation of weak contrast gas / water regions in pore seepage, according to claim 2, characterized in that: The deep convolutional network semantic segmentation method includes: Step S001: Annotate an image set of gas / water two-phase seepage at different times in no less than 3 chips, calibrate the gas intrusion area, and produce a corresponding Ground Truth set. Step S002: Train the produced Ground Truth set and the corresponding pictures with a deep convolutional network, and select a deeplabv3+, U-Net, or RefineNet network for deep learning. Step S003: Use the trained convolutional neural network to perform semantic segmentation on the image to be segmented. The image to be segmented has the same size and preprocessing process as the training image. The trained network is the enhanced image processed in Step E001, and the image to be segmented is also the enhanced image processed in Step E001. The output is the semantic segmentation region.

4. A method for image segmentation of a weak contrast gas / water region in pore seepage, characterized in that: The cooperative game method includes: Step C001: Fill the closed contour of the refined contour output in Step R004 and number the filled area. Step C002: Traverse the filled area. Assume that the currently traversed filled area is area i, and obtain the intersection area N between area i and the semantic segmentation region obtained in Step S003 through the union operation. Step C003: Judge the area ratio of the intersection area N to area i. If the area ratio is greater than the set threshold ε, fill area i and confirm that the filled area i is the gas intrusion area. Otherwise, do not fill and consider that this area is not the gas intrusion area. Step C004: Judge whether all of area i has been traversed. If not, repeat Step C002. If all have been traversed, output the gas intrusion area α of the cooperative game. Step C005: Perform a union operation on the semantic segmentation region obtained in Step S003 and the high-probability region obtained in Step D006 to obtain the common region M, which is the confirmed gas intrusion area. Step C006: Perform an OR operation on the gas intrusion area α of the cooperative game obtained in Step C004 and the common region M obtained in Step C005 to obtain the final gas segmentation region.

5. The method for image segmentation of weak-contrast gas / water regions in pore seepage according to claim 1, characterized in that: In the contour recognition method, the denoised variance map is assigned values in a region-by-region assignment manner, which is specifically divided into three major assignment regions, namely the top and bottom boundary regions of the microfluidic chip, the particle boundary regions, and the high-probability regions.

6. The method for image segmentation of weak-contrast gas / water regions in pore seepage according to claim 1, characterized in that: In the contour recognition method; In Step D005, assign values to the top and bottom boundaries and particle boundaries in the denoised variance map of Step D002. The variance map after assignment is called the 1st variance map. In step D006, the variance map after convolution operation is converted into a grayscale map with a value of 255. The frequency of grayscale values from 0 to 255 is counted, and the cumulative probability percentage is calculated in the way of cumulative addition from small to large of the grayscale values. The grayscale value corresponding to a certain cumulative probability value is set as the segmentation threshold. The binary map after segmentation is the high-probability region. A pixel value higher than the top and bottom boundaries and the particle boundaries is assigned to the high-probability region in the 1st variance map in step D005. The variance map after assignment becomes the 2nd variance map.

7. An apparatus for image segmentation of a weak contrast gas / water region in pore seepage, characterized in that, The device executes the method for image segmentation of the weak contrast gas / water region in pore seepage of any one of claims 1-4; it further includes an image enhancement module E, a contour recognition module D, an alternative module R for contour defect repair, a deep convolutional network semantic segmentation module S, and a cooperative game module C.

8. An apparatus for image segmentation of a weak contrast gas / water region in pore seepage, characterized in that, The described image enhancement module E is respectively connected to the picture taken by the experimental device, the deep convolutional network semantic segmentation module S, and the contour recognition module D. The cooperative game module C is respectively connected to the deep convolutional network semantic segmentation module S, the alternative module R for contour defect repair, and the output segmentation image.

9. An apparatus for image segmentation of a weak contrast gas / water region in pore seepage, characterized in that, The described contour recognition module D and the deep convolutional network semantic segmentation module S are connected after the image enhancement module E, wherein the contour recognition module D and the deep convolutional network semantic segmentation module S are in parallel.

10. An apparatus for image segmentation of a weak contrast gas / water region in pore seepage, characterized in that, The described alternative module R for contour defect repair is between the contour recognition module D and the cooperative game module C.

Images