Process pipeline and instrument flow image processing method and system

Abstract

The invention provides a process pipeline and instrument flow image processing method and system, and relates to the technical field of image enhancement. The method is applied to a process pipeline and instrument flow image processing system, and comprises the following steps: acquiring a training image; preprocessing the first process pipeline and instrument flow image and the second process pipeline and instrument flow image respectively to obtain a first target image and a second target image, and taking the first target image and the second target image as a pipeline data set; training a preset basic pipeline cutting image processing model based on the pipeline data set until a preset training completion condition is reached, and determining the trained basic pipeline cutting image processing model as a standard pipeline cutting image processing model; according to the method, firstly, non-target element removal processing is carried out on process pipeline and instrument flow images through a preset non-pipeline target recognition model, then definition enhancement is carried out on the processed images through a trained standard pipeline cutting image processing model, and the readability of the images and the subsequent image analysis efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a method and system for processing process pipeline and instrument flow images. Background Art

[0002] In modern industrial processes, process piping and instrumentation diagrams (P&IDs) are key design tools used to represent process equipment, pipelines, control systems and their interconnected relationships. However, high-resolution process piping and instrumentation diagrams often result in high computational costs, while low-resolution process piping and instrumentation diagrams often have poor image quality and complex image elements, making it difficult to identify and analyze these key information during subsequent use, affecting the readability of the image and the efficiency of subsequent image analysis. Summary of the invention

[0003] The object of the present invention is to provide a method and system for processing process pipeline and instrument flow images. First, non-target elements of process pipeline and instrument flow images are removed through a preset non-pipeline target recognition model, and then the processed images are enhanced in clarity through a trained standard pipeline cutting image processing model, thereby improving the readability of the image and the efficiency of subsequent image analysis.

[0004] In a first aspect, the present invention provides a process pipeline and instrument flow image processing method, which is applied to a process pipeline and instrument flow image processing system, and the method comprises:

[0005] Acquire training images; wherein the training images include a plurality of first process pipeline and instrument flow images and a plurality of corresponding second process pipeline and instrument flow images; the definition of the second process pipeline and instrument flow images is N times the definition of the first process pipeline and instrument flow images; N is a positive integer greater than 1;

[0006] Preprocessing the first process pipeline and instrument flow image to obtain a first target image; preprocessing the second process pipeline and instrument flow image to obtain a second target image; using the first target image and the second target image as pipeline data sets; wherein the preprocessing includes performing non-target element removal processing and image cutting processing based on a pre-trained non-pipeline target recognition model;

[0007] Based on the pipeline data set, a preset basic pipeline cutting image processing model is trained until a preset training completion condition is reached, and the trained basic pipeline cutting image processing model is determined as a standard pipeline cutting image processing model; wherein the basic pipeline cutting image processing model includes a plurality of multi-scale feature processing modules and a plurality of dual-branch fusion modules;

[0008] The image to be processed is input into the standard pipeline cutting image processing model to obtain a clear image corresponding to the image to be processed.

[0009] In some preferred embodiments of the present invention, the multi-scale feature processing module processes the image by the following steps:

[0010] The first image I input into the multi-scale feature processing module 1o4 The high-dimensional coordinate information C of the first image is obtained by mapping it to a high-dimensional space through random Fourier feature coding. 1o4H ; Wherein, the first image is obtained by downsampling the first target image by A times; A is a positive integer greater than 1;

[0011] Input the first image into the preset encoder to obtain the first feature F corresponding to the first image F ;

[0012] Based on the high-dimensional coordinate information C of the first image 1o4 The first feature F corresponding to the first image F Perform self-attention operation to obtain the second feature F 1o4A ;

[0013] Based on the second feature F 1o4A and the first image I 1o4 Determine the second image I 1o4u ;

[0014] For the second image I 1o4u Perform upsampling operation to obtain the third image I 1o4u2 ;

[0015] Get the fourth image I input into the multi-scale feature processing module 1o2 The two-dimensional space coordinates C 1o2 ; Wherein, the fourth image is obtained by downsampling the first target image by B times; B is a positive integer greater than A;

[0016] The fourth image I 1o2 The two-dimensional space coordinates C 1o2 Mapping to high-dimensional space through hash function, the high-dimensional coordinate information C of the fourth image is obtained 1o2H ;

[0017] Through the third image I 1o4u2 and the fourth image I 1o2 Determine the third feature C 1o2M ;

[0018] The third feature C 1o2M Connect with the high-dimensional coordinate information to obtain the fourth feature C 1o2M1 ;

[0019] For the fourth feature C1o2M1 Perform depth convolution to obtain the fifth image I 1o2u ;

[0020] For the fifth image I 1o2i Perform an upsampling operation to obtain the sixth image I 1o2u2 ;

[0021] Take the third image I 1o4u2 and the sixth image I 1o2u2 as the output images of the multi-scale feature processing module.

[0022] In some preferred embodiments of the present invention, the dual-branch fusion module processes images through the following steps:

[0023] Based on the input fifth feature I TB1 and the sixth feature I TB2 Through a feature fusion operation, obtain the seventh feature F TBm ; where the feature fusion operation includes at least one of the following: element-wise multiplication, element-wise addition, and function activation operation;

[0024] Based on the fifth feature I TB1 , the sixth feature I TB2 and the seventh feature F TBm After performing a feature stacking operation, then perform a channel dimension concatenation operation to obtain the eighth feature F TBO ; where the feature stacking operation includes at least one of the following: element-wise multiplication, element-wise addition, and depth convolution operation;

[0025] Take the eighth feature as the output of the dual-branch fusion module.

[0026] In some preferred embodiments of the present invention, the step of obtaining the seventh feature by performing a feature fusion operation based on the input fifth feature and sixth feature includes:

[0027] Perform a convolution operation on the fifth feature, and then perform element-wise multiplication with the sixth feature to obtain the ninth feature;

[0028] Perform a convolution operation on the sixth feature, and then perform element-wise multiplication with the fifth feature to obtain the tenth feature;

[0029] Add the ninth feature and the tenth feature element-wise and then perform a function activation operation to obtain the seventh feature F TBm .

[0030] In some preferred embodiments of the present invention, the step of performing a feature stacking operation based on the fifth feature I TB1 , the sixth feature I TB2 and the seventh feature F TBm and then performing a channel dimension concatenation operation to obtain the eighth feature includes:

[0031] The seventh feature is multiplied element by element by the fifth feature, and then the result of the depthwise convolution is added element by element to the sixth feature to obtain the eleventh feature;

[0032] The seventh feature is multiplied element by element by the sixth feature, and then the result of the depthwise convolution is added element by element to the fifth feature to obtain the twelfth feature;

[0033] The eleventh feature and the twelfth feature are concatenated in the channel dimension to obtain the eighth feature F TBO .

[0034] In some preferred embodiments of the present invention, the basic pipeline cutting image processing model further includes: a first branch, a second branch and a third branch; wherein the first branch includes a first double-branch fusion module; the second branch includes a second double-branch fusion module; the basic pipeline cutting image processing model processes the image by the following steps:

[0035] The input first target image I is downsampled to obtain the first image I 1o4 and the fourth image I 1o2 ;

[0036] The first image I 1o4 Input the multi-scale feature processing module to obtain the third image I 1o4u2 ;

[0037] The fourth image I 1o2 Input the multi-scale feature processing module to obtain the sixth image I 1o2u2 ;

[0038] The third image I 1o4u2 With the fourth image I 1o2 Perform element-by-element addition to obtain the seventh image I 2 ;

[0039] The sixth image I 1o2u2 Add the first target image I element by element to obtain the eighth image I 1 ;

[0040] In the first branch, based on the eighth image I 1 Determine the thirteenth feature F 1Fus and the ninth image I 1+ ;

[0041] In the second branch, based on the seventh image I 2 Determine the fourteenth feature F 2Fus and the tenth image I 2o ;

[0042] In the third branch, based on the first image I 1o4、Fourteenth Feature F 2Fus Determine the eleventh image I 3O ;

[0043] The tenth image I 2o and the eleventh image I 3O After element-by-element addition, upsampling is performed to obtain the twelfth image I 2+u ;

[0044] The twelfth image I 2+u and the ninth image I 1+ After element-by-element addition, a convolution operation is performed to obtain the output image.

[0045] In some preferred embodiments of the present invention, based on the eighth image I 1 Determine the thirteenth feature F 1Fus and the ninth image I 1+ The steps include:

[0046] Based on the first image I 1 Perform convolution operation to obtain the fifteenth feature F 1 ;

[0047] The fifteenth feature F 1 Input the preset first encoder to obtain the sixteenth feature F 1E ;

[0048] The sixteenth feature F 1E Input the preset first decoder to obtain the seventeenth feature F 1ED ;

[0049] The seventeenth feature F 1ED Input to the preset second encoder to obtain the eighteenth feature F 1EDE ;

[0050] The eighteenth feature F 1EDE Input the preset second decoder to obtain the nineteenth feature F 1EDED ;

[0051] The sixteenth feature F 1E and the eighteenth feature F 1EDE Input the pre-set first two-branch fusion module to obtain the thirteenth feature F 1Fus ;

[0052] The nineteenth feature F 1EDED Perform convolution operation to obtain the ninth image I 1+ .

[0053] In some preferred embodiments of the present invention, based on the seventh image I 2 Determine the fourteenth feature F 2Fus and the tenth image I2o The steps include:

[0054] The seventh image I 2 Perform convolution operation to obtain the twentieth feature F 2 ;

[0055] The twentieth feature F 2 Input to the preset third encoder to obtain the twenty-first feature F 2E ;

[0056] The twenty-first feature F 2E With the thirteenth characteristic F 1Fus Add element by element to get the 22nd feature F 2E+ ;

[0057] The twenty-second feature F 2E+ Input the preset third decoder to obtain the twenty-third feature F 2E+D ;

[0058] The twenty-second feature F 2E+ and the eighteenth feature F 1EDE Input the pre-set second dual-branch fusion module to obtain the fourteenth feature F 2Fus ;

[0059] The twenty-third feature F 2E+D Perform convolution operation to obtain the tenth image I 2o .

[0060] In some preferred embodiments of the present invention, based on the first image I 1o4 、Fourteenth Feature F 2Fus Determine the eleventh image I 3O The steps include:

[0061] The first image I 1o4 Perform convolution operation to obtain the 24th feature F 3 ;

[0062] The twenty-fourth feature F 3 Input the preset fourth encoder to obtain the twenty-fifth feature F 3E ;

[0063] The twenty-fifth feature F 3E With the fourteenth characteristic F 2Fus Add element by element to get the 26th feature F 3E+ ;

[0064] The twenty-sixth feature F 3E+ Input the preset fourth decoder and get the twenty-seventh feature F 3E+D ;

[0065] The twenty-seventh feature F 3E+D Perform convolution operation to obtain the eleventh image I 3O .

[0066] In a second aspect, the present invention provides a process piping and instrument flow image processing system, which is used to execute any one of the process piping and instrument flow image processing methods provided in the first aspect.

[0067] The present invention brings the following beneficial effects:

[0068] The present invention provides a process pipeline and instrument flow image processing method and system, the method is applied to the process pipeline and instrument flow image processing system, the image processing comprises: obtaining training images; wherein the training images comprise a plurality of first process pipeline and instrument flow images and a plurality of corresponding second process pipeline and instrument flow images; the definition of the second process pipeline and instrument flow images is N times the definition of the first process pipeline and instrument flow images; N is a positive integer greater than 1; preprocessing the first process pipeline and instrument flow images to obtain a first target image; preprocessing the second process pipeline and instrument flow images to obtain a second target image; using the first target image and the second target image as pipeline data sets; wherein the preprocessing comprises performing a preprocessing based on a pre-trained non-pipeline target recognition model; The method comprises the following steps: performing non-target element removal processing and image cutting processing; training a preset basic pipeline cutting image processing model based on a pipeline data set until a preset training completion condition is reached, and determining the trained basic pipeline cutting image processing model as a standard pipeline cutting image processing model; wherein the basic pipeline cutting image processing model comprises a plurality of multi-scale feature processing modules and a plurality of dual-branch fusion modules; inputting the image to be processed into the standard pipeline cutting image processing model to obtain a clear image corresponding to the image to be processed; firstly performing non-target element removal processing on the process pipeline and instrument flow images through a preset non-pipeline target recognition model, and then performing clarity enhancement on the processed images through the trained standard pipeline cutting image processing model, thereby improving the readability of the images and the efficiency of subsequent image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 A flow chart of a process pipeline and instrument flow image processing method provided by an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of the structure of a dual-branch fusion module provided in an embodiment of the present invention;

[0072] Figure 3 A schematic diagram of the structure of a multi-scale feature processing module provided by an embodiment of the present invention;

[0073] Figure 4 A standard definition image of a process pipeline and instrument flow provided by an embodiment of the present invention;

[0074] Figure 5 The embodiment of the present invention provides Figure 4 Corresponding ultra-clear images of process piping and instrument flow. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0076] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0078] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0079] In addition, the terms "horizontal", "vertical", "overhanging" and the like do not mean that the components are required to be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0080] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0081] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0082] Embodiment 1

[0083] The embodiment of the present invention provides a process pipeline and instrument flow image processing method, which is applied to a process pipeline and instrument flow image processing system, see Figure 1 The embodiment of the present invention shown in the figure provides a flow chart of a process pipeline and instrument flow image processing method, the method comprising:

[0084] Step S102, obtaining training images; wherein the training images include a plurality of first process piping and instrument flow images and a corresponding plurality of second process piping and instrument flow images; the clarity of the second process piping and instrument flow images is N times the clarity of the first process piping and instrument flow images; and N is a positive integer greater than 1.

[0085] Specifically, the image processing method provided in the embodiment of the present invention aims to process an unclear standard image to obtain an ultra-clear image, so a plurality of standard images and clear images corresponding to each other need to be prepared in the training process; the first process pipeline and instrument flow image is an image of standard definition, and the second process pipeline and instrument flow image is an ultra-clear image corresponding to the first process pipeline and instrument flow image. In some preferred embodiments of the present invention, N can be 3, that is, the clarity of the ultra-clear image is 3 times that of the standard image.

[0086] Step S104, preprocessing the first process pipeline and instrument flow image to obtain a first target image; preprocessing the second process pipeline and instrument flow image to obtain a second target image; using the first target image and the second target image as pipeline data sets; wherein the preprocessing includes performing non-target element removal processing and image cutting processing based on a pre-trained non-pipeline target recognition model.

[0087] Specifically, 30 process pipeline and instrument flow charts in standard image format are collected as 30 first process pipeline and instrument flow images, each of which has a resolution of 3317×2584. All non-pipeline targets in the 30 first process pipeline and instrument flow images are labeled. In some preferred embodiments of the present invention, labelimg (an open source image annotation software) can be used to uniformly annotate them as a type of target, including all non-pipeline elements such as valves, instruments, text, title boxes, etc., to train a non-pipeline target recognition model. In some preferred embodiments of the present invention, a YOLOv5 model can be used as a pre-trained model, trained for 300 rounds, and a single-round training batch size of 16 is input. All non-pipeline targets are identified using a non-pipeline target recognition model, and in multiple process pipeline and instrument flow images, all non-pipeline target areas are filled with pure white areas for deletion, and 30 process pipeline and instrument flow images containing only pipelines are obtained after deletion. Then, 30 triple super-resolution versions of the same process piping and instrumentation flow chart are collected as 30 second ultra-clear process piping and instrumentation flow images, each of which has a resolution of 9951×7752, and the height and width of the second ultra-clear process piping and instrumentation flow image are three times the height and width of the process piping and instrumentation flow image. All non-pipeline targets in the 30 triple super-clear process piping and instrumentation flow images are labeled. In some preferred embodiments of the present invention, labelimg labeling software is used to uniformly label them as one type of target, including Including all non-pipeline elements such as valves, instruments, text, title boxes, etc., train a triple ultra-clear non-pipeline target recognition model. In some preferred embodiments of the present invention, the YOLOv5 model is used as a pre-training model, trained for 300 rounds, and the input single-round training batch size is 16. The triple ultra-clear non-pipeline target recognition model is used to identify all non-pipeline targets, and in 30 triple ultra-clear process pipelines and instrument flow images, all non-pipeline target areas are filled with pure white areas for deletion. After deletion, 30 triple ultra-clear process pipelines and instrument flow images containing only pipelines are obtained. The above-mentioned non-pipeline target recognition model and the triple ultra-clear non-pipeline target recognition model can be the same model or different models. If it is the same model, it only needs to be trained once.

[0088] The non-pipeline target recognition model provided by the embodiment of the present invention removes non-pipeline information such as equipment symbols and instrument identification in the image through image preprocessing technology, simplifies the low-resolution process flow chart into a skeleton diagram of the pipeline, and after removing the messy elements, the noise and unnecessary information in the image are greatly reduced, making subsequent processing easier. At the same time, the geometric structure information of the pipeline is retained, providing cleaner input data for the subsequent deep learning enhancement algorithm.

[0089] The first process pipeline and instrument flow image with non-target elements removed is cut using a square sliding window with a step size of 512 to obtain 1260 standard pipeline cutting images, namely the first target image, and the resolution of each standard pipeline cutting image is 512×512.

[0090] The second process pipeline and instrument flow images with non-target elements removed are cut to obtain 1260 triple ultra-high-definition pipeline cutting images, and multiple triple ultra-high-definition pipeline cutting images are subjected to a three-fold resolution reduction operation to obtain multiple ultra-high-definition pipeline cutting images, i.e., the second target images; multiple standard pipeline cutting images and multiple ultra-high-definition pipeline cutting images are matched one by one and combined into 1260 training pairs to form a pipeline data set.

[0091] Step S106, training a pre-set basic pipeline cutting image processing model based on the pipeline data set until a pre-set training completion condition is reached, and determining the trained basic pipeline cutting image processing model as the standard pipeline cutting image processing model; wherein the basic pipeline cutting image processing model includes multiple multi-scale feature processing modules and multiple dual-branch fusion modules.

[0092] Specifically, in some preferred embodiments of the present invention, the basic pipeline cutting image processing model is implemented based on the PyTorch framework, trained using the NVIDIA GeForce RTX 4090 GPU with 32GB of video memory, using the Adam optimizer, the learning rate is initialized to 0.0001, and an exponential annealing scheme is used. When the final learning rate is gradually reduced to 0.000001, the model training is considered complete, and the trained model is determined as the standard pipeline cutting image processing model.

[0093] In some preferred embodiments of the present invention, 80% of the pipeline data set is used to train the basic pipeline cutting image processing model, 300 rounds of single training are performed, the input single-round training batch size is 16, and the peak signal-to-noise ratio, structural similarity index and mean square error are used as quantitative indicators. Five trainings are performed and the basic pipeline cutting image processing model with the best effect is selected as the standard pipeline cutting image processing model. After the training is completed, the standard pipeline cutting image processing model can input a single standard pipeline cutting image and output a single ultra-clear pipeline cutting image.

[0094] In some preferred embodiments of the present invention, the multi-scale feature processing module processes the image by the following steps: the first image I input to the multi-scale feature processing module 1o4 The high-dimensional coordinate information C of the first image is obtained by mapping it to a high-dimensional space through random Fourier feature coding. 1o4 ; Wherein, the first image is obtained by downsampling the first target image by A times; A is a positive integer greater than 1; the first image is input into the preset encoder to obtain the first feature F corresponding to the first image F ; Based on the high-dimensional coordinate information C of the first image 1o4 The first feature F corresponding to the first image F Perform self-attention operation to obtain the second feature I 1o4A Based on the second feature I 1o4A and the first image I 1o4 Determine the second image I 1o4u ; For the second image I 1o4u Perform upsampling operation to obtain the third image I 1o4u2 ; Get the fourth image I input into the multi-scale feature processing module 1o2 The two-dimensional space coordinates C 1o2 ; The fourth image is obtained by downsampling the first target image by B times; B is a positive integer greater than A; the fourth image I 1o2 The two-dimensional space coordinates C 1o2 Mapping to high-dimensional space through hash function, the high-dimensional coordinate information C of the fourth image is obtained 1o2H ; Through the third image I 1o4u2 and the fourth image I 1o2 Determine the third feature C 1o2M ; The third feature C 1o2M Connect with the high-dimensional coordinate information to obtain the fourth feature C 1o2M1 ; For the fourth feature C 1o2M1 Perform depth convolution to obtain the fifth image I 1o2u ; For the fifth image I 1o2u Perform upsampling operation to obtain the sixth image I 1o2u2 ; The third image I 1o4u2 and the sixth image I 1o2u2 Determine the output image of the multi-scale feature processing module.

[0095] Specifically, the first target image, that is, the standard pipeline cutting image I, is downsampled to obtain the first image I 1o4 and the fourth image I 1o2; Wherein, the first image is obtained by downsampling the first target image by A times; A is a positive integer greater than 1; the fourth image is obtained by downsampling the first target image by B times; B is a positive integer greater than A; in some preferred embodiments of the present invention, A is 2 and B is 4.

[0096] The multi-scale feature processing module includes random Fourier feature encoding, self-attention operation and deep convolution, etc. For the multi-scale feature processing module, given the input first image I 1o4 and the fourth image I 1o2 , where the first image Fourth Image and 3 represent the first image I 1o4 height, width and passages, and 3 represent the fourth image I 1o2 The height, width and channels of , H and W are two real values, see Figure 3 The structure diagram of a multi-scale feature processing module provided by an embodiment of the present invention is shown in FIG. 1o4 The pixel position of the first image obtains the two-dimensional space coordinate C 1o4 , Among them, the value 2 represents the horizontal and vertical coordinates, and the two-dimensional space coordinate C 1o4 Through random Fourier feature coding and mapping to high-dimensional space, high-dimensional coordinate information C is obtained 1o4H , Then image C 1o4 Input to the encoder to get the first feature F F , where the encoder consists of convolutional layers, pooling layers, activation functions, batch normalization, and fully connected layers. The high-dimensional coordinate information C 1o4H and feature F F Perform self-attention operation and first get the query value Q 1o4 , key value K 1o4 and the linear value V 1o4 , where Q 1o4 =W 1o4Q C 1o4H , K 1o4 =W 1o4K F F , V 1o4 =W 1o4V F F , where W 1o4Q , W 1o4K and W 1o4V There are three linear layers with dimensions d is a fixed dimension value. In some preferred embodiments of the present invention, d is set to 64. Then the second feature F is obtained. 1o4A ,

[0097] Softmax represents the Softmax activation function, K 1o4 T Represents the key value K 1o4 Then we get the second image I 1o4u , I 1o4u =W 1o4u F 1o4A +I 1o4 , W 1o4u Represents a linear layer, which transforms the image W 1o4u Perform upsampling operation to obtain the third image I 1o4u2 ,

[0098] Continue to see Figure 3 , through the fourth image I 1o2 The pixel position obtains the two-dimensional space coordinate C 1o2 , Among them, the value 2 represents the horizontal and vertical coordinates, and the two-dimensional space coordinate C 1o2 Mapping to high-dimensional space through hash function, the high-dimensional coordinate information C of the fourth image is obtained 1o2H , Then we get the third feature C 1o2M , C 1o2M =ReLU(Conv(I 1o2 +I 1o4u2 )), Conv represents 3×3 convolution, ReLU represents ReLU activation function, + represents element-by-element addition, The third feature C 1o2M Connect with the high-dimensional coordinate information in the channel dimension to obtain the fourth feature C 1o2M1 , Then we get the fifth image I 1o2u , I 1o2u =DWConv 5 (C 1o2M1 ), DWConv 5 represents a depthwise convolution with a convolution kernel size of 5, which transforms the fifth image I 1o2u Up-sampling is performed to obtain the sixth image I 1o2u2 , I 1o2u2 ∈R H×W×3 .

[0099] Finally, the third image I 1o4u2 and the sixth image I 1o2u2 It is the output of the multi-scale feature processing module.

[0100] The multi-scale feature processing module provided by the embodiment of the present invention first maps the pixel positions of the image to two-dimensional space coordinates, and projects these low-dimensional spatial information into a high-dimensional space through random Fourier feature encoding to obtain high-dimensional coordinate information, and then performs self-attention operations on the high-dimensional coordinate information and image features to focus on important image areas and enhance the ability to understand complex structures. Multiple convolutional layers are used to extract multi-scale features from the image, and each layer of convolution can capture features of different scales. Through multi-scale processing, the model can simultaneously obtain global and local feature information.

[0101] In some preferred embodiments of the present invention, the dual-branch fusion module processes the image by the following steps: based on the fifth feature I of the input TB1 and the sixth feature I TB2 Through feature fusion operation, the seventh feature F is obtained TBm ; The feature fusion operation includes at least one of the following: element-by-element multiplication, element-by-element addition, and function activation operation; Based on the fifth feature I TB1 、The sixth characteristic T TB2 and the seventh characteristic F TBm After the feature superposition operation, the channel dimension splicing operation is performed to obtain the eighth feature F TBO ; Wherein, the feature superposition operation includes at least one of the following: element-by-element multiplication, element-by-element addition and depth convolution operation; the eighth feature is used as the output of the dual-branch fusion module.

[0102] Further, in some preferred embodiments of the present invention, the step of obtaining the seventh feature through a feature fusion operation based on the fifth and sixth features of the input includes: performing a convolution operation on the fifth feature, and then performing element-by-element multiplication with the sixth feature to obtain a ninth feature; performing a convolution operation on the sixth feature, and then performing element-by-element multiplication with the fifth feature to obtain a tenth feature; and performing a function activation operation after adding the ninth feature and the tenth feature element-by-element to obtain the seventh feature F. TBm .

[0103] Further, in some preferred embodiments of the present invention, based on the fifth feature I TB1 、Sixth Feature I TB2 and the seventh characteristic F TBm After the feature superposition operation is performed, the channel dimension splicing operation is performed to obtain the eighth feature, and the steps include: performing element-by-element multiplication of the seventh feature and the fifth feature, and then performing element-by-element addition of the result of the depth convolution to the sixth feature to obtain the eleventh feature; performing element-by-element multiplication of the seventh feature and the sixth feature, and then performing element-by-element addition of the result of the depth convolution to the fifth feature to obtain the twelfth feature; performing the channel dimension splicing operation on the eleventh feature and the twelfth feature to obtain the eighth feature F. TBO .

[0104] For details, see Figure 2 The schematic diagram of the structure of a dual-branch fusion module provided by an embodiment of the present invention is shown, and the dual-branch fusion module includes 3×3 convolution, Sigmod activation function, element-by-element addition and element-by-element multiplication. Given the fifth feature I TB1 and the sixth feature I TB2 , H 1 , W 1 and C represents the fifth characteristic I TB1 and the sixth feature I TB2 Height, width and channel, fifth feature I TB1 and the sixth feature I TB2 The height, width and channel of can be the same or different. In this embodiment, only the same parameters are used to illustrate. First, the seventh feature F is calculated. TBm , C3 represents 3×3 convolution, Sig represents Sigmod activation function, represents element-by-element multiplication, + represents element-by-element addition, and then calculates the eighth feature F TBO , DWC 3 represents a depthwise convolution with a kernel size of 3, represents element-by-element multiplication, + represents element-by-element addition, Concat represents concatenation in the channel dimension, and feature F TBO is the output of the dual-branch fusion module. We only know that the eighth feature F TBO 、Fifth Feature I TB1 and the sixth feature I TB2 The dimensions remain consistent.

[0105] The dual-branch fusion module provided in the embodiment of the present invention processes different features of the input image respectively through two independent feature processing branches, and then combines the advantages of the two through an effective fusion mechanism. It can process complex image structures, retain the overall image framework while enhancing detail information, and realizes efficient feature fusion and information extraction through different convolution operations and element-by-element operations.

[0106] Furthermore, in some preferred embodiments of the present invention, the basic pipeline cutting image processing model further includes: a first branch, a second branch and a third branch; wherein the first branch includes a first double-branch fusion module; the second branch includes a second double-branch fusion module; the basic pipeline cutting image processing model processes the image by the following steps:

[0107] The input first target image I is downsampled to obtain the first image I 1o4 and the fourth image I 1o2 ; The first image I 1o4 Input the multi-scale feature processing module to obtain the third image I 1o4u2 ; The fourth image I 1o2 Input the multi-scale feature processing module to obtain the sixth image I 1o2u2 ; Add the third image and the fourth image element by element to obtain the seventh image I 2 ; Sixth image I 1o2u2 Add the standard pipeline cutting image I element by element to obtain the eighth image I 1 .

[0108] In the first branch, based on the eighth image I 1 Determine the thirteenth feature F 1Fus and the ninth image I 1+ .

[0109] Further, in some preferred embodiments of the present invention, based on the eighth image I 1 Determine the thirteenth feature F 1Fus and the ninth image I 1+ The steps include:

[0110] Based on the first image I 1 Perform convolution operation to obtain the fifteenth feature F 1 ; The fifteenth feature F 1 Input the preset first encoder to obtain the sixteenth feature F 1E ; The sixteenth feature F 1E Input the preset first decoder to obtain the seventeenth feature F 1ED ; The seventeenth feature F 1ED Input to the preset second encoder to obtain the eighteenth feature F 1EDE ; The eighteenth feature F 1EDE Input the preset second decoder to obtain the nineteenth feature F 1EDED ; The sixteenth feature F 1E and the eighteenth feature F 1EDE Input the pre-set first two-branch fusion module to obtain the thirteenth feature F 1Fus ; The nineteenth feature F 1EDED Perform convolution operation to obtain the ninth image I 1+ .

[0111] In the second branch, based on the seventh image I 2 Determine the fourteenth feature F 2Fus and the tenth image I 2o .

[0112] Further, in some preferred embodiments of the present invention, based on the seventh image I 2 Determine the fourteenth feature F 2Fus and the tenth image I 2o The steps include:

[0113] The seventh image I 2 Perform convolution operation to obtain the twentieth feature F 2 ; The twentieth feature F 2 Input to the preset third encoder to obtain the twenty-first feature F 2E ; The twenty-first feature F 2E With the thirteenth characteristic F 1Fus Add element by element to get the 22nd feature F 2E+ ; The twenty-second feature F 2E+ Input the preset third decoder to obtain the twenty-third feature F 2E+D ; The twenty-second feature F 2E+ and the eighteenth feature F 1EDE Input the pre-set second dual-branch fusion module to obtain the fourteenth feature F 2Fus ; The twenty-third feature F 2E+D Perform convolution operation to obtain the tenth image I 2o .

[0114] In the third branch, based on the first image I 1o4 、Fourteenth Feature F 2Fus Determine the eleventh image I 3O .

[0115] Further, in some preferred embodiments of the present invention, based on the first image I 1o4 、Fourteenth Feature F 2Fus Determine the eleventh image I 3O The steps include:

[0116] The first image I 1o4 Perform convolution operation to obtain the 24th feature F 3 ; The twenty-fourth feature F 3 Input the preset fourth encoder to obtain the twenty-fifth feature F 3E ; The twenty-fifth feature F 3E With the fourteenth characteristic F 2Fus Add element by element to get the 26th feature F 3E+ ; The twenty-sixth feature F 3E+ Input the preset fourth decoder and get the twenty-seventh feature F 3E+D ; The twenty-seventh feature F 3E+D Perform convolution operation to obtain the eleventh image I 3O .

[0117] The tenth image I 2o and the eleventh image I 3O After element-by-element addition, upsampling is performed to obtain the twelfth image I 2+u ; The twelfth image I 2+u and the ninth image I 1+ After element-by-element addition, a convolution operation is performed to obtain the output image.

[0118] Specifically, for the standard pipe cutting image processing model, the first target image I is input, I∈R H×W×3 , H, W and 3 represent the height, width and channel of the standard pipeline cutting image I. In some preferred embodiments of the present invention, H and W are both 512, and the standard pipeline cutting image I is downsampled by 2 times and 4 times to obtain the first image I 1o4 and the fourth image I 1o2 , and 3 represent the first image I 1o4 height, width and passages, and 3 represent the fourth image I 1o2 The height, width and channels of the first image I 1o4 and the fourth image I 1o2 Input to the multi-scale feature processing module to obtain the third image I 1o4u2 and the sixth image I 1o2u2 , The third image I 1o4u2 and the fourth image I 1o2 Perform element-by-element addition to obtain the seventh image I 2 , The sixth image I 1o2u2 The eighth image I is obtained by element-by-element addition of the first target image I 1 , I 1 ∈R H×W×3 , set up branch one, branch two and branch three.

[0119] In branch one, the eighth image I 1 Input to the first 3×3 convolution to get the fifteenth feature F 1 , H 1 , W 1 and C 1 Represents the fifteenth characteristic F 1 In some preferred embodiments of the present invention, H 1 and W 1 Take 32, c 1Take 64, set the encoder and decoder, the encoder and decoder contain convolution layer, pooling layer, activation function, batch normalization and full connection layer, there is a jump connection between the encoder and the corresponding decoder, and the fifteenth feature F 1 Input to the first encoder to get the sixteenth feature F 1E , The sixteenth feature F 1E Input to the first decoder to obtain the seventeenth feature F 1ED , The seventeenth feature F 1ED Input to the second encoder to get the eighteenth feature F 1EDE , The eighteenth feature F 1E Input to the second decoder to obtain the nineteenth feature F 1EDED , The sixteenth feature F 1E and the eighteenth feature F 1EDE Input to the first two-branch fusion module to obtain feature F 1Fus ,

[0120] In branch 2, the seventh image I 2 Input to the second 3×3 convolution to get the twentieth feature F 2 , H 1 , W 1 and C 1 Represents the twentieth feature F 2 The height, width and channel of the twentieth feature F 2 Input to the third encoder to get the twenty-first feature F 2E , The twenty-first feature F 2E and the thirteenth feature F of branch 1 1Fus Add element by element to get the 22nd feature F 2E+ , The twenty-second feature F 2E+ Input to the third decoder to obtain the twenty-third feature F 2E+D , The twenty-second feature F 2E+ and the eighteenth feature F of branch 1 1EDE Input to the second two-branch fusion module to obtain the fourteenth feature F 2Fus ,

[0121] In branch three, the first image I 1o4 Input to the third 3×3 convolution to get the 24th feature F 3 , The twenty-fourth feature F 3Input to the fourth encoder to obtain the twenty-fifth feature F 3E , The twenty-fifth feature F 3E and the fourteenth feature F of branch 2 2Fus Add element by element to get the twenty-sixth feature F 3E+ , The twenty-sixth feature F 3E+ Input to the fourth decoder to obtain the twenty-seventh feature F 3E+D , The twenty-seventh feature F 3E+D Input to the fourth 3×3 convolution to get the eleventh image I 3O ,

[0122] Furthermore, in branch 2, the twenty-third feature F 2E+D Input to the fifth 3×3 convolution to get the tenth image I 2O , The tenth image I 2o and the eleventh image I of branch three 3O Add element by element to get image I 2+ , Image I 2+ Perform 2 times upsampling to obtain the twelfth image I 2+u , In branch 1, the nineteenth feature F 1EDED Input to the sixth 3×3 convolution to get the ninth image I 1+ , The ninth image I 1+ and the twelfth image I of branch two 2+u Perform element-by-element addition to obtain image I 1+f , Then the image I 1+f Input to the seventh 3×3 convolution to obtain the ultra-clear pipeline cutting image I + , I + ∈R H×W×3 , ultra-clear pipeline cutting image I + is the output of the standard pipe cutting image processing model.

[0123] The above-mentioned standard pipeline cutting image processing model can be used to optimize the pipeline recognition task in process pipelines and instrument flow images. Before pipeline recognition, the pipeline recognition image is input into the standard pipeline cutting image processing model to obtain an ultra-clear pipeline recognition image with more details, and subsequent pipeline recognition is performed, which is beneficial to improve the accuracy of pipeline recognition.

[0124] The standard pipeline cutting image processing model provided in the embodiment of the present invention performs feature extraction and refinement processing on the input standard pipeline cutting image layer by layer through multi-scale feature processing and dual-branch fusion modules. The overall structure utilizes the multi-level feature extraction capability of the convolutional network, and prevents information loss through jump connections and multi-branch design. The final output image retains the precise pipeline structure while maintaining high detail expression, providing high-quality ultra-clear pipeline cutting image input for subsequent automated analysis.

[0125] The standard pipeline cutting image processing model provided by the embodiment of the present invention, wherein the multi-scale feature processing module can perform high-dimensional mapping of the two-dimensional coordinates of the input image and the three primary colors of the three-dimensional image, accurately capture the pipeline details by capturing the different scale information of the image, and the dual-branch fusion module fuses features by element-by-element multiplication and addition, which can effectively enhance the image feature expression, and through layer-by-layer fusion and jump connection, the fused features are made richer and more accurate. The standard pipeline cutting image processing model integrates the multi-scale feature processing module and the dual-branch fusion module, inputs the standard pipeline cutting image and undergoes upsampling, convolution and dual-branch fusion, and finally generates an ultra-clear pipeline cutting image with richer details.

[0126] Step S108, inputting the image to be processed into the standard pipeline cutting image processing model to obtain a clear image corresponding to the image to be processed.

[0127] Specifically, after the standard pipeline cutting image processing model is trained, the process pipeline and instrument flow images to be processed are input into the model to output clear images.

[0128] See also Figure 4 The embodiment of the present invention shown provides a standard definition image of a process piping and instrumentation flow and Figure 5 The embodiment of the present invention shown provides Figure 4 The corresponding ultra-high-definition images of process pipelines and instrument flow, both images have a resolution of 512×512. Figure 4 The image shown is processed by the standard pipeline cutting image processing model and the output is Figure 5 image.

[0129] The embodiment of the present invention provides a process piping and instrument flow image processing method, which is applied to a process piping and instrument flow image processing system, wherein the image processing comprises: obtaining a training image; wherein the training image comprises a plurality of first process piping and instrument flow images and a plurality of corresponding second process piping and instrument flow images; the clarity of the second process piping and instrument flow images is N times the clarity of the first process piping and instrument flow images; N is a positive integer greater than 1; preprocessing the first process piping and instrument flow images to obtain a first target image; preprocessing the second process piping and instrument flow images to obtain a second target image; using the first target image and the second target image as pipeline data sets; wherein the preprocessing comprises performing a preprocessing based on a pre-trained non-pipeline target recognition model; Non-target element removal and image cutting processing; training a pre-set basic pipeline cutting image processing model based on the pipeline data set until the pre-set training completion conditions are met, and determining the trained basic pipeline cutting image processing model as the standard pipeline cutting image processing model; wherein the basic pipeline cutting image processing model includes multiple multi-scale feature processing modules and multiple dual-branch fusion modules; inputting the image to be processed into the standard pipeline cutting image processing model to obtain a clear image corresponding to the image to be processed; firstly, performing non-target element removal processing on the process pipeline and instrument flow images through the preset non-pipeline target recognition model, and then performing clarity enhancement on the processed image through the trained standard pipeline cutting image processing model, thereby improving the readability of the image and the efficiency of subsequent image analysis.

[0130] An embodiment of the present invention provides a method for processing process pipeline and instrument flow images. After enhanced processing by a deep learning algorithm, the simplified pipeline image not only has improved resolution, but also enhances the details and contours of the pipeline. Through this method, the image quality of low-resolution process flow charts can be effectively improved, facilitating subsequent automated recognition and analysis, and providing more intuitive and clear image support for industrial process design and optimization.

[0131] An embodiment of the present invention provides a method for processing process pipeline and instrument flow images, aiming to propose a standard pipeline cutting image processing model, wherein a multi-scale feature processing module can perform high-dimensional mapping of the two-dimensional coordinates of an input image and the three primary colors of a three-dimensional image, and accurately capture pipeline details by capturing different scale information of the image. A dual-branch fusion module fuses features by element-by-element multiplication and addition, which can effectively enhance the image feature expression. Through layer-by-layer fusion and jump connection, the fused features are made richer and more accurate. The standard pipeline cutting image processing model integrates the multi-scale feature processing module and the dual-branch fusion module, inputs a standard pipeline cutting image and undergoes upsampling, convolution and dual-branch fusion, and finally generates an ultra-clear pipeline cutting image with richer details.

[0132] Embodiment 2

[0133] On the basis of the above-mentioned embodiments, an embodiment of the present invention provides a process piping and instrument flow image processing system, which is used to execute the process piping and instrument flow image processing method provided in the above-mentioned embodiments.

[0134] Technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the process piping and instrument flow image processing system described above can refer to the corresponding process in the aforementioned embodiment of the process piping and instrument flow image processing method, and will not be repeated here.

[0135] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0136] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device, such as a personal computer, a server, or a network device, to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory ROM, Read-Only Memory, random access memory RAM, Random Access Memory, disk or optical disk, and other media that can store program codes.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A process pipeline and instrument flow image processing method, characterized in that: Applied to process pipeline and instrument flow image processing system, the method comprises: Acquire training images; wherein the training images include a plurality of first process piping and instrument flow images and a plurality of corresponding second process piping and instrument flow images; the definition of the second process piping and instrument flow images is N times the definition of the first process piping and instrument flow images; N is a positive integer greater than 1; The first process pipeline and instrument flow image is preprocessed to obtain a first target image; the second process pipeline and instrument flow image is preprocessed to obtain a second target image; the first target image and the second target image are used as pipeline data sets; wherein the preprocessing includes non-target element removal and image cutting based on a pre-trained non-pipeline target recognition model; Based on the pipeline data set, a preset basic pipeline cutting image processing model is trained until a preset training completion condition is reached, and the trained basic pipeline cutting image processing model is determined as a standard pipeline cutting image processing model; wherein the basic pipeline cutting image processing model includes a plurality of multi-scale feature processing modules and a plurality of dual-branch fusion modules; The image to be processed is input into the standard pipeline cutting image processing model to obtain a clear image corresponding to the image to be processed.

2. The process pipeline and instrument flow image processing method according to claim 1 is characterized in that: The multi-scale feature processing module processes the image by the following steps: The first image I input into the multi-scale feature processing module 1o4 The high-dimensional coordinate information C of the first image is obtained by mapping it to a high-dimensional space through random Fourier feature coding. 1o4H ; Wherein, the first image is obtained by downsampling the first target image by A times; A is a positive integer greater than 1; The first image is input into a preset encoder to obtain a first feature F corresponding to the first image F ; Based on the high-dimensional coordinate information C of the first image 1o4H The first feature F corresponding to the first image F Perform self-attention operation to obtain the second feature F 1o4A ; Based on the second feature F 1o4A and the first image I 1o4 Determine the second image I 1o4u ; For the second image I 1o4u Perform upsampling operation to obtain the third image I 1o4u2 ; Obtain a fourth image I input into the multi-scale feature processing module 1o2 The two-dimensional space coordinates C 1o2 ; Wherein, the fourth image is obtained by downsampling the first target image by B times; B is a positive integer greater than A; The fourth image I 1o2 The two-dimensional space coordinates C 1o2 Mapping to a high-dimensional space through a hash function obtains the fourth image I 1o2 The high-dimensional coordinate information C 1o2H ; Through the third image I 1o4u2 and the fourth image I 1o2 Determine the third feature C 1o2M ; The third feature C 1o2M Connect with the high-dimensional coordinate information to obtain the fourth feature C 1o2M1 ; Regarding the fourth feature C 1o2M1 Perform depth convolution to obtain the fifth image I 1o2u ; For the fifth image I 1o2u Perform upsampling operation to obtain the sixth image I 1o2u2 ; The third image I 1o4u2 and the sixth image I 1o2u2 The output image of the multi-scale feature processing module is determined.

3. The process pipeline and instrument flow image processing method according to claim 1 is characterized in that: The dual-branch fusion module processes the image by the following steps: The fifth feature based on input TB1 and the sixth feature I TB2 Through feature fusion operation, the seventh feature F is obtained TBm ; wherein the feature fusion operation includes at least one of the following: element-by-element multiplication, element-by-element addition and function activation operation; Based on the fifth feature I TB1 The sixth feature I TB2 And the seventh feature F TBm After the feature superposition operation, the channel dimension splicing operation is performed to obtain the eighth feature F TBO ; Wherein, the feature superposition operation includes at least one of the following: element-by-element multiplication, element-by-element addition and depth convolution operation; The eighth feature is used as the output of the dual-branch fusion module.

4. The process pipeline and instrument flow image processing method according to claim 3 is characterized in that: The step of obtaining the seventh feature through feature fusion operation based on the fifth feature and the sixth feature of the input includes: Regarding the fifth feature I TB1 After the convolution operation, the sixth feature I TB2 Perform element-by-element multiplication to obtain the ninth feature; After performing a convolution operation on the sixth feature, the sixth feature is then element-wise multiplied with the fifth feature to obtain a tenth feature; The ninth feature and the tenth feature are added element by element and then a function activation operation is performed to obtain the seventh feature F TBm .

5. The process pipeline and instrument flow image processing method according to claim 4 is characterized in that: Based on the fifth feature I TB1 The sixth feature I TB2 And the seventh feature F TBm After the feature superposition operation is performed, the channel dimension splicing operation is performed, and the steps of obtaining the eighth feature include: The seventh feature is multiplied by the fifth feature element by element, and then the result of the depthwise convolution is added to the sixth feature element by element to obtain an eleventh feature; Multiply the seventh feature by the sixth feature element by element, perform a depthwise convolution, and then add the result to the fifth feature element by element to obtain a twelfth feature; The eleventh feature and the twelfth feature are concatenated in the channel dimension to obtain the eighth feature F TBO .

6. The process pipeline and instrument flow image processing method according to claim 2 is characterized in that: The basic pipeline cutting image processing model further includes: a first branch, a second branch and a third branch; wherein the first branch includes a first double-branch fusion module; the second branch includes a second double-branch fusion module; the basic pipeline cutting image processing model processes the image by the following steps: The first target image I is downsampled to obtain the first image I 1o4 and the fourth image I 1o2 ; The first image I 1o4 Input the multi-scale feature processing module to obtain the third image I 1o4u2 ; The fourth image I 1o2 Input the multi-scale feature processing module to obtain the sixth image I 1o2u2 ; Add the third image and the fourth image element by element to obtain a seventh image I2; The sixth image I 1o2u2 Adding the first target image I element by element to obtain an eighth image I1; In the first branch, a thirteenth feature F is determined based on the eighth image I1. 1Fus and the ninth image I 1+ ; In the second branch, a fourteenth feature F is determined based on the seventh image I2. 2Fus and the tenth image I 2O ; In the third branch, based on the first image I Io4 The fourteenth feature F 2Fus Determine the eleventh image I 3O ; The tenth image I 2O and the eleventh image I 3I After element-by-element addition, upsampling is performed to obtain the twelfth image I 2+u ; The twelfth image I 2+u and the ninth image I 1+ After element-by-element addition, a convolution operation is performed to obtain the output image.

7. The process pipeline and instrument flow image processing method according to claim 6 is characterized in that: Determine a thirteenth feature F based on the eighth image I1 1Fus and the ninth image I 1+ The steps include: Perform a convolution operation based on the first image I1 to obtain a fifteenth feature F1; The fifteenth feature F1 is input into the preset first encoder to obtain the sixteenth feature F 1E ; The sixteenth feature F 1E Input the preset first decoder to obtain the seventeenth feature F 1ED ; The seventeenth feature F 1ED Input to the preset second encoder to obtain the eighteenth feature F 1EDE ; The eighteenth feature F 1EDE Input the preset second decoder to obtain the nineteenth feature F 1EDE ; The sixteenth feature F 1E and the eighteenth feature F 1EDE Input the preset first dual-branch fusion module to obtain the thirteenth feature F 1Fus ; The nineteenth feature F 1EDED Perform a convolution operation to obtain the ninth image I 1+ .

8. The process pipeline and instrument flow image processing method according to claim 7 is characterized in that: Determine the fourteenth feature F based on the seventh image I2 2Fus and the tenth image I 2O The steps include: Perform a convolution operation on the seventh image I2 to obtain a twentieth feature F2; The 20th feature F2 is input into the preset third encoder to obtain the 21st feature F 2E ; The twenty-first feature F 2E With the thirteenth feature F 1Fus Add element by element to get the 22nd feature F 2E+ ; The twenty-second feature F 2E+ Input the preset third decoder to obtain the twenty-third feature F 2E+D ; The twenty-second feature F 2E+ and the eighteenth feature F 1EDE Input the preset second dual-branch fusion module to obtain the fourteenth feature F 2Fus ; The twenty-third feature F 2E+D Perform a convolution operation to obtain the tenth image I 2O .

9. The process pipeline and instrument flow image processing method according to claim 8, characterized in that: Based on the first image I 1o4 The fourteenth feature F 2Fus Determine the eleventh image I 3O The steps include: The first image I 1o4 Perform convolution operation to obtain the twenty-fourth feature F3; The twenty-fourth feature F3 is input into the preset fourth encoder to obtain the twenty-fifth feature F 3E ; The twenty-fifth feature F 3E With the fourteenth feature F 2Fus Add element by element to get the 26th feature F 3E+ ; The twenty-sixth feature F 3E+ Input the preset fourth decoder and get the twenty-seventh feature F 3E+D ; The twenty-seventh feature F 3E+D Perform a convolution operation to obtain the eleventh image I 3O .

10. A process pipeline and instrument flow image processing system, characterized in that: The process piping and instrument flow image processing system is used to execute the process piping and instrument flow image processing method according to any one of claims 1 to 9.

Images