A neural operator-based image segmentation method and system

Abstract

This invention discloses an image segmentation method based on neural operators, belonging to the technical field of image segmentation methods. The method includes: S1: preprocessing an RGB image; S2: image feature extraction; S3: image encoding; S4: distance encoding; S5: vector concatenation; S6: solving the Cheng Han equation using a solver; and S7: threshold segmentation. The beneficial effect of this invention is that it models image segmentation through solutions to partial differential equations, approximates a set of Cheng Han equations using implicit neural representations and convolutional neural networks, and achieves image segmentation by solving the zero-level set. This method does not require pre-segmentation input or multiple rounds of correction, and can generate high-resolution segmentation results from low-resolution images, significantly improving upon directly upscaling low-resolution segmentation results to high resolution.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation method technology, specifically an image segmentation method and system based on neural operators. Background Technology

[0002] With the continuous development of artificial intelligence, machine vision algorithms based on neural networks have been widely applied in many fields such as security inspection, autonomous driving, and robotics. Especially in the field of autonomous driving, visual perception is a crucial method for information acquisition. Current image segmentation methods can be divided into traditional image segmentation methods and deep model-based image segmentation methods.

[0003] Traditional image segmentation methods include thresholding, region growing, active contouring, and level set methods. These methods typically require complex image preprocessing, feature extraction, or human interaction. Although active contouring and level set methods can produce continuous segmentation results, traditional image segmentation methods generally have poor versatility.

[0004] Image segmentation methods based on deep models include codec-based methods, methods based on convolutional networks and graph models, methods based on multi-scale pyramid networks, methods based on dilated convolutions, and methods based on R-CNNs (mainly used for instance segmentation). Generally speaking, deep neural network-based methods achieve better segmentation results than traditional methods, but their processing efficiency is relatively low and they often perform poorly on high-resolution images.

[0005] In the field of visual analysis, there is a great demand for object detection and image segmentation. Current image segmentation algorithms perform poorly on high-resolution images, and existing high-resolution image segmentation algorithms do not effectively utilize global information, resulting in unclear boundaries.

[0006] Existing technologies include patent literature: [1] Patent Publication No. CN114882227A, a method for segmenting human tissue images and related equipment; the application field of the described method is mainly human tissue images. The method is based on weak labels and pseudo labels to generate label data and combines preset deep learning models to train images into a network. The method introduces two-stage segmentation to increase accuracy and introduces labels to reduce the time required for labeling.

[0007] [2] Patent Publication No. CN114972300A, A material image segmentation and recognition method based on computer vision and deep learning; The application field of the described method is mainly material images. The method is a two-stage method. First, the traditional method is used to segment the image, and then the deep learning model is used to identify the microstructure of the image.

[0008] [3] Patent Publication No. CN114820636A, A Three-Dimensional Medical Image Segmentation Model and Its Training Method and Application; describes a deep learning-based three-dimensional medical image segmentation model and its training method and application. The method designs a network model consisting of a prototype calculation module, an encoder-decoder module, a task controller module, a dynamic convolution kernel generator module, and a dynamic segmentation head module. It solves some labeling problems in the field of medical image segmentation, solves the problem of too many parameters and inflexible task expansion in current mainstream methods, and improves the segmentation results to a certain extent.

[0009] One major drawback of the aforementioned patent documents [1]-[3] is that they cannot perform continuous segmentation of images. Existing mainstream image segmentation methods can only segment images with a fixed resolution.

[0010] Journal: [4] Shen, Tianncheng, et al. "High Quality Segmentation for UltraHigh-resolution Images." Proceedings of the IEEE / CVF Conference on ComputerVision and Pattern Recognition.2022;

[0011] [5] Hu, Hanzhe, et al. "Learning Implicit Feature Alignment Function for Semantic Segmentation." arXiv preprint arXiv:2206.08655(2022).

[0012] The method in reference [4] requires a coarse image segmentation result and pre-segmentation; it also requires multiple rounds of correction to obtain a reliable result, resulting in high computational cost. Compared with [4], the method in reference [5] does not require pre-segmentation input and multiple rounds of correction, thus solving the problems of needing pre-segmentation and high computational cost. In addition, the global information introduced in [4] and [5] mainly relies on image feature maps and location information. Summary of the Invention

[0013] To address the aforementioned technical problems, the purpose of this invention is to provide an image segmentation method based on neural operators. This method models image segmentation through solutions to partial differential equations, approximates the solutions to a set of equations using implicit neural representations and convolutional neural networks, and achieves image segmentation by solving the zero-level set. This method does not require pre-segmentation input or multiple rounds of correction.

[0014] To achieve the above objectives, the present invention adopts the following technical solution:

[0015] A neural operator-based image segmentation method includes the following steps:

[0016] S1: Preprocessing the RGB image: The RGB image is represented by I, where I∈[3×H×W]. The preprocessing includes pixel normalization and size normalization.

[0017] S2: Image feature extraction: composed of neural networks. θ The input is I inp The output is image features, expressed as F. feat It means that F feat ∈[C feat ×H feat ×W feat Its mathematical expression is F. feat =F … (I inp );

[0018] S3: Image encoding: by neural network E θ Composition, input is F feat The output is an image encoding, in I enc It means, I enc ∈[1×H grid ×W grid The output image coding size matches the grid size in the distance coding, and its mathematical expression is I. enc =E θ (F feat );

[0019] S4: Distance Encoding: The input contains the coordinates of the encoded grid points and the search point, where the encoded grid points are P. grid ∈[2×H grid ×W grid The grid points are uniformly sampled within the domain [1,-1]×[1,-1].

[0020] S5: Vector concatenation: [I] enc ∈[1×H grid ×W grid Transform into a column vector V I ∈[L×1], P enc ∈[1×H grid ×W grid Transformed vector V P ∈[1×L], where L=H grid ×W grid Then V I With V P Concatenate them into an encoding vector Vcat ∈[1×2L];

[0021] S6: Solving the Cheng Han equation using a solver: The solver consists of a neural network S θ The structure consists of an input encoding vector V. cat The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Mathematical expression Its output is the signed distance function;

[0022] S7: Threshold Segmentation: Input is grid points P grid By concatenating vectors and using the sign distance function of the solver, u grid ∈[1×H grid ×W grid ] indicates that threshold segmentation judgment u grid If the value is greater than 0, set the part greater than 0 as the target; otherwise, treat it as background.

[0023] Preferably, in S1, the preprocessing of the RGB image further includes random cropping, color transformation, and contrast transformation, with the transformed image using I... inp It means, I inp ∈[3×H inp ×W inp ].

[0024] Preferably, in S1, the pixel normalization operation specifically involves scaling the pixel values ​​proportionally to a specific range, and the size normalization operation specifically involves scaling the image size to a specific size through interpolation.

[0025] Preferably, the encoded grid points in S4 are specifically uniformly sampled W along the x-axis direction in the range [1, -1]. grid There are points, including 1 and -1, and similarly, H is uniformly sampled in the [1, -1] region along the y-axis. grid A grid of points, including 1 and -1, is shown in the table below:

[0026]

[0027] Where the search point x∈[2×1] is a point in the field [1,-1]×[1,-1], and the distance code P enc ∈[1×H grid ×W grid ],P enc By calculating the retrieval point x and the coded grid point P enc The Euclidean distance of each point is obtained, mathematically expressed as P. enc =||xP grid ||2.

[0028] Preferably, the solver used in S6 to solve the Cheng Han equation using a neural network is a multilayer perceptron with input-output dimensions.

[0029] This invention also provides an image segmentation method based on neural operators, comprising the following steps:

[0030] S1: Preprocessing the RGB image: The RGB image is represented by I, where I∈[3×H×W]. The preprocessing includes pixel normalization and size normalization.

[0031] S2: Image feature extraction: composed of neural networks. θ The input is I inp The output is image features, expressed as F. feat It means that F feat ∈[C feat ×H feat ×W feat Its mathematical expression is F. feat =F θ (I inp );

[0032] S3: Image encoding: by neural network E θ Composition, input is F feat The output is an image encoding, in I enc It means, I enc ∈[1×H grid ×W grid The output image coding size matches the grid size in the distance coding, and its mathematical expression is I. enc =E θ (F feat );

[0033] S4: Distance Encoding: The input contains the coordinates of the encoded grid points and the search point, where the encoded grid points are P. grid ∈[2×H grid ×W grid The grid points are uniformly sampled within the domain [1,-1]×[1,-1].

[0034] S5: Vector concatenation: [I] enc ∈[1×H grid ×W grid Transform into a column vector V I ∈[L×1], P enc ∈[1×H grid ×W grid Transformed vector V P ∈[1×L], where L=H grid ×W grid Then VI With V P Concatenate them into an encoding vector V cat ∈[1×2L];

[0035] S6: Solving the Cheng Han equation using a solver: The solver consists of a neural network S θ The structure consists of an input encoding vector V. cat The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Mathematical expression Its output is the signed distance function;

[0036] S7: Loss Calculation: Used to calculate the loss function during the training of the entire network. It consists of two parts: one part calculates the boundary loss L. b The other part calculates the PDE loss, which is the constraint L of the Chenghan equation. pde , can be represented as: This is the estimated signed distance function at the retrieval point x, calculated from the output of the solver network module during training. It needs to be computed when the retrieval point x is on the target boundary. express The absolute value; in addition, regardless of the location of the search point x within the domain, it must satisfy... Therefore, the automatic differentiation method is used to calculate... gradient Then calculate After calculating the two parts of the loss, the gradient of the learnable parameters is calculated by backpropagation, and then the learnable parameters are updated by the stochastic gradient descent algorithm until convergence.

[0037] This invention also provides an image segmentation system based on neural operators, comprising:

[0038] The preprocessing module performs image normalization on RGB images, where the RGB image is represented by I, where I ∈ [3 × H × W]. The transformed image is represented by I. inp It means, I inp ∈[3×H inp ×W inp ];

[0039] The image feature extraction module is composed of a neural network F. θ The input is I inp The output is image features, expressed as F. feat It means that F feat ∈[C feat ×H feat ×W feat Its mathematical expression is: F feat =F θ (Iinp );

[0040] The image encoding module is composed of a neural network E. θ Composition, input is F feat The output is an image encoding, in I enc It means, I enc ∈[1×H grid ×W grid The output image encoding size needs to match the grid size in the distance encoding, and its mathematical expression is: I enc =E θ (F feat );

[0041] The distance encoding module takes as input the coordinates of the encoded grid points and the search point, where the encoded grid points are P. grid ∈[2×H grid ×W grid The grid points are uniformly sampled within the domain [1,-1]×[1,-1].

[0042] The vector concatenation module will concatenate I enc ∈[1×H grid ×W grid Transform into a column vector V I ∈[L×1], P enc ∈[1×H grid ×W grid Transformed vector V P ∈[1×L], where L=H grid ×W grid Then V I With V P Concatenate them into an encoding vector V cat ∈[1×2L], and fed into the solver network module;

[0043] The solver network module consists of a neural network S θ The structure consists of an input encoding vector V. cat The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Its mathematical expression is:

[0044] The threshold segmentation module takes grid points P as input. grid The sign distance function between the vector concatenation module and the solver module is obtained from u. grid ∈[1×H grid ×W grid The threshold segmentation module will determine u. grid If the value is greater than 0, set the portion greater than 0 as the target; otherwise, treat it as the background and output the segmentation result.

[0045] Preferably, the image feature extraction module can be selected from: ResNet, non-local, or Visual Transformer backbone networks.

[0046] Preferably, the input grid points of the distance encoding module are uniformly sampled W points in the [1,-1] region along the x-axis. grid H points, including 1 and -1; uniformly sampled in the [1, -1] region along the y-axis. grid There are points, including 1 and -1; as shown in the table below:

[0047]

[0048] The search point x∈[2×1] is a point in the field [1,-1]×[1,-1], and the distance code P is... enc ∈[1×H grid ×W grid ],P enc By calculating the retrieval point x and the coded grid point P enc The Euclidean distance to each point is obtained, and its mathematical expression is: P enc =||xP grid ||2.

[0049] Beneficial effects

[0050] This invention models image segmentation using solutions to partial differential equations, approximates the solutions to a set of equations using implicit neural representations and convolutional neural networks, and achieves image segmentation by solving the zero-level set. This method eliminates the need for pre-segmentation input and multiple rounds of correction, and can generate high-resolution segmentation results from low-resolution images, representing a significant improvement over directly upscaling low-resolution segmentation results to high resolution. Attached Figure Description

[0051] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0052] Figure 2 This is a flowchart of the method in Embodiment 2 of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0054] Definition: Convolutional Neural Network (CNN): A type of feedforward neural network that incorporates convolutional computations and has a deep structure; it is one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling them to perform translation-invariant classification of input information according to their hierarchical structure.

[0055] Neural hidden function: A function that approximates a continuous signal using a neural network. After the network is trained, it can be used as a neural network representation of the continuous signal.

[0056] Operator learning: using neural networks to learn the mapping relationship of solutions to specific types of partial differential equations.

[0057] Image segmentation: Image segmentation is the technique and process of dividing an image into several specific regions with unique properties and extracting targets of interest.

[0058] This approach models the image segmentation problem as a problem of solving the Cheng-Han equation, which has the following form:

[0059]

[0060] u=0 for x∈Γ

[0061] x represents the coordinates of a point within the domain, and Ω and Γ in the formula represent the domain and boundary set of x, respectively, and their gradient magnitudes. It can also be written as |u x |) is 1, and u = 0 on the boundary. The solution u(x) of the Cheng Han equation represents the sign distance function. Here, the sign of the solution inside the boundary is defined as positive, and the sign of the solution outside the boundary is defined as negative. The goal of image segmentation is to extract the target boundary from the image domain, so the Cheng Han equation can be used to model the image segmentation problem. The image itself is used as an abstract expression of the boundary conditions, and the coordinates x of the search point in any domain are used as input to estimate the distance from x to the boundary. This method uses a neural network to extract features from the image and encode the image, encodes the coordinates by distance, and then concatenates the two codes as input to the neural network. The boundary conditions and gradient constraints of the Cheng Han equation are used as the loss function to train the neural network until convergence. After the network converges, the distance from the boundary can be estimated for any coordinate in the domain. In actual use, the domain of the image is defined as [1,-1]×[1,-1], and the coordinates of grid points with arbitrary precision are extracted as shown in the table below:

[0062]

[0063] The image and grid point coordinates are fed into the network model to estimate the symbolic distance from each grid point to the boundary. The position with a value greater than zero is the target in the image, which is the segmentation result.

[0064] Example 1, as Figure 1 As shown, an image segmentation method based on neural operators includes the following steps:

[0065] 1. RGB images with I∈[3×128×128] are sent to the image preprocessing module;

[0066] 2. The image preprocessing module will perform image normalization operations on the image. inp = (I / 255 - [0.5, 0.5, 0.5]) / [0.5, 0.5, 0.5], First divide the pixel value of each channel of the image by 255, then subtract 0.5 from each channel value and divide by 0.5 again to get I. inp ∈[3×128×128];

[0067] 3. The image feature extraction module selects nonlocal-ResNet, and the input is I. inp ∈[3×128×128], output F feat ∈[64×32×32];

[0068] 4. The image encoding module is a two-layer cascaded convolutional neural network, with input F. feat ∈[64×32×32], output I enc ∈[1×32×32];

[0069] 5. The position grid is x grid Uniform sampling points in the range [1,-1]×[1,-1] ∈[2×512×512].

[0070] 6. The distance encoding module input is a position grid, and the encoding grid is P. grid ∈[2×32×32], for x grid The position coordinates of P can be calculated. enc ∈[512 2 [32×32] position codes.

[0071] 7. In the vector concatenation module, I will... enc ∈[1×32×32] copied to I enc ∈[512 2 [×32×32] and P enc By splicing, we get V cat ∈[512 2 [×2048], sent to the decoder network module.

[0072] 8. The solver network module is a multilayer perceptron, specifically a 7-layer fully connected network with 1 neuron per layer (2048, 100, 100, 100, 100, 100, 1). ELU is used as the activation function. Therefore, the input is V. cat ∈[512 2 [×2048], the output is u grid ∈[512 2 ×1], and convert it to u grid ∈[512×512].

[0073] 9. The threshold segmentation module will calculate u grid If the value is >0, output the segmentation result.

[0074] Example 2, as Figure 2 As shown, an image segmentation method based on neural operators includes the following steps:

[0075] 1. RGB images with I∈[3×128×128] are sent to the image preprocessing module;

[0076] 2. The image preprocessing module will perform image normalization operations on the image. inp = (I / 255 - [0.5, 0.5, 0.5]) / [0.5, 0.5, 0.5], First divide the pixel value of each channel of the image by 255, then subtract 0.5 from each channel value and divide by 0.5 again to get I. inp ∈[3×128×128];

[0077] 3. The image feature extraction module selects nonlocal-ResNet, and the input is I. inp ∈[3×128×128], output F feat ∈[64×32×32];

[0078] 4. The image encoding module is a two-layer cascaded convolutional neural network, with input F. feat ∈[64×32×32], output I enc ∈[1×32×32];

[0079] 5. Random sampling points with position coordinates x∈[1×1] in the range [1,-1]×[1,-1].

[0080] 6. The distance encoding module input is a position grid, and the encoding grid is P. grid ∈[2×32×32], for x grid The position coordinates of P can be calculated. enc ∈ [1×32×32] position codes;

[0081] 7. In the vector concatenation module, I will... enc ∈[1×32×32] and P enc By splicing, we get V cat ∈[1×2048], and fed into the solver network module;

[0082] 8. The solver network module is a multilayer perceptron, specifically a 7-layer fully connected network with 1 neuron per layer (2048, 100, 100, 100, 100, 100, 1). ELU is used as the activation function. Therefore, the input is V. cat ∈[1×2048], the output is And calculate The signed distance function at the retrieval point x is estimated from the output of the solver network module during training. When the retrieval point x is on the target boundary, it needs to be calculated. express The absolute value; in addition, regardless of the location of the search point x within the domain, it must satisfy... Therefore, the automatic differentiation method is used to calculate... gradient Then calculate After calculating the two parts of the loss, the gradient of the learnable parameters is calculated by backpropagation, and then the learnable parameters are updated by the stochastic gradient descent algorithm until convergence.

[0083] This invention also provides an image segmentation system based on neural operators, comprising:

[0084] The image preprocessing module in this scheme only processes RGB images, denoted by I, where I ∈ [3×H×W]. The preprocessing module includes, but is not limited to: pixel normalization (scaling pixel values ​​proportionally to a specific range), size normalization (scaling the image size to a specific size through interpolation), etc. During training, the image preprocessing module includes, but is not limited to: random cropping (randomly cropping the image to a certain size), color transformation, contrast transformation, etc. The transformed image is denoted by I. inp It means, I inp ∈

[0085] [3×H inp ×W inp ];

[0086] The image feature extraction module is composed of a neural network F θ The input is I inp The output is image features in F feat It means that F feat ∈[C feat ×H feat ×W feat ], expressed mathematically as F feat =F θ (I inp The image feature extraction module can be implemented by any neural network that meets the input-output dimensions, and can be selected from, but is not limited to, common backbone networks such as ResNet, non-local, and Visual Transformer.

[0087] The image encoding module is composed of a neural network E θ Composition, input is F feat The output is an image encoded in I encIt means, I enc ∈[1×H grid ×W grid The output image encoding size needs to match the grid size in the distance encoding. This module is mathematically represented as I. enc =E θ (F feat The image feature coding module can be implemented by any neural network that meets the input-output dimensions, including but not limited to: ResNet and ordinary multi-layer convolutional networks.

[0088] The input to the distance encoding module includes the coordinates of the encoded grid points and the search point, where the encoded grid point is P. grid ∈[2×H grid ×W grid The encoded grid points are obtained by uniform sampling within the domain [1,-1]×[1,-1], which can be understood as uniform sampling W along the x-axis direction within the [1,-1] domain. grid Each point contains 1 and -1, and similarly, H is uniformly sampled in the [1, -1] region along the y-axis. grid Each point contains both 1 and -1. The grid formed by these points is shown in the table below.

[0089]

[0090] The search point x∈[2×1] is a point in the field [1,-1]×[1,-1], and the distance code P is... enc ∈[1×H grid ×W grid ],P enc By calculating the retrieval point x and the coded grid point P enc The Euclidean distance of each point is obtained, mathematically expressed as P. enc =||xP grid ||2;

[0091] The vector concatenation module will I enc ∈[1×H grid ×W grid Transform into a column vector V I ∈[L×1], P enc ∈[1×H grid ×W grid Transformed vector V P ∈[1×L], where L=H grid ×W grid Then V I With V P Concatenate them into an encoding vector V cat ∈[1×2L];

[0092] The solver module consists of a neural network S θThe structure consists of an input encoding vector V. cat The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Mathematical expression This process essentially uses a neural network to solve the Cheng Han equation, so the output is a signed distance function. The solver network module is a neural network that meets the input and output dimensions, and can be selected from, but is not limited to, a multilayer perceptron.

[0093] The input to the threshold segmentation module is grid points P. grid By using the signed distance function between the vector concatenation module and the solver module, u can be obtained. grid ∈[1×H grid ×W grid The threshold segmentation module will determine u. grid >0, and set the part greater than 0 as the target; otherwise, treat it as the background.

[0094] The loss module is mainly used to calculate the loss function during the training of the entire network. It consists of two parts: one part calculates the boundary loss L. b The other part calculates the PDE loss, which is the constraint L of the process equation. pde , can be represented as: The signed distance function at the retrieval point x is estimated from the output of the solver network module during training. When the retrieval point x is on the target boundary, it needs to be calculated. express The absolute value; in addition, regardless of the location of the search point x within the domain, it must satisfy... Therefore, the automatic differentiation method is used to calculate... gradient Then calculate After calculating the two parts of the loss, the gradient of the learnable parameters is calculated using backpropagation, and then the learnable parameters are updated using the stochastic gradient descent algorithm.

[0095] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. An image segmentation method based on neural operators, characterized in that, Includes the following steps: S1: Preprocessing the RGB image: RGB images are represented by I. Preprocessing includes pixel normalization and size normalization. S2: Image Feature Extraction: Composed of a Neural Network The input is The output is image features, with express, Its mathematical expression is as follows ; S3: Image encoding: by neural network Composition, input is The output is an image encoding, in order to express, The output image encoding size matches the grid size in the distance encoding, and its mathematical expression is as follows: ; S4: Distance Encoding: The input contains the coordinates of the encoded grid points and the search point, where the encoded grid points are... The coded grid points are Obtained by uniform sampling within the domain; S5: Vector concatenation: ... Transform into column vector ,Will Transformed vector ,in Then and Concatenate them into an encoding vector ; S6: Solving the Cheng Han equation using a solver: The solver is a neural network. The structure consists of an input encoding vector. The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Mathematical expression is Its output is the signed distance function; S7: Threshold Segmentation: Input is grid points By concatenating vectors and using the sign distance function of the solver, from This indicates threshold segmentation judgment. And set the part greater than 0 as the target, otherwise treat it as the background.

2. The image segmentation method based on neural operators as described in claim 1, characterized in that, In S1, the preprocessing of the RGB image also includes random cropping, color transformation, and contrast transformation. The transformed image is then... express, .

3. The image segmentation method based on neural operators as described in claim 1, characterized in that, In S1, the pixel normalization operation specifically scales the pixel values ​​to a specific range proportionally, while the size normalization operation specifically scales the image size to a specific size through interpolation.

4. The image segmentation method based on neural operators as described in claim 1, characterized in that, In S4, the coded grid points are specifically located along the x-axis. Uniform sampling There are points, including 1 and -1, and similarly, in the y-axis direction... Uniform sampling A grid of points, including 1 and -1, is formed by these points, where the retrieval point is... ,for Points in the domain, distance encoding , By calculating the retrieval point x and the coded grid points The Euclidean distance of each point is obtained, mathematically expressed as: .

5. The image segmentation method based on neural operators as described in claim 1, characterized in that, The solver used in S6 to solve the Cheng Han equation using a neural network is a multilayer perceptron that satisfies the input-output dimensions.

6. An image segmentation method based on neural operators, characterized in that, Includes the following steps: S1: Preprocessing the RGB image: RGB images are represented by I. Preprocessing includes pixel normalization and size normalization. S2: Image Feature Extraction: Composed of a Neural Network The input is The output is image features, with express, Its mathematical expression is as follows ; S3: Image encoding: by neural network Composition, input is The output is an image encoding, in order to express, The output image encoding size matches the grid size in the distance encoding, and its mathematical expression is as follows: ; S4: Distance Encoding: The input contains the coordinates of the encoded grid points and the search point, where the encoded grid points are... The coded grid points are Obtained by uniform sampling within the domain; S5: Vector concatenation: ... Transform into column vector ,Will Transformed vector ,in Then and Concatenate them into an encoding vector ; S6: Solving the Cheng Han equation using a solver: The solver is a neural network. The structure consists of an input encoding vector. The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Mathematical expression is Its output is the signed distance function; S7: Loss Calculation: Used to calculate the loss function during the training of the entire network. It consists of two parts: one part calculates the boundary loss. The other part calculates the PDE loss, which is the constraint of the Chenghan equation. , can be represented as: ; This is the estimated signed distance function at the retrieval point x, calculated from the output of the solver network module during training. It needs to be computed when the retrieval point x is on the target boundary. , express The absolute value; in addition, regardless of the location of the search point x within the domain, it must satisfy... Therefore, the automatic differentiation method is used to calculate... gradient , and then calculate After calculating the two parts of the loss, the gradient of the learnable parameters is calculated by backpropagation, and then the learnable parameters are updated by stochastic gradient descent until convergence.

7. An image segmentation system based on neural operators, characterized in that, include: The preprocessing module performs image normalization on RGB images, denoted by I. The transformed image is express, ; The image feature extraction module is composed of neural networks. The input is The output is image features, with express, Its mathematical expression is ; Image encoding module, composed of neural network Composition, input is The output is an image encoding, in order to express, The output image encoding size needs to match the grid size in the distance encoding, and its mathematical expression is: ; The distance encoding module takes as input the coordinates of the encoded grid points and the search point, where the encoded grid points are... The coded grid points are Obtained by uniform sampling within the domain; The vector concatenation module will Transform into column vector ,Will Transformed vector ,in Then and Concatenate them into an encoding vector The data is fed into the solver network module. The solver network module consists of a neural network. The structure consists of an input encoding vector. The output is the estimated nearest distance from the retrieval point x to the boundary of the segmented target. Its mathematical expression is: ; The threshold segmentation module takes grid points as input. By using the signed distance function between the vector concatenation module and the solver module, This indicates that the threshold segmentation module will determine... The part greater than 0 is set as the target; otherwise, it is treated as the background, and the segmentation result is output.

8. The image segmentation system based on neural operators as described in claim 7, characterized in that, The image feature extraction module can be selected from: ResNet, non-local, or Visual Transformer backbone network.

9. The image segmentation system based on neural operators as described in claim 7, characterized in that, The input grid points for the distance encoding module are in the x-axis direction. Uniform sampling There are points, including 1 and -1; along the y-axis direction. Uniform sampling There are 1 points, including 1 and -1; These points form a grid. Search point ,for Points in the domain, distance encoding , By calculating the retrieval point x and the coded grid points The Euclidean distance to each point is obtained, and its mathematical expression is: .

Images