Curve structure image enhancement method and device based on filtering and geometric constraint interaction

Abstract

The invention discloses a curve structure image enhancement method and device based on filtering and geometric constraint interaction, and relates to the technical field of image enhancement. The method comprises the following steps: performing multi-scale feature coding on a to-be-processed degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels; and based on the structure guiding information, respectively determining a linear / curve structure region to be repaired on the feature maps of the multiple levels, and intercepting corresponding region-of-interest features. And performing spatial filtering on the features of the region of interest by using a large-scale convolution filter to realize global context sensing processing so as to repair texture fractures caused by shielding and obtain repaired features. And on the basis of a pixel-level self-attention mechanism, geometric feature interaction and smooth filtering are carried out on the curve sampling point sequence represented by the repaired features so as to suppress high-frequency geometric noise, and smoothed features are obtained. And carrying out image texture reconstruction on the smoothed features to generate an enhanced image containing complete curve textures.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, in particular to a curve structure image enhancement method and device based on interaction of filtering and geometric constraint. BACKGROUND

[0002] In the field of digital image processing, there is an urgent need for image denoising, structure reconstruction and geometric distortion correction of slender curve structures such as road linear markings in complex backgrounds. In the actual image acquisition process, due to the perspective projection characteristics of the imaging device and external environmental interference such as object occlusion, road wear or strong light reflection, the linear structure in the original image often appears serious signal degradation, which is manifested as structure fracture, edge blur and geometric morphological distortion. This degradation problem needs to be solved through efficient digital signal processing means to realize clear, continuous and geometrically accurate image restoration to improve visual quality and application reliability.

[0003] The prior art mainly deals with the above challenges through three methods. First, the image filtering method based on cascading small-scale convolution kernels adopts a backbone network such as ResNet that stacks multiple 3x3 convolution kernels, gradually expands the feature extraction range through the hierarchical receptive field theory, and combines the multi-scale information of the feature pyramid. Second, the discrete point mapping strategy based on local region of interest aggregation uses anchor point prior and ROI Gather technology to extract local features near the curve path, and independently regresses the coordinate offset of each sampling point through a fully connected layer. Finally, the structure coincidence degree calculation based on fixed bandwidth extends the line segment with a fixed pixel radius, and the LineIoU algorithm is used to measure the error between the predicted and true structures to optimize the model parameters.

[0004] However, these existing methods have significant defects. The effective receptive field distribution of cascading small convolution kernel filtering is uneven, making it difficult to establish long-distance pixel correlation across occluded areas, resulting in the inability to solve the texture fracture problem. The discrete point independent reconstruction mechanism ignores the topological correlation within the curve, and is easily disturbed by local noise, resulting in high-frequency jitter and geometric artifacts. The fixed bandwidth error calculation violates the optical perspective principle, introducing a large fault tolerance space in the image long-range area, causing perspective geometric distortion and positioning accuracy to decline, limiting the effect of full-scene high-fidelity restoration. SUMMARY

[0005] The present application provides a curve structure image enhancement method and device based on interaction of filtering and geometric constraint to improve at least one of the above technical problems.

[0006] In a first aspect, the present application provides a curve structure image enhancement method based on interaction of filtering and geometric constraint, which comprises steps S1 to S5.

[0007] S1, obtain a to-be-processed degraded image, encode multi-scale features of the degraded image through a backbone network and a feature pyramid network, and obtain feature maps of multiple levels.

[0008] S2, determine line / curve structure areas to be repaired on the feature maps of multiple levels respectively based on structure guidance information, and intercept corresponding region-of-interest features.

[0009] S3, perform spatial filtering on the region-of-interest features by using a large-scale convolution filter, implement global context-aware processing to repair texture breaks caused by occlusion, and obtain repaired features.

[0010] S4, based on a pixel-level self-attention mechanism, perform geometric feature interaction and smoothing filtering on a curve sampling point sequence represented by the repaired features, to suppress high-frequency geometric noise, and obtain smoothed features.

[0011] S5, perform image texture reconstruction on the smoothed features, and generate an enhanced image containing complete curve texture. Perform step-by-step refinement reconstruction between the multi-level feature maps, use the structure guidance features output by the previous level to guide the feature fusion and detail restoration of the next level, to improve the continuity and clarity of the line / curve texture structure in the enhanced image.

[0012] In a second aspect, the application provides a curve structure image enhancement device based on filtering and geometric constraint interaction, which comprises an encoding module, a region module, a spatial filtering module, a smoothing filtering module, and a decoding module.

[0013] The encoding module is configured to obtain a to-be-processed degraded image, encode multi-scale features of the degraded image through a backbone network and a feature pyramid network, and obtain feature maps of multiple levels.

[0014] The region module is configured to determine line / curve structure areas to be repaired on the feature maps of multiple levels respectively based on structure guidance information, and intercept corresponding region-of-interest features.

[0015] The spatial filtering module is configured to perform spatial filtering on the region-of-interest features by using a large-scale convolution filter, implement global context-aware processing to repair texture breaks caused by occlusion, and obtain repaired features.

[0016] The smoothing filtering module is configured to perform geometric feature interaction and smoothing filtering on a curve sampling point sequence represented by the repaired features based on a pixel-level self-attention mechanism, to suppress high-frequency geometric noise, and obtain smoothed features.

[0017] The decoding module is used for image texture reconstruction on the smoothed features to generate an enhanced image containing complete curve texture. The gradual refinement reconstruction is performed between multi-level feature maps, and the structure-guided features output by the previous level are used to guide the feature fusion and detail recovery of the next level, so that the continuity and clarity of the linear / curve texture structure in the enhanced image are improved.

[0018] By adopting the technical solution, the following technical effects can be achieved.

[0019] The curve structure image enhancement method based on the interaction of filtering and geometric constraint can significantly improve the image structure integrity (deocclusion / completion) of the enhanced image, effectively suppress (denoising / smoothing) geometric artifacts and noise, and correct (high-fidelity recovery) perspective distortion with high precision. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the specific embodiments of the present application. It should be understood that the following drawings only show some specific embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0021] Fig. 1 is a flowchart of the curve structure image enhancement method.

[0022] Fig. 2 is an algorithm structure diagram of the curve structure image enhancement method. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application.

[0024] Embodiment one, please refer to Figs. 1-2 , aiming at the structure texture breakage problem caused by the insufficient local filtering receptive field of the existing curve structure image enhancement method, the geometric artifact (jitter) problem caused by the lack of internal topological constraint, and the long-range geometric distortion problem caused by the neglect of optical perspective effect. The first embodiment of the present application provides a curve structure image enhancement method based on the interaction of filtering and geometric constraint. The curve structure image enhancement method based on the interaction of filtering and geometric constraint in the embodiment can effectively realize high-fidelity recovery and geometric enhancement of the elongated curve structure in the degraded image. The core processing logic follows the data flow direction of "texture signal completion-geometric noise filtering-perspective distortion correction".

[0025] The global texture signal completion specifically is: for the structure fracture caused by occlusion, a large-scale decomposition convolution filter is introduced at the feature extraction end. By significantly expanding the passband range of the filter at the physical level (for example, using a non-local window of 9x9 or even larger), the signal correlation between long-distance pixels in the image is established. This module uses the effective texture signal of the unoccluded area to interpolate and fill the features at the signal interruption of the occluded area, thereby repairing the broken image structure at the data level.

[0026] The geometric noise smoothing filter specifically is: for the artifacts generated by discrete reconstruction, the system constructs a pixel-level self-attention topology-dependent model. This model calculates the signal correlation matrix of the sequence of sampling points inside the curve and performs a non-local feature weighted average operation. This process forces the strong features of high-confidence areas to correct the weak features of low-confidence areas, which essentially performs a geometric smoothing filter on the reconstructed point list, effectively filtering out high-frequency jitter noise and ensuring the topological continuity of the output curve.

[0027] The perspective distortion adaptive correction specifically is: for the scale distortion caused by optical perspective, the system introduces a spatially variable tolerance (SV-LineIoU) constraint mechanism. The system discards the fixed error radius and instead constructs a dynamic tolerance function with respect to the longitudinal depth of the image. This mechanism dynamically adjusts the pixel-level calculation tolerance according to the perspective law of "wide near and narrow far", applies stricter constraints in the far view area, effectively offsets the quantization error, and realizes consistent high-precision geometric restoration throughout the image.

[0028] The curve structure image enhancement method based on the interaction of filtering and geometric constraints of the embodiment receives a degraded road scene image as input, and after feature encoding, global texture repair, geometric smoothing filtering, and cascading refinement processing, finally outputs a continuous, smooth, and geometrically accurate curve structure mathematical model.

[0029] The curve structure image enhancement method can be executed by a curve structure image enhancement device based on the interaction of filtering and geometric constraints (hereinafter referred to as: image enhancement device). In particular, it is executed by one or more processors in the image enhancement device to implement steps S1 to S5.

[0030] S1, acquire a degraded image to be processed, and encode the degraded image through a backbone network and a feature pyramid network to obtain multiple levels of feature maps. Among them, the multiple levels of feature maps at least include feature maps of deep high semantic features and shallow high resolution features.

[0031] Preferably, S1 specifically is: obtaining the to-be-processed degraded image and using ResNet as the backbone network to extract the basic features, and then fusing the deep and shallow features through the feature pyramid network to generate three feature maps of different resolution levels of L0, L1 and L2. Among them, the L0 layer is a high semantic low resolution feature map, which is used to extract the global context representation of the linear / curved structure to guide the subsequent enhanced reconstruction and structure continuity repair (i.e., roughly positioning the global trend of the curve). The L1 layer is a medium resolution feature map. The L2 layer is a high resolution low semantic feature map, which is used to finely recover the edge details of the curve.

[0032] S1 performs image input and multi-scale feature coding through the backbone network plus the feature pyramid.

[0033] Firstly, the original two-dimensional RGB image (input) is received, which may contain degradation phenomena such as uneven illumination, object occlusion or texture wear.

[0034] Then, the basic feature extraction (backbone network, ResNet is used in this embodiment) is performed: the preprocessed image is input into the backbone network. The network acts as an image feature encoder, which converts pixel-level signals into high-dimensional semantic features through layer-by-layer downsampling and nonlinear activation. In this process, the shallow network extracts edge, texture and other detail features, and the deep network extracts semantic, contour and other abstract features.

[0035] Finally, the multi-scale pyramid construction (FPN) is performed: the feature pyramid network (FPN) is introduced to fuse deep and shallow features and construct three different resolution feature map levels: L0 (high semantic low resolution), L1 (medium resolution) and L2 (high resolution low semantic). The L0 layer is used to roughly position the global trend of the curve, and the L2 layer is used to finely recover the edge details of the curve.

[0036] S2, based on the structure guiding information (i.e., the preset curve prior), determines the linear / curved structure area to be repaired on the multi-level feature maps respectively, and intercepts the corresponding region of interest features. The structure guiding information is used to limit the processing area of image enhancement. Preferably, S2 specifically includes: initializing a set of learnable curve priors at the L0 layer. The curve prior is a learnable straight line or ray (representing the initial position of the curve in the image), which is used to define the sampling path or region aggregation window of the linear / curved structure area, so as to extract the texture information to be repaired on the multi-level feature maps. For each curve prior, a region of interest aggregation module is used to intercept feature vectors on each level feature map along the path of the curve prior, to obtain a feature matrix containing curve path texture information.

[0037] The operation of region of interest (ROI) feature extraction and primary aggregation is as follows.

[0038] First, a priori initialization is performed, and a set of learnable "curve priors" (i.e., preset straight lines or rays) is initialized at the L0 layer, representing the initial positions of the possible curves in the image.

[0039] Then, ROI feature extraction is performed. For each curve prior, the system uses a region of interest gather (ROIGather) module to extract the corresponding feature vector along the path of the curve on the current feature map (L0, L1, L2). It reduces the feature processing of the entire image to local processing of only the area near the curve, improving computational efficiency. The feature matrix obtained at this time (explained later) contains texture information on the curve path, but may have signal breaks due to occlusion.

[0040] S3, spatial filtering of the ROI feature is performed using a large-scale convolution filter to achieve global context-aware processing to repair the texture breaks caused by occlusion, and the repaired feature is obtained.

[0041] The ROI feature output by S2 is limited to a local receptive field and may contain invalid noise points (such as areas occluded by a preceding vehicle). This embodiment applies large-scale decomposition convolution (Large Kernel Convolution) (such as 1x9 and 9x1) to the ROI feature for spatial filtering again. This step forcibly expands the physical receptive field of the feature, enabling it to "see" the valid texture upstream and downstream of the occluded area. The algorithm uses the signals of the unoccluded area to interpolate and complete (In-painting) the features of the occluded area, repairing the broken structure signals in the feature space.

[0042] Prior art pixel feature extraction before the fusion of large kernel convolution technology mainly uses 3x3 convolution or standard convolution to aggregate features. Assuming that the feature map is , the standard convolution operation can be represented as a weighted sum within the local neighborhood .

[0043] .

[0044] In the formula, x represents the position after convolution. is the weight parameter of the convolution kernel (or filter) at a specific position . is the feature at position in the feature map. is the feature at position in the feature map.

[0045] If is very small, when ​When in the occluded region (pixel value is noise or invalid texture), its neighborhood Also often invalid information, resulting in output Unable to restore the image features.

[0046] In order to realize the image texture repair, the embodiment introduces large core convolution. In order to reduce the calculation complexity and strengthen the orthogonal direction (horizontal / vertical) feature flow, the embodiment is designed as large-scale decomposed convolution.

[0047] First, the large-scale decomposed convolution is used to filter the region of interest features. Wherein, the large-scale decomposed convolution is composed of vertical direction ×1 convolution kernel And horizontal direction 1× Convolution kernel Cascade, Is an integer greater than 3.

[0048] Assume that the original region of interest feature of the input is The embodiment defines two orthogonal large-scale one-dimensional convolution kernels (vertical direction ×1) and (horizontal direction 1× ). Preferably, .

[0049] The core formula is: In the formula, the global context feature obtained after large-scale decomposed convolution filtering is The original region of interest feature is .

[0050] And Represent the large-scale pixel aggregation operation in the vertical and horizontal directions of the image respectively. It is equivalent to establishing a long-distance “signal transmission channel” in the vertical direction of the image. The physical meaning is that the vertical convolution can be along the longitudinal extension direction of the long distance, across the intermediate occlusion, and “transport” the clear image texture of upstream and downstream to the occluded pixel position.

[0051] Is the spatial filtering window size in image processing. By increasing The calculation of a single pixel point is no longer limited to the 3×3 grid adjacent to it, but is directly mathematically related to the background information in the distance (such as tens of pixels away) in the image.

[0052] Then, the global context features extracted after filtering are residually fused with the original region of interest features to obtain the repaired features.

[0053] Specifically, in order to construct a global context-aware feature map In this embodiment, a residual fusion structure is used to inject the long-range features extracted by the large kernel into the original features.

[0054] .

[0055] In the formula These are the features after the repair. The fusion coefficient is a learnable coefficient. This is the activation function. For batch normalization processing. To enhance the signal. This represents the global context features obtained after large-scale depth-separable decomposition convolutional filtering. Features of the original region of interest.

[0056] Learnable fusion coefficients This is used to control the degree to which global context information modifies local details. When local textures are missing, the model automatically increases the weight of this item, utilizing... Repair the broken image structure.

[0057] The addition operation (+) in the code is not a simple superposition, but rather a signal enhancement. Original region of interest features. It retains local details (such as edge sharpness), while It provides an overview of the macro-structural trend. Weighting Dynamically adjust the intensity of the "repair" in areas with clear texture. Smaller, in areas of occlusion and blurring It automatically enlarges, thereby achieving intelligent image structure repair.

[0058] This embodiment addresses the problem of insufficient effective receptive field caused by stacking small convolutional kernels in existing systems by introducing large-size convolutional kernels at the head of the feature extraction network. The image processing mechanism uses large-kernel convolution to directly expand the pixel range covered by a single convolution operation at the physical level, forcibly establishing direct connections between distant pixels in the image. Technical effect: This processing method endows the model with the ability of "image in-painting". Even if the middle of a curve is completely occluded (texture loss), this module can still capture the linear texture features upstream and downstream of the occluded area, using global structural information to "cross" the occluded area, inferring and recovering the image structure of the missing part, thus solving the problem of feature map fragmentation.

[0059] S4. Based on the pixel-level self-attention mechanism, geometric feature interaction and smoothing filtering are performed on the sequence of curve sampling points represented by the repaired features to suppress high-frequency geometric noise and obtain smoothed features.

[0060] Although the features repaired by S3 are connected, they may contain local noise. In this embodiment, the sampling point features of a single curve are regarded as a topological sequence, and the pixel dependency matrix within the sequence is calculated to quantify the geometric correlation between points.

[0061] S4's in-line self-attention-based geometric feature interaction mechanism is a geometric smoothing filter operation. It forces strong features from sharp regions in the image to flow to blurred regions, uses internal geometric constraints to correct outliers, eliminates high-frequency jitter (artifacts), and ensures that the output features are topologically smooth and continuous.

[0062] In the curve-based self-attention feature interaction mechanism, the Region of Interest (ROIGather) module in existing curve-structured image processing models (CLRNet, Curve Lane Detection Network), while using an attention mechanism, calculates the relationship between "ROI features" and "overall image features." The expression is: In the formula This is the weight matrix. The sequence of curve sampling points characterizing the restored features. Features of the entire image. This represents the number of channels. This indicates transpose. It is a normalized exponential function.

[0063] For a single curve sampling point sequence Existing methods typically use fully connected (FC) layers for each sampling point during regression. Processed independently. This means... The adjustment does not refer to the sampling point or sampling point This point-wise independence leads to an unsmooth output geometry.

[0064] Therefore, this embodiment introduces intra-lane pixel-wise self-attention, which will be applied to a single curve. Each sampled point feature is treated as a sequence of image pixels, and the interdependencies among them are calculated. This represents the number of sampling points (e.g., 72).

[0065] Specifically, the single curve Each sample point feature is treated as a topological sequence, and a query matrix is ​​generated through linear transformation. Key matrix Sum matrix Assuming This is the ROI feature matrix corresponding to a single curve. In this embodiment, the query matrix is ​​first generated using three linear transformation matrices. Key matrix Sum matrix .

[0066] .

[0067] .

[0068] .

[0069] In the formula To query the linear transformation matrix. Let be the linear transformation matrix of the key. It is a linear transformation matrix. For querying the matrix. This is the key matrix. It is a value matrix. This is the ROI feature matrix corresponding to a single curve.

[0070] Then calculate the pixel dependency matrix. Define the pixel dependency matrix as... , indicating the first The point and the first Geometric correlation strength between points: .

[0071] In the formula Let be the pixel dependency matrix, representing the first... The point and the first The strength of the geometric correlation between points. It is a normalized exponential function. For the query matrix Row vectors. The first key matrix Row vectors. is the dimension of the key vector. This indicates transpose. The first key matrix Row vectors. Natural exponential function.

[0072] By using the dependency matrix to perform weighted aggregation of the values ​​of all points, a geometrically corrected feature sequence is generated. .

[0073] .

[0074] In the formula This is the ROI feature matrix corresponding to a single curve. This is the adjustment coefficient. This is the pixel dependency matrix. It is a value matrix.

[0075] It is an N×N matrix that quantifies the relationship between any two pixels on a curve. For example, if the points at the bottom of the image (sharp) and the points in the middle of the image (blurred) follow the same straight line trend, then... The value will be very large.

[0076] This matrix multiplication operation represents the physical flow of features. It forces points in blurred regions (weak features) to "absorb" information from points in sharp regions (strong features). For each point on the curve, the algorithm no longer relies solely on its own features, but instead references the weighted features of all other points on the entire curve.

[0077] Through this interaction, the "weak features" located in the blurred region are corrected by the "strong features" in the clear region. Mathematically, this is equivalent to performing a low-pass filter on the fitted point sequence, effectively filtering out high-frequency jitter noise that violates geometric continuity, ensuring that the restored curve is smooth and continuous in its topological structure.

[0078] Through this full-sequence interaction, the final feature of the i-th point no longer depends solely on the pixel value of its own location, but rather on the weighted average of the features of all points along the entire line. Mathematically, this is equivalent to performing a non-local geometric filtering and smoothing on the curve, thereby eliminating outlier noise in the output image and ensuring the continuity of the curve.

[0079] To address the high-frequency geometric noise (artifacts) generated by independent regression of discrete points, this embodiment will use a single curve... Each sampling point is considered a topological sequence. A self-attention mechanism is used to calculate the geometric dependencies between points, performing non-local feature smoothing filtering. The image processing mechanism utilizes a self-attention algorithm to calculate a dependency matrix within the sequence. This matrix quantifies the feature correlation between each sampling point and all other sampling points in the image. Based on this weight matrix, each point can aggregate feature information from other locations along the entire curve.

[0080] S4 achieves complementary enhancement of image features. Strong texture features located in sharp areas near the image can be propagated to distant or blurred areas through an attention mechanism to correct weak features. This internal geometric constraint ensures that the output point sequence remains highly smooth and continuous in the image topology, eliminating high-frequency noise.

[0081] S5. Reconstruct the image texture of the smoothed features to generate an enhanced image containing complete curve texture. A progressively refined reconstruction is performed between multi-level feature maps, using the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, thereby improving the continuity and clarity of the linear / curved texture structure in the enhanced image. Specifically, geometric decoding is performed on the smoothed features to output the geometric parameters of the curve structure, and the curve prior is updated based on the decoding results for cascaded refinement, making the curve structure gradually approximate the real curve from coarse to fine.

[0082] Preferably, S5 specifically involves: inputting the smoothed curve features into the decoding layer, outputting the enhanced image, and an enhancement weight map and orientation guidance field for adaptive fusion of the enhancement results. Specifically, the smoothed curve features are input into the fully connected layer for decoding, outputting the probability of the existence of the curve structure, the coordinates and angle of the curve's starting point, and an offset for fine-tuning the prior shape of the curve.

[0083] The progressively refined reconstruction is as follows: starting from the L0 layer feature map, the structural distribution features (i.e. geometric parameters) of the decoded output are passed to the L1 layer feature map for feature fusion and enhancement, and then the output of the L1 layer is passed to the L2 layer feature map for further detail restoration, thus achieving iterative optimization from coarse to fine.

[0084] Specifically, the features from each layer, after global repair and smoothing, are fed into fully connected (FC) layers for decoding. The key geometric parameters of the network output curve include the probability of existence (whether there is a curve at that location), the coordinates and angle of the starting point, and fine-tuning of the prior shape.

[0085] During decoding, a cascaded refinement loop is performed, refining the low-resolution layer using the output of the high-semantic low-resolution layer. Specifically, the output of layer L0 is used as a new "prior" and passed to layer L1, repeating steps 2-5, and then to layer L2. Through this "coarse-fine" iteration, the actual physical curve is gradually approximated.

[0086] The specific process of the cascaded refinement is as follows: the decoding result of the L0 layer is used as a new curve prior and passed to the L1 layer to repeat steps S2 to S5. Then, the decoding result of the L1 layer is used as a new curve prior and passed to the L2 layer to repeat steps S2 to S5. The real physical curve is approximated through the "coarse-fine" iteration.

[0087] During the training phase, the difference between the enhanced image and the target image is constrained based on a spatially variable tolerance loss function. Specifically, a dynamic tolerance radius is determined based on the normalized vertical coordinates of the sampling positions. Based on this dynamic tolerance radius, the reconstruction error between the enhanced image and the target image is calculated within the corresponding window, and the spatially variable tolerance loss is obtained by summing the results over multiple sampling positions. Preferably, during the training phase, the error between the predicted curve and the true curve is calculated based on the spatially variable tolerance loss function, and the calculation tolerance for each sampling point is dynamically adjusted according to the image depth.

[0088] Specifically, during the training phase, in order to calculate the difference between the restored result and the true value, the system no longer uses a fixed radius. Instead, it calculates the dynamic tolerance radius in real time based on the image ordinate of each sampling point (the formula is shown below).

[0089] Loss calculation is based on constructing predicted and true line segments using dynamic radii, and calculating their intersection-over-union (IoU) loss. A tolerance is used at the bottom of the image (near view) and a narrow tolerance is used at the top of the image (far view). This forces the network to focus on optimizing the localization accuracy in the far view region during backpropagation, thereby correcting geometric distortions caused by perspective effects.

[0090] The image data obtained after repair and updating by each layer of the decoding head is then fed into the next layer of interest. Each layer then trains the next image based on the obtained loss function. The L0, L1, and L2 features of the three-layer feature pyramid are iteratively updated, and finally the training of the model is completed, resulting in an enhanced and restored curve structure image method and system.

[0091] By passing optimized feature maps and dynamically adjusted loss weights layer by layer, the network's ability to perceive distant regions is further enhanced. The final segmentation output maintains the continuity of the overall structure while significantly improving the accuracy of distant boundary localization, effectively mitigating misjudgment caused by perspective compression.

[0092] During training, the loss function (LOSS) calls the IoU function. In the IoU function, current techniques use a fixed scalar radius. (e.g., 15 pixels) to calculate the predicted texture energy center location. and the location of the texture energy center of the target In the Effective pixel response range of each sampling point and : .

[0093] .

[0094] In the formula To predict image texture in the first... The effective pixel response range of each sampling point. For the target image texture in the first The true pixel distribution range of each sampling point.

[0095] This embodiment makes a key modification to the data flow and computation logic of the prior art: transitive coordinate dependency. The loss function is modified so that when calling the LineIoU loss function, additional coordinate dependencies are passed in. (i.e., curve-first detection) coordinate vector. Because The vector contains the positional information (y-coordinate) required for "spatial variability" and must be passed to the core of the SV-LineIoU calculation. Internally, the SV-LineIoU function replaces the "fixed scalar" with a "dynamic vector." To achieve a variable radius in space: First, obtain the normalized vertical coordinates of each sampling point. .

[0096] Specifically, define a key coordinate vector. This is a 1-dimensional vector with a length of (e.g., 72). It stores the distance from the bottom (1.0) of the image to the top (0.0). Normalized vertical coordinates The SV-LineIoU function receives... Vector (of shape [N], e.g.,

[72] ).

[0097] Then, calculate the first... Dynamic tolerance radius of each sampling point .

[0098] .

[0099] In the formula It is the calculated number of The dynamic tolerance radius of each sampling point. It is a basic radius scalar (e.g., 15 pixels). For normalized vertical coordinates, representing The first in the vector Each element is a normalized vertical coordinate (0.0 for the top and 1.0 for the bottom). It is a configurable base scaling factor (e.g., 0.5). It is a configurable dynamic scaling factor (e.g., 1.0).

[0100] Specifically, two new hyperparameters are read from the configuration file: the base scaling factor. (e.g., 0.5) and dynamic scaling factor (e.g., 1.0). And define a spatially variable function, such as a linear function, to compute the dynamic radius vector. .

[0101] Then, the enhanced image and the target image are constructed based on the dynamic tolerance radius.

[0102] Assume L=15. =0.5. =1.0.

[0103] At the bottom of the image ( ≈1.0), radius It is approximately 15 × (0.5 + 1.0) = 22.5 pixels.

[0104] At the top of the image ( ≈0.0), radius It is approximately 15 × (0.5 + 0.0) = 7.5 pixels.

[0105] It is a vector of shape [N] { , ..., }

[0106] Use this dynamic radius vector This is used to calculate the boundary between the enhanced image and the target image, replacing the original fixed radius scalar. .

[0107] .

[0108] .

[0109] In the formula To predict image texture in the first... The effective pixel response range of each sampling point. For the target image texture in the first The true pixel distribution range of each sampling point. This indicates that it is calculated based on the dynamic radius. This represents the predicted location of the texture energy center. The target is the location of the texture energy center.

[0110] The predicted image texture calculated based on the dynamic radius is represented in the first... The effective pixel response range of each sampling point. The target image texture calculated based on the dynamic radius is represented in the first... The true pixel distribution range of each sampling point.

[0111] Next, calculate the intersection and union of the images.

[0112] Specifically, redefine the intersection. Union The form of the formula remains the same, but the content has changed.

[0113] .

[0114] .

[0115] In the formula The intersection. It is a union. To take the smaller value. To take the larger value. This represents the intersection calculated based on the dynamic radius. This represents the union calculated based on the dynamic radius.

[0116] The final spatial variable tolerance loss (LOSS) is calculated as follows.

[0117] LOSS= .

[0118] In the formula This represents the number of sampling points.

[0119] To address the issue of fixed radii violating perspective principles, this solution constructs a dynamic error calculation model based on the image's ordinate. The image processing mechanism involves converting the original scalar radius... Replace with dynamic vector The system reads the normalized ordinate of each pixel in real time. The computational tolerance at that location is dynamically generated through a linear mapping function.

[0120] This mechanism implements an adaptive constraint of "wider tolerance for near objects and narrower tolerance for distant objects" in the loss calculation stage of image processing. A larger tolerance is assigned to the bottom (foreground) of the image, and a very small tolerance is assigned to the top (background). This forces the image processing model to learn more refined pixel localization features in the background region, thereby significantly reducing the quantization error caused by perspective distortion and improving the geometric restoration accuracy of the entire image.

[0121] This embodiment's curve-structure image enhancement method based on the interaction of filtering and geometric constraints significantly improves the structural integrity (de-occlusion / completion) of the enhanced image. Compared to baseline methods, this embodiment utilizes the spatial filtering characteristics of large-kernel convolution to effectively solve the signal loss problem in complex scenes. Experiments show that this method can successfully restore continuous texture structures from occluded degraded images, significantly reducing the structural breakage rate and achieving complete reconstruction of the physical morphology.

[0122] The curve structure image enhancement method in this embodiment can effectively suppress (denoise / smooth) geometric artifacts and noise. By introducing pixel-level self-attention interaction, a strong internal geometric constraint is established. This mechanism acts like an intelligent smoothing filter, effectively filtering out high-frequency jitter and outlier noise during the reconstruction process, resulting in smooth and natural curve edges after reconstruction, and eliminating non-physical visual artifacts.

[0123] The curve structure image enhancement method in this embodiment can accurately correct (high-fidelity restore) perspective distortion. The spatially variable tolerance calculation model strictly follows the optical principles of physical imaging, eliminating quantization errors in distant areas. This allows the restored image structure to maintain extremely high geometric positioning accuracy across the entire depth range, achieving true high-fidelity image restoration.

[0124] Example 2: The present invention provides a curve structure image enhancement device based on the interaction of filtering and geometric constraints, which includes an encoding module, a region module, a spatial filtering module, a smoothing filtering module and a decoding module.

[0125] The encoding module is used to acquire the degraded image to be processed, and to perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels.

[0126] The region module is used to determine the linear / curved structure regions to be repaired on feature maps at multiple levels based on structural guidance information, and to extract the corresponding region of interest features.

[0127] The spatial filtering module is used to perform spatial filtering on the features of the region of interest using a large-scale convolutional filter, thereby achieving global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features.

[0128] The smoothing filter module is used to perform geometric feature interaction and smoothing filtering on the sequence of curve sampling points represented by the repaired features based on a pixel-level self-attention mechanism, so as to suppress high-frequency geometric noise and obtain smoothed features.

[0129] The decoding module is used to reconstruct the image texture of the smoothed features, generating an enhanced image containing complete curve textures. It performs progressively refined reconstruction across multiple levels of feature maps, using the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, thereby improving the continuity and clarity of linear / curved texture structures in the enhanced image.

[0130] It is understood that the image enhancement device may be an electronic device with computing power, such as a laptop computer, desktop computer, server, smartphone or tablet computer.

[0131] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0132] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0134] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks. It should be noted that in this embodiment, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0136] It should be understood that the term "and / or" used in this embodiment is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, in this embodiment, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0137] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0138] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing curve-structured images based on the interaction of filtering and geometric constraints, characterized in that, Include: S1. Obtain the degraded image to be processed, and perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels. S2. Based on structural guidance information, determine the linear / curved structure regions to be repaired on feature maps at multiple levels, and extract the corresponding regions of interest features. S3. Spatial filtering of the region of interest features is performed using a large-scale convolutional filter to achieve global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features. S4. Based on the pixel-level self-attention mechanism, geometric feature interaction and smoothing filtering are performed on the sequence of curve sampling points represented by the repaired features to suppress high-frequency geometric noise and obtain smoothed features. S5. Reconstruct the image texture of the smoothed features to generate an enhanced image containing complete curve texture; perform progressive refinement reconstruction between multi-level feature maps, and use the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, so as to improve the continuity and clarity of the linear / curved texture structure in the enhanced image.

2. The method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to claim 1, characterized in that, S3 specifically includes: Large-scale depthwise separable convolutions are used to filter the features of the region of interest; wherein, the large-scale depthwise separable convolutions consist of vertically oriented... ×1 convolution kernel 1× in the horizontal direction convolution kernel Cascaded configuration, It is an integer greater than 3; In the formula The global context features are obtained after large-scale depth-separable decomposition convolutional filtering. Features of the original region of interest; The filtered global context features are residually fused with the original region of interest features to obtain the repaired features.

3. The method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to claim 1, characterized in that, The expression for residual fusion is as follows; ； In the formula Features after repair; The learnable fusion coefficient; For activation functions; For batch normalization processing; To enhance the signal.

4. The method for enhancing curve structures based on the interaction of filtering and geometric constraints according to claim 1, characterized in that, S4 is detailed below; Single curve Each sample point feature is treated as a topological sequence, and a query matrix is ​​generated through linear transformation. Key matrix Sum matrix ; Then calculate the pixel dependency matrix: In the formula The pixel dependency matrix; It is a normalized exponential function; For the query matrix Row vectors; The first key matrix Row vectors; The dimension of the key vector; Indicates transpose; By using the dependency matrix to weight and aggregate the values ​​of all points, a geometrically corrected feature sequence is generated. ; In the formula This is the ROI feature matrix corresponding to a single curve; This is the adjustment coefficient.

5. The method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to claim 1, characterized in that, S5 is detailed below; The smoothed curve features are input into the decoding layer, which outputs the enhanced image, as well as an enhancement weight map and a direction guidance field for adaptive fusion of the enhancement results. The progressively refined reconstruction is as follows: starting from the L0 layer feature map, the structural distribution features of the decoded output are used as guiding information to be passed to the L1 layer feature map for feature fusion and enhancement, and then the output of the L1 layer is passed to the L2 layer feature map for further detail restoration, thus achieving iterative optimization from coarse to fine.

6. A method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to any one of claims 1 to 5, characterized in that, During the training phase, the difference between the enhanced image and the target image is constrained based on the spatially variable tolerance loss function. Specifically, the dynamic tolerance radius is determined according to the normalized longitudinal coordinates of the sampling position, and the reconstruction error between the enhanced image and the target image is calculated within the corresponding window based on the dynamic tolerance radius. The spatially variable tolerance loss is obtained by summing the results over multiple sampling positions.

7. The method for enhancing curve structures based on the interaction of filtering and geometric constraints according to claim 6, characterized in that, The calculation process of the spatially variable tolerance loss function is as follows; Obtain the normalized vertical coordinates of each sampling point ; Calculate the first Dynamic tolerance radius of each sampling point; In the formula It is the calculated number of Dynamic tolerance radius of each sampling point; It is a basic radius scalar; It is a configurable base scaling factor; It is a configurable dynamic scaling factor; Constructing enhanced and target images based on dynamic tolerance radius; ； ； In the formula To predict image texture in the first... Effective pixel response range of each sampling point; For the target image texture in the first The true pixel distribution range of each sampling point; This is based on calculations using the dynamic radius; The predicted location of the texture energy center; The target is the location of the texture energy center; Calculate the intersection and union of the images; ； ； In the formula For intersection; It is a union; To take the smaller value; To take the larger value; Calculate the spatially variable tolerance loss (LOSS); LOSS = In the formula This represents the number of sampling points.

8. A method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to any one of claims 1 to 5, characterized in that, S1 specifically involves: acquiring the degraded image to be processed and using ResNet as the backbone network to extract basic features; then fusing deep and shallow features through a feature pyramid network to generate feature maps at three different resolution levels: L0, L1, and L2. Among them, the L0 layer is a high-semantic, low-resolution feature map used to extract global contextual representations of linear / curved structures to guide subsequent enhancement reconstruction and structural continuity restoration; the L1 layer is a medium-resolution feature map; and the L2 layer is a high-resolution, low-semantic feature map used to finely restore the edge details of the curves.

9. The method for enhancing curve structures based on the interaction of filtering and geometric constraints according to claim 8, characterized in that, The structural guidance information is used to define the processing area for image enhancement; S2 specifically includes: Initialize a set of learnable curve priors at L0 layer; the curve priors are learnable straight lines or rays used to define the sampling path or region aggregation window of the linear / curved structure region so as to extract the texture information to be repaired on the multi-level feature map. For each curve prior, the feature vector is extracted from the feature map at each level along the path of the curve prior by the region of interest aggregation module, resulting in a feature matrix containing the texture information of the curve path.

10. A curve structure image enhancement device based on the interaction of filtering and geometric constraints, characterized in that, Suitable for performing the curve structure image enhancement method based on the interaction of filtering and geometric constraints as described in any one of claims 1 to 9; The curve structure image enhancement device includes: The encoding module is used to acquire the degraded image to be processed, and to perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels. The region module is used to determine the linear / curved structure regions to be repaired on feature maps at multiple levels based on structural guidance information, and to extract the corresponding region of interest features. The spatial filtering module is used to perform spatial filtering on the features of the region of interest using a large-scale convolutional filter, thereby achieving global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features. The smoothing filter module is used to perform geometric feature interaction and smoothing filtering on the sequence of curve sampling points represented by the repaired features based on the pixel-level self-attention mechanism, so as to suppress high-frequency geometric noise and obtain smoothed features. The decoding module is used to reconstruct the image texture of the smoothed features and generate an enhanced image containing complete curve texture. It performs progressive refinement reconstruction between multi-level feature maps and uses the structural guidance features output from the previous level to guide the feature fusion and detail restoration of the next level, so as to improve the continuity and clarity of the linear / curved texture structure in the enhanced image.

Images