Curve structure image enhancement method and device based on interaction of filtering and geometric constraint

Abstract

The application discloses a curve structure image enhancement method and device based on filtering and geometric constraint interaction, and relates to the technical field of image enhancement. The method comprises the following steps: a to-be-processed degraded image is subjected to multi-scale feature coding through a backbone network and a feature pyramid network to obtain feature maps of multiple levels. Linear / curve structure regions to be repaired are determined on the feature maps of the multiple levels based on structure guiding information, and corresponding region-of-interest features are intercepted. Spatial filtering is performed on the region-of-interest features by using a large-scale convolution filter to realize global context perception processing to repair texture breakage caused by occlusion, and repaired features are obtained. A pixel-level self-attention mechanism is used to perform geometric feature interaction and smoothing filtering on a curve sampling point sequence represented by the repaired features to suppress high-frequency geometric noise, and smoothed features are obtained. Image texture reconstruction is performed on the smoothed features to generate an enhanced image containing complete curve texture.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and more specifically, to a method and apparatus for enhancing curve-structured images based on the interaction of filtering and geometric constraints. Background Technology

[0002] In the field of digital image processing technology, there is an urgent need for image denoising, structural reconstruction, and geometric distortion correction of slender curved structures such as road linear markings against complex backgrounds. During actual image acquisition, the linear structures in the original image often suffer severe signal degradation due to the perspective projection characteristics of imaging equipment and external environmental interference such as object occlusion, road surface wear, or strong light reflection. This degradation manifests as structural breakage, blurred edges, and geometric distortion. This degradation problem urgently requires efficient digital signal processing techniques to achieve clear, continuous, and geometrically accurate image restoration, thereby improving visual quality and application reliability.

[0003] Existing technical solutions address these challenges primarily through three approaches. First, image filtering methods based on cascaded small-scale convolutional kernels employ a backbone network like ResNet with stacked 3x3 convolutional kernels. This gradually expands the feature extraction range through hierarchical receptive field theory and integrates multi-scale information using feature pyramids. Second, a discrete point mapping strategy based on local region of interest (ROI) aggregation utilizes anchor point priors and ROI Gathering techniques to extract local features near curve paths, independently regressing the coordinate offset of each sampling point through fully connected layers. Finally, a fixed-pixel radius line segment is calculated based on fixed-bandwidth structural overlap, and the LineIoU algorithm is used to measure the error between the predicted and actual structures to optimize model parameters.

[0004] However, these existing methods have significant drawbacks. The uneven distribution of the effective receptive field in cascaded small convolutional kernel filtering makes it difficult to establish long-distance pixel relationships across occluded regions, resulting in unresolved texture breakage issues. Discrete point independent reconstruction mechanisms ignore the internal topological correlation of curves, making them susceptible to local noise interference, leading to high-frequency jitter and geometric artifacts. Fixed bandwidth error calculation violates the principles of optical perspective, introducing excessively large tolerance space in distant areas of the image, causing perspective geometric distortion and decreased positioning accuracy, thus limiting the effectiveness of high-fidelity restoration across the entire scene. Summary of the Invention

[0005] The present invention provides a method and apparatus for enhancing curve structure images based on the interaction of filtering and geometric constraints, so as to improve at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a method for enhancing curve structure images based on the interaction of filtering and geometric constraints, which includes steps S1 to S5.

[0007] S1. Obtain the degraded image to be processed, and perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels.

[0008] S2. Based on structural guidance information, determine the linear / curved structure regions to be repaired on feature maps at multiple levels, and extract the corresponding regions of interest features.

[0009] S3. Spatial filtering of the region of interest features is performed using a large-scale convolutional filter to achieve global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features.

[0010] S4. Based on the pixel-level self-attention mechanism, geometric feature interaction and smoothing filtering are performed on the sequence of curve sampling points represented by the repaired features to suppress high-frequency geometric noise and obtain smoothed features.

[0011] S5. Reconstruct the smoothed features to generate an enhanced image containing complete curve textures. A progressively refined reconstruction is performed between multi-level feature maps, using the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, thereby improving the continuity and clarity of linear / curved texture structures in the enhanced image.

[0012] Secondly, the present invention provides a curve structure image enhancement device based on the interaction of filtering and geometric constraints, which includes an encoding module, a region module, a spatial filtering module, a smoothing filtering module and a decoding module.

[0013] The encoding module is used to acquire the degraded image to be processed, and to perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels.

[0014] The region module is used to determine the linear / curved structure regions to be repaired on feature maps at multiple levels based on structural guidance information, and to extract the corresponding region of interest features.

[0015] The spatial filtering module is used to perform spatial filtering on the features of the region of interest using a large-scale convolutional filter, thereby achieving global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features.

[0016] The smoothing filter module is used to perform geometric feature interaction and smoothing filtering on the sequence of curve sampling points represented by the repaired features based on a pixel-level self-attention mechanism, so as to suppress high-frequency geometric noise and obtain smoothed features.

[0017] The decoding module is used to reconstruct the image texture of the smoothed features, generating an enhanced image containing complete curve textures. It performs progressively refined reconstruction across multiple levels of feature maps, using the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, thereby improving the continuity and clarity of linear / curved texture structures in the enhanced image.

[0018] By adopting the above technical solution, the present invention can achieve the following technical effects.

[0019] The curve structure image enhancement method based on the interaction of filtering and geometric constraints of the present invention can significantly improve the image structure integrity (de-occlusion / completion) of the enhanced image, effectively suppress (de-noiseing / smoothing) geometric artifacts and noise, and correct (high-fidelity restoration) perspective distortion with high precision. Attached Figure Description

[0020] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the method for enhancing curved structure images.

[0022] Figure 2 This is a diagram of the algorithm structure for image enhancement methods based on curve structures. Detailed Implementation

[0023] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention.

[0024] Example 1, please refer to Figures 1 to 2 This invention addresses the problems of texture breakage due to insufficient receptive field in local filtering, geometric artifacts (jitter) due to lack of internal topological constraints, and geometric distortion of distant views caused by neglecting optical perspective effects in existing curve structure image enhancement methods. The first embodiment of this invention provides a curve structure image enhancement method based on the interaction of filtering and geometric constraints. This method effectively restores and geometrically enhances slender curve structures in degraded images with high fidelity. Its core processing logic follows a data flow of "texture signal completion—geometric noise filtering—perspective distortion correction".

[0025] Global texture signal completion specifically involves introducing a large-scale decomposition convolution filter at the feature extraction stage to address structural breaks caused by occlusion. By significantly expanding the passband range of the filter at the physical level (e.g., using a 9×9 or even larger non-local window), signal correlations between long-distance pixels in the image are established. This module utilizes the effective texture signal from the unoccluded area to interpolate and fill features at the signal interruptions in the occluded area, thereby repairing the broken image structure at the data level.

[0026] The geometric noise smoothing filter specifically involves constructing a pixel-level self-attention topological dependency model to address artifacts generated during discrete reconstruction. This model calculates the signal correlation matrix of the sampling point sequence within the curve and performs a non-local feature-weighted averaging operation. This process forcibly utilizes the strong features of high-confidence regions to correct the weak features of low-confidence regions, essentially performing a geometric smoothing filter on the reconstructed point sequence. This effectively filters out high-frequency jitter noise and ensures the topological continuity of the output curve.

[0027] The adaptive perspective distortion correction specifically involves introducing a spatially variable tolerance (SV-LineIoU) constraint mechanism to address scale distortion caused by optical perspective. Instead of a fixed error radius, the system constructs a dynamic tolerance function related to the vertical depth of the image. Based on the perspective principle of "wider near, narrower far," this mechanism dynamically adjusts the pixel-level computational tolerance, applying stricter constraints in distant areas to effectively offset quantization errors and achieve consistent high-precision geometric restoration across the entire image.

[0028] The curve structure image enhancement method based on the interaction of filtering and geometric constraints in this embodiment receives a degraded road scene image as input, and after feature encoding, global texture restoration, geometric smoothing filtering and cascade thinning processing, it finally outputs a continuous, smooth and geometrically accurate mathematical model of the curve structure.

[0029] The curve structure image enhancement method can be performed by a curve structure image enhancement device based on the interaction of filtering and geometric constraints (hereinafter referred to as: image enhancement device). In particular, it is performed by one or more processors in the image enhancement device to implement steps S1 to S5.

[0030] S1. Obtain the degraded image to be processed, and perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels. Among them, the feature maps of multiple levels include feature maps of deep high semantic features and shallow high resolution features.

[0031] Preferably, S1 specifically involves: acquiring the degraded image to be processed and extracting basic features using ResNet as the backbone network; then fusing deep and shallow features through a feature pyramid network to generate feature maps at three different resolution levels: L0, L1, and L2. The L0 layer is a high-semantic, low-resolution feature map used to extract global contextual representations of linear / curved structures to guide subsequent enhancement reconstruction and structural continuity restoration (i.e., roughly locating the global direction of the curve). The L1 layer is a medium-resolution feature map. The L2 layer is a high-resolution, low-semantic feature map used for fine-tuning the edge details of the curve.

[0032] S1 uses a backbone network and feature pyramids to encode image input and multi-scale features.

[0033] First, receive the raw 2D RGB image (input), which may contain degradation phenomena such as uneven lighting, object occlusion, or texture wear.

[0034] Then, basic feature extraction (backbone network, ResNet in this embodiment) is performed: the preprocessed image is input into the backbone network. This network acts as an image feature encoder, converting pixel-level signals into high-dimensional semantic features through layer-by-layer downsampling and non-linear activation. In this process, shallow networks extract detailed features such as edges and textures, while deep networks extract abstract features such as semantics and contours.

[0035] Finally, a multi-scale pyramid network (FPN) is constructed: a feature pyramid network (FPN) is introduced to fuse deep and shallow features, constructing three feature layers with different resolutions: L0 (high semantic low resolution), L1 (medium resolution), and L2 (high resolution low semantic). The L0 layer is used to coarsely locate the global direction of the curve, and the L2 layer is used to finely restore the edge details of the curve.

[0036] S2. Based on structural guidance information (i.e., preset curve priors), determine the linear / curved structure regions to be repaired on feature maps at multiple levels, and extract the corresponding regions of interest (ROI) features. The structural guidance information is used to define the processing area for image enhancement. Preferably, S2 specifically includes: initializing a set of learnable curve priors at layer L0. The curve priors are learnable straight lines or rays (representing the initial position of the curve in the image), used to define the sampling path or region aggregation window of the linear / curved structure regions, so as to extract the texture information to be repaired on multi-level feature maps. For each curve prior, the ROI aggregation module extracts feature vectors along the path of the curve prior on the feature maps at each level, obtaining a feature matrix containing the texture information of the curve path.

[0037] The specific operations for region of interest (ROI) feature extraction and primary aggregation are as follows.

[0038] First, prior initialization is performed. At the L0 layer, a set of learnable "curve priors" (i.e., pre-defined straight lines or rays) are initialized to represent the initial positions where the curves may exist in the image.

[0039] Then, the system performs Region of Interest (ROI) feature extraction. For each curve prior, the system uses the ROIGather module to extract the corresponding feature vector along the path of the curve in the current feature map (L0, L1, L2). This reduces the feature processing of the entire image to local processing only for the region near the curve, improving computational efficiency. The resulting feature matrix... (Explanation follows) It contains texture information on the curved path, but there may be signal breaks due to occlusion.

[0040] S3. Spatial filtering of the region of interest features is performed using a large-scale convolutional filter to achieve global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features.

[0041] The ROI features output by S2 are limited by their local receptive field and may contain invalid noise (such as areas obscured by vehicles ahead). This embodiment applies large kernel convolution (such as 1×9 and 9×1) to perform another spatial filtering on the ROI features. This step forcibly expands the physical receptive field of the features, enabling them to "see" the effective texture upstream and downstream of the occluded area. The algorithm uses the signal from the unoccluded area to interpolate and in-paint the features of the occluded area, repairing the broken structural signals in the feature space.

[0042] Existing techniques for pixel feature extraction before incorporating large kernel convolution primarily rely on 3×3 convolutions or standard convolutions to aggregate features. Assume the feature map is... Standard convolution operations can be represented as local neighborhoods. Weighted sum within.

[0043] .

[0044] In the formula Indicates position Features after convolution. For convolution kernels (or filters) at specific locations The weight parameters at that location. Location in the feature map The characteristics of the location. Location in the feature map The characteristics of the location.

[0045] like The range is very small, when When it is in an occluded area (pixel value is noise or invalid texture), its neighborhood The content inside is often invalid information, leading to poor output. Unable to recover image features.

[0046] To achieve image texture restoration, this embodiment introduces large-kernel convolution. To reduce computational complexity and enhance the feature flow in orthogonal directions (horizontal / vertical), this embodiment designs it as a large-scale decomposed convolution.

[0047] First, large-scale depthwise separable convolutions are used to filter the features of the region of interest. Specifically, the large-scale depthwise separable convolutions consist of vertically oriented... ×1 convolution kernel 1× in the horizontal direction convolution kernel Cascaded configuration, It is an integer greater than 3.

[0048] Assume the original region of interest features are: This embodiment defines two orthogonal large-scale one-dimensional convolution kernels. (vertical direction) ×1) and (Horizontal direction 1×) Preferred, .

[0049] The core formula is: In the formula This represents the global context features obtained after large-scale depth-separable decomposition convolutional filtering. Features of the original region of interest.

[0050] and This represents a large-scale pixel aggregation operation performed in both the vertical and horizontal directions of an image. Essentially, it establishes a long-distance "signal transmission channel" in the vertical direction of the image. Physically, vertical convolution can extend along a long vertical direction, crossing occlusions, and "transport" clear image textures from upstream and downstream to the occluded pixel locations.

[0051] This refers to the size of the spatial filtering window in image processing. It can be increased... The calculation of a single pixel is no longer limited to the immediate 3×3 grid, but establishes a direct mathematical relationship with the background information in the distant part of the image (such as dozens of pixels away).

[0052] Then, the global context features extracted after filtering are residually fused with the original region of interest features to obtain the repaired features.

[0053] Specifically, in order to construct a global context-aware feature map In this embodiment, a residual fusion structure is used to inject the long-range features extracted by the large kernel into the original features.

[0054] .

[0055] In the formula These are the features after the repair. The fusion coefficient is a learnable coefficient. This is the activation function. For batch normalization processing. To enhance the signal. This represents the global context features obtained after large-scale depth-separable decomposition convolutional filtering. Features of the original region of interest.

[0056] Learnable fusion coefficients This is used to control the degree to which global context information modifies local details. When local textures are missing, the model automatically increases the weight of this item, utilizing... Repair the broken image structure.

[0057] The addition operation (+) in the code is not a simple superposition, but rather a signal enhancement. Original region of interest features. It retains local details (such as edge sharpness), while It provides an overview of the macro-structural trend. Weighting Dynamically adjust the intensity of the "repair" in areas with clear texture. Smaller, in areas of occlusion and blurring It automatically enlarges, thereby achieving intelligent image structure repair.

[0058] This embodiment addresses the problem of insufficient effective receptive field caused by stacking small convolutional kernels in existing systems by introducing large-size convolutional kernels at the head of the feature extraction network. The image processing mechanism uses large-kernel convolution to directly expand the pixel range covered by a single convolution operation at the physical level, forcibly establishing direct connections between distant pixels in the image. Technical effect: This processing method endows the model with the ability of "image in-painting". Even if the middle of a curve is completely occluded (texture loss), this module can still capture the linear texture features upstream and downstream of the occluded area, using global structural information to "cross" the occluded area, inferring and recovering the image structure of the missing part, thus solving the problem of feature map fragmentation.

[0059] S4. Based on the pixel-level self-attention mechanism, geometric feature interaction and smoothing filtering are performed on the sequence of curve sampling points represented by the repaired features to suppress high-frequency geometric noise and obtain smoothed features.

[0060] Although the features repaired by S3 are connected, they may contain local noise. In this embodiment, the sampling point features of a single curve are regarded as a topological sequence, and the pixel dependency matrix within the sequence is calculated to quantify the geometric correlation between points.

[0061] S4's in-line self-attention-based geometric feature interaction mechanism is a geometric smoothing filter operation. It forces strong features from sharp regions in the image to flow to blurred regions, uses internal geometric constraints to correct outliers, eliminates high-frequency jitter (artifacts), and ensures that the output features are topologically smooth and continuous.

[0062] In the curve-based self-attention feature interaction mechanism, the Region of Interest (ROIGather) module in existing curve-structured image processing models (CLRNet, Curve Lane Detection Network), while using an attention mechanism, calculates the relationship between "ROI features" and "overall image features." The expression is: In the formula This is the weight matrix. The sequence of curve sampling points characterizing the restored features. Features of the entire image. This represents the number of channels. This indicates transpose. It is a normalized exponential function.

[0063] For a single curve sampling point sequence Existing methods typically use fully connected (FC) layers for each sampling point during regression. Handled independently. This means... The adjustment does not refer to the sampling point or sampling point This point-wise independence leads to an unsmooth output geometry.

[0064] Therefore, this embodiment introduces intra-lane pixel-wise self-attention, which will be applied to a single curve. Each sampled point feature is treated as a sequence of image pixels, and the interdependencies among them are calculated. This represents the number of sampling points (e.g., 72).

[0065] Specifically, the single curve Each sample point feature is treated as a topological sequence, and a query matrix is ​​generated through linear transformation. Key matrix Sum matrix Assuming This is the ROI feature matrix corresponding to a single curve. In this embodiment, the query matrix is ​​first generated using three linear transformation matrices. Key matrix Sum matrix .

[0066] .

[0067] .

[0068] .

[0069] In the formula To query the linear transformation matrix. Let be the linear transformation matrix of the key. It is a linear transformation matrix. For querying the matrix. It is the key matrix. It is a value matrix. This is the ROI feature matrix corresponding to a single curve.

[0070] Then calculate the pixel dependency matrix. Define the pixel dependency matrix as... , indicating the first The point and the first Geometric correlation strength between points:

[0071] .

[0072] In the formula Let be the pixel dependency matrix, representing the first... The point and the first The strength of the geometric correlation between points. It is a normalized exponential function. For the query matrix Row vectors. The first key matrix Row vectors. is the dimension of the key vector. This indicates transpose. The first key matrix Row vectors. Natural exponential function.

[0073] By using the dependency matrix to weight and aggregate the values ​​of all points, a geometrically corrected feature sequence is generated. .

[0074] .

[0075] In the formula This is the ROI feature matrix corresponding to a single curve. This is the adjustment coefficient. This is the pixel dependency matrix. It is a value matrix.

[0076] It is an N×N matrix that quantifies the relationship between any two pixels on a curve. For example, if the points at the bottom of the image (sharp) and the points in the middle of the image (blurred) follow the same straight line trend, then... The value will be very large.

[0077] This matrix multiplication operation represents the physical flow of features. It forces points in blurred regions (weak features) to "absorb" information from points in sharp regions (strong features). For each point on the curve, the algorithm no longer relies solely on its own features, but instead references the weighted features of all other points on the entire curve.

[0078] Through this interaction, the "weak features" located in the blurred region are corrected by the "strong features" in the clear region. Mathematically, this is equivalent to performing a low-pass filter on the fitted point sequence, effectively filtering out high-frequency jitter noise that violates geometric continuity, ensuring that the restored curve is smooth and continuous in its topological structure.

[0079] Through this full-sequence interaction, the final feature of the i-th point no longer depends solely on the pixel value of its own location, but rather on the weighted average of the features of all points along the entire line. Mathematically, this is equivalent to performing a non-local geometric filtering and smoothing on the curve, thereby eliminating outlier noise in the output image and ensuring the continuity of the curve.

[0080] To address the high-frequency geometric noise (artifacts) generated by independent regression of discrete points, this embodiment will use a single curve... Each sampling point is considered a topological sequence. A self-attention mechanism is used to calculate the geometric dependencies between points, performing non-local feature smoothing filtering. The image processing mechanism utilizes a self-attention algorithm to calculate a dependency matrix within the sequence. This matrix quantifies the feature correlation between each sampling point and all other sampling points in the image. Based on this weight matrix, each point can aggregate feature information from other locations along the entire curve.

[0081] S4 achieves complementary enhancement of image features. Strong texture features located in sharp areas near the image can be propagated to distant or blurred areas through an attention mechanism to correct weak features. This internal geometric constraint ensures that the output point sequence remains highly smooth and continuous in the image topology, eliminating high-frequency noise.

[0082] S5. Reconstruct the image texture of the smoothed features to generate an enhanced image containing complete curve texture. A progressively refined reconstruction is performed between multi-level feature maps, using the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, thereby improving the continuity and clarity of the linear / curved texture structure in the enhanced image. Specifically, geometric decoding is performed on the smoothed features to output the geometric parameters of the curve structure, and the curve prior is updated based on the decoding results for cascaded refinement, making the curve structure gradually approximate the real curve from coarse to fine.

[0083] Preferably, S5 specifically involves: inputting the smoothed curve features into the decoding layer, outputting the enhanced image, and an enhancement weight map and orientation guidance field for adaptive fusion of the enhancement results. Specifically, the smoothed curve features are input into the fully connected layer for decoding, outputting the probability of the existence of the curve structure, the coordinates and angle of the curve's starting point, and an offset for fine-tuning the prior shape of the curve.

[0084] The progressively refined reconstruction is as follows: starting from the L0 layer feature map, the structural distribution features (i.e. geometric parameters) of the decoded output are passed to the L1 layer feature map for feature fusion and enhancement, and then the output of the L1 layer is passed to the L2 layer feature map for further detail restoration, thus achieving iterative optimization from coarse to fine.

[0085] Specifically, the features from each layer, after global repair and smoothing, are fed into fully connected (FC) layers for decoding. The key geometric parameters of the network output curve include the probability of existence (whether there is a curve at that location), the coordinates and angle of the starting point, and fine-tuning of the prior shape.

[0086] During decoding, a cascaded refinement loop is performed, refining the low-resolution layer using the output of the high-semantic low-resolution layer. Specifically, the output of layer L0 is used as a new "prior" and passed to layer L1, repeating steps 2-5, and then to layer L2. Through this "coarse-fine" iteration, the actual physical curve is gradually approximated.

[0087] The specific process of the cascaded refinement is as follows: the decoding result of the L0 layer is used as a new curve prior and passed to the L1 layer to repeat steps S2 to S5. Then, the decoding result of the L1 layer is used as a new curve prior and passed to the L2 layer to repeat steps S2 to S5. The real physical curve is approximated through the "coarse-fine" iteration.

[0088] During the training phase, the difference between the enhanced image and the target image is constrained based on a spatially variable tolerance loss function. Specifically, a dynamic tolerance radius is determined based on the normalized vertical coordinates of the sampling positions. Based on this dynamic tolerance radius, the reconstruction error between the enhanced image and the target image is calculated within the corresponding window, and the spatially variable tolerance loss is obtained by summing the results over multiple sampling positions. Preferably, during the training phase, the error between the predicted curve and the true curve is calculated based on the spatially variable tolerance loss function, and the calculation tolerance for each sampling point is dynamically adjusted according to the image depth.

[0089] Specifically, during the training phase, in order to calculate the difference between the restored result and the true value, the system no longer uses a fixed radius. Instead, it calculates the dynamic tolerance radius in real time based on the image ordinate of each sampling point (the formula is shown below).

[0090] Loss calculation is based on constructing predicted and true line segments using dynamic radii, and calculating their intersection-over-union (IoU) loss. A tolerance is used at the bottom of the image (near view) and a narrow tolerance is used at the top of the image (far view). This forces the network to focus on optimizing the localization accuracy in the far view region during backpropagation, thereby correcting geometric distortions caused by perspective effects.

[0091] The image data obtained after repair and updating by each layer of the decoding head is then fed into the next layer of interest. Each layer then trains the next image based on the obtained loss function. The L0, L1, and L2 features of the three-layer feature pyramid are iteratively updated, and finally the training of the model is completed, resulting in an enhanced and restored curve structure image method and system.

[0092] By passing optimized feature maps and dynamically adjusted loss weights layer by layer, the network's ability to perceive distant regions is further enhanced. The final segmentation output maintains the continuity of the overall structure while significantly improving the accuracy of distant boundary localization, effectively mitigating misjudgment caused by perspective compression.

[0093] During training, the loss function (LOSS) calls the IoU function. In the IoU function, current techniques use a fixed scalar radius. (e.g., 15 pixels) to calculate the predicted texture energy center location. and the location of the texture energy center of the target In the Effective pixel response range of each sampling point and :

[0094] .

[0095] .

[0096] In the formula To predict image texture in the first... The effective pixel response range of each sampling point. For the target image texture in the first The true pixel distribution range of each sampling point.

[0097] This embodiment makes a key modification to the data flow and computation logic of the prior art: transitive coordinate dependency. The loss function is modified so that when calling the LineIoU loss function, additional coordinate dependencies are passed in. (i.e., curve-first detection) coordinate vector. Because The vector contains the positional information (y-coordinate) required for "spatial variability" and must be passed to the core of the SV-LineIoU calculation. Internally, the SV-LineIoU function replaces the "fixed scalar" with a "dynamic vector." To achieve a variable radius in space:

[0098] First, obtain the normalized vertical coordinates of each sampling point. .

[0099] Specifically, define a key coordinate vector. This is a 1-dimensional vector with a length of (e.g., 72). It stores the distance from the bottom (1.0) of the image to the top (0.0). Normalized vertical coordinates The SV-LineIoU function receives... Vector (of shape [N], e.g.,

[72] ).

[0100] Then, calculate the first... Dynamic tolerance radius of each sampling point .

[0101] .

[0102] In the formula It is the calculated number of The dynamic tolerance radius of each sampling point. It is a basic radius scalar (e.g., 15 pixels). For normalized vertical coordinates, representing The first in the vector Each element is a normalized vertical coordinate (0.0 for the top and 1.0 for the bottom). It is a configurable base scaling factor (e.g., 0.5). It is a configurable dynamic scaling factor (e.g., 1.0).

[0103] Specifically, two new hyperparameters are read from the configuration file: the base scaling factor. (e.g., 0.5) and dynamic scaling factor (e.g., 1.0). And define a spatially variable function, such as a linear function, to compute the dynamic radius vector. .

[0104] Then, the enhanced image and the target image are constructed based on the dynamic tolerance radius.

[0105] Assume L=15. =0.5. =1.0.

[0106] At the bottom of the image ( ≈1.0), radius It is approximately 15 × (0.5 + 1.0) = 22.5 pixels.

[0107] At the top of the image ( ≈0.0), radius It is approximately 15 × (0.5 + 0.0) = 7.5 pixels.

[0108] It is a vector of shape [N] { , ..., }

[0109] Use this dynamic radius vector This is used to calculate the boundary between the enhanced image and the target image, replacing the original fixed radius scalar. .

[0110] .

[0111] .

[0112] In the formula To predict image texture in the first... The effective pixel response range of each sampling point. For the target image texture in the first The true pixel distribution range of each sampling point. This indicates that it is calculated based on the dynamic radius. This represents the predicted location of the texture energy center. The target is the location of the texture energy center.

[0113] The predicted image texture calculated based on the dynamic radius is represented in the first... The effective pixel response range of each sampling point. The target image texture calculated based on the dynamic radius is represented in the first... The true pixel distribution range of each sampling point.

[0114] Next, calculate the intersection and union of the images.

[0115] Specifically, redefine the intersection. Union The form of the formula remains the same, but the content has changed.

[0116] .

[0117] .

[0118] In the formula The intersection. It is a union. To take the smaller value. To take the larger value. This represents the intersection calculated based on the dynamic radius. This represents the union calculated based on the dynamic radius.

[0119] The final spatial variable tolerance loss (LOSS) is calculated as follows.

[0120] LOSS= .

[0121] In the formula This represents the number of sampling points.

[0122] To address the issue of fixed radii violating perspective principles, this solution constructs a dynamic error calculation model based on the image's ordinate. The image processing mechanism involves converting the original scalar radius... Replace with dynamic vector The system reads the normalized ordinate of each pixel in real time. The computational tolerance at that location is dynamically generated through a linear mapping function.

[0123] This mechanism implements an adaptive constraint of "wider tolerance for near objects and narrower tolerance for distant objects" in the loss calculation stage of image processing. A larger tolerance is assigned to the bottom (foreground) of the image, and a very small tolerance is assigned to the top (background). This forces the image processing model to learn more refined pixel localization features in the background region, thereby significantly reducing the quantization error caused by perspective distortion and improving the geometric restoration accuracy of the entire image.

[0124] This embodiment's curve-structure image enhancement method based on the interaction of filtering and geometric constraints significantly improves the structural integrity (de-occlusion / completion) of the enhanced image. Compared to baseline methods, this embodiment utilizes the spatial filtering characteristics of large-kernel convolution to effectively solve the signal loss problem in complex scenes. Experiments show that this method can successfully restore continuous texture structures from occluded degraded images, significantly reducing the structural breakage rate and achieving complete reconstruction of the physical morphology.

[0125] The curve structure image enhancement method in this embodiment can effectively suppress (denoise / smooth) geometric artifacts and noise. By introducing pixel-level self-attention interaction, a strong internal geometric constraint is established. This mechanism acts like an intelligent smoothing filter, effectively filtering out high-frequency jitter and outlier noise during the reconstruction process, resulting in smooth and natural curve edges after reconstruction, and eliminating non-physical visual artifacts.

[0126] The curve structure image enhancement method in this embodiment can accurately correct (high-fidelity restore) perspective distortion. The spatially variable tolerance calculation model strictly follows the optical principles of physical imaging, eliminating quantization errors in distant areas. This allows the restored image structure to maintain extremely high geometric positioning accuracy across the entire depth range, achieving true high-fidelity image restoration.

[0127] Example 2: The present invention provides a curve structure image enhancement device based on the interaction of filtering and geometric constraints, which includes an encoding module, a region module, a spatial filtering module, a smoothing filtering module and a decoding module.

[0128] The encoding module is used to acquire the degraded image to be processed, and to perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels.

[0129] The region module is used to determine the linear / curved structure regions to be repaired on feature maps at multiple levels based on structural guidance information, and to extract the corresponding region of interest features.

[0130] The spatial filtering module is used to perform spatial filtering on the features of the region of interest using a large-scale convolutional filter, thereby achieving global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features.

[0131] The smoothing filter module is used to perform geometric feature interaction and smoothing filtering on the sequence of curve sampling points represented by the repaired features based on a pixel-level self-attention mechanism, so as to suppress high-frequency geometric noise and obtain smoothed features.

[0132] The decoding module is used to reconstruct the image texture of the smoothed features, generating an enhanced image containing complete curve textures. It performs progressively refined reconstruction across multiple levels of feature maps, using the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, thereby improving the continuity and clarity of linear / curved texture structures in the enhanced image.

[0133] It is understood that the image enhancement device may be an electronic device with computing power, such as a laptop computer, desktop computer, server, smartphone or tablet computer.

[0134] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0135] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0137] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks. It should be noted that in this embodiment, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0138] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0139] It should be understood that the term "and / or" used in this embodiment is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, in this embodiment, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0140] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0141] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0142] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing curve-structured images based on the interaction of filtering and geometric constraints, characterized in that, Include: S1. Obtain the degraded image to be processed, and perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels. S2. Based on structural guidance information, determine the linear / curved structure regions to be repaired on feature maps at multiple levels, and extract the corresponding regions of interest features. S3. Spatial filtering of the region of interest features is performed using a large-scale convolutional filter to achieve global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features. S4. Based on the pixel-level self-attention mechanism, geometric feature interaction and smoothing filtering are performed on the sequence of curve sampling points represented by the repaired features to suppress high-frequency geometric noise and obtain smoothed features. S5. Reconstruct the image texture of the smoothed features to generate an enhanced image containing complete curve texture; perform progressive refinement reconstruction between multi-level feature maps, and use the structural guidance features output from the previous level to guide feature fusion and detail restoration in the next level, so as to improve the continuity and clarity of the linear / curved texture structure in the enhanced image. S3 specifically includes: Large-scale depthwise separable convolutions are used to filter the features of the region of interest; wherein, the large-scale depthwise separable convolutions consist of vertically oriented... ×1 convolution kernel 1× in the horizontal direction convolution kernel Cascaded configuration, It is an integer greater than 3; In the formula The global context features are obtained after large-scale depth-separable decomposition convolutional filtering. Features of the original region of interest; The filtered global context features are residually fused with the original region of interest features to obtain the repaired features; The expression for residual fusion is as follows; ； In the formula Features after repair; The learnable fusion coefficient; For activation functions; For batch normalization processing; To enhance the signal; S4 is detailed below; Single curve Each sample point feature is treated as a topological sequence, and a query matrix is ​​generated through linear transformation. Key matrix Sum matrix ; Then calculate the pixel dependency matrix: In the formula The pixel dependency matrix; It is a normalized exponential function; For the query matrix Row vectors; The first key matrix Row vectors; The dimension of the key vector; Indicates transpose; By using the dependency matrix to weight and aggregate the values ​​of all points, a geometrically corrected feature sequence is generated. ; In the formula This is the ROI feature matrix corresponding to a single curve; This is the adjustment coefficient.

2. The method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to claim 1, characterized in that, S5 is detailed below; The smoothed curve features are input into the decoding layer, which outputs the enhanced image, as well as an enhancement weight map and a direction guidance field for adaptive fusion of the enhancement results. The progressively refined reconstruction is as follows: starting from the L0 layer feature map, the structural distribution features of the decoded output are used as guiding information to be passed to the L1 layer feature map for feature fusion and enhancement, and then the output of the L1 layer is passed to the L2 layer feature map for further detail restoration, thus achieving iterative optimization from coarse to fine.

3. A method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to any one of claims 1 to 2, characterized in that, During the training phase, the difference between the enhanced image and the target image is constrained based on the spatially variable tolerance loss function. Specifically, the dynamic tolerance radius is determined according to the normalized longitudinal coordinates of the sampling position, and the reconstruction error between the enhanced image and the target image is calculated within the corresponding window based on the dynamic tolerance radius. The spatially variable tolerance loss is obtained by summing the results over multiple sampling positions.

4. The method for enhancing curve structures based on the interaction of filtering and geometric constraints according to claim 3, characterized in that, The calculation process of the spatially variable tolerance loss function is as follows; Obtain the normalized vertical coordinates of each sampling point ; Calculate the first Dynamic tolerance radius of each sampling point; In the formula It is the calculated number of Dynamic tolerance radius of each sampling point; It is a basic radius scalar; It is a configurable base scaling factor; It is a configurable dynamic scaling factor; Constructing enhanced and target images based on dynamic tolerance radius; ； ； In the formula To predict image texture in the first... Effective pixel response range of each sampling point; For the target image texture in the first The true pixel distribution range of each sampling point; This is based on calculations using the dynamic radius; The predicted location of the texture energy center; The target is the location of the texture energy center; Calculate the intersection and union of the images; ； ； In the formula For intersection; It is a union; To take the smaller value; To take the larger value; Calculate the spatially variable tolerance loss (LOSS); LOSS = In the formula This represents the number of sampling points.

5. A method for enhancing curve-structured images based on the interaction of filtering and geometric constraints according to any one of claims 1 to 2, characterized in that, S1 specifically involves: acquiring the degraded image to be processed and using ResNet as the backbone network to extract basic features; then fusing deep and shallow features through a feature pyramid network to generate feature maps at three different resolution levels: L0, L1, and L2. Among them, the L0 layer is a high-semantic, low-resolution feature map used to extract global contextual representations of linear / curved structures to guide subsequent enhancement reconstruction and structural continuity restoration; the L1 layer is a medium-resolution feature map; and the L2 layer is a high-resolution, low-semantic feature map used to finely restore the edge details of the curves.

6. The method for enhancing curve structures based on the interaction of filtering and geometric constraints according to claim 5, characterized in that, The structural guidance information is used to define the processing area for image enhancement; S2 specifically includes: Initialize a set of learnable curve priors at L0 layer; the curve priors are learnable straight lines or rays used to define the sampling path or region aggregation window of the linear / curved structure region so as to extract the texture information to be repaired on the multi-level feature map. For each curve prior, the feature vector is extracted from the feature map at each level along the path of the curve prior by the region of interest aggregation module, resulting in a feature matrix containing the texture information of the curve path.

7. A curve structure image enhancement device based on the interaction of filtering and geometric constraints, characterized in that, Suitable for performing the curve structure image enhancement method based on the interaction of filtering and geometric constraints as described in any one of claims 1 to 6; The curve structure image enhancement device includes: The encoding module is used to acquire the degraded image to be processed, and to perform multi-scale feature encoding on the degraded image through a backbone network and a feature pyramid network to obtain feature maps of multiple levels. The region module is used to determine the linear / curved structure regions to be repaired on feature maps at multiple levels based on structural guidance information, and to extract the corresponding region of interest features. The spatial filtering module is used to perform spatial filtering on the features of the region of interest using a large-scale convolutional filter, thereby achieving global context-aware processing to repair texture breaks caused by occlusion and obtain the repaired features. The smoothing filter module is used to perform geometric feature interaction and smoothing filtering on the sequence of curve sampling points represented by the repaired features based on the pixel-level self-attention mechanism, so as to suppress high-frequency geometric noise and obtain smoothed features. The decoding module is used to reconstruct the image texture of the smoothed features and generate an enhanced image containing complete curve texture. It performs progressive refinement reconstruction between multi-level feature maps and uses the structural guidance features output from the previous level to guide the feature fusion and detail restoration of the next level, so as to improve the continuity and clarity of the linear / curved texture structure in the enhanced image.

Images