Adaptive image enhancement and layout geometry reconstruction method for engineering configuration table

Abstract

The present application relates to the field of image processing, and more particularly to a kind of adaptive image enhancement and layout geometry reconstruction method for engineering configuration table, comprising obtaining the photograph image of engineering configuration table;Photograph image is input into image quality assessment network IQAN, and the quality vector that quality state of photograph image is characterized is output;Quality vector is input into adaptive enhancement strategy selection network Policy-Net, and at least one enhancement operation type and its corresponding operation parameter are output;Table boundary detection is carried out to enhanced image, and table four corner point extraction is carried out, and table four corner point coordinates are obtained;Perspective transformation matrix is solved according to table four corner point coordinates;Edge defect repair and local structure completion are carried out to layout geometry reconstruction image, and the image of engineering configuration table after regulation is output.The present application realizes adaptive enhancement and geometry reconstruction and structure completion of engineering configuration table image, significantly improves readability, structure regulation degree and recognition accuracy and reduces artificial cost.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to an adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables. Background Technology

[0002] During on-site management of engineering construction, equipment installation, commissioning, and acceptance, engineering configuration tables (such as equipment parameter tables, material lists, wiring / terminal configuration tables, circuit / point tables, etc.) are often circulated on paper or posted. Actual data collection largely relies on taking photos with mobile phones or portable terminals for archiving, retrieval, and table structure recognition in the backend system. Due to the complex shooting environment, configuration table images commonly suffer from degradation phenomena such as uneven lighting, shadow occlusion, low resolution, noise and blurring, tilt and perspective distortion, and edge cropping defects. This results in unclear table lines, unclosed cell boundaries, and obscured or distorted table headers and field areas, significantly reducing the stability of subsequent table structure analysis and field recognition, and even making it impossible to reliably locate table areas and key structural elements.

[0003] To address the aforementioned issues, existing technologies typically employ fixed or human-experience-driven image enhancement and geometric correction processes. For example, common practices in image enhancement include global / local histogram equalization, fixed gamma correction, linear adjustment of contrast and brightness, filtering for noise reduction, and sharpening. Geometric correction often employs edge / line detection (such as Hough transform) to estimate tilt angles, or perspective correction by fitting a homography matrix after detecting table boundaries. For cropped or occluded areas, general image inpainting or simple outward filling strategies are often used. The above methods can provide some improvement in scenarios with simple image degradation types and limited quality changes, but they still have significant shortcomings in real-world engineering data with "multi-factor superimposed degradation": First, fixed enhancement parameters are difficult to take into account different image quality distributions, and are prone to over-enhancement causing noise amplification, text breakage, or table line artifacts, or insufficient enhancement resulting in lines and text remaining indistinguishable; Second, traditional geometric correction is highly sensitive to the accuracy of table boundary and corner detection, and is prone to corner drift and boundary omissions in cases of shadows, reflections, creases, occlusions, and missing lines, leading to unstable perspective matrix estimation and problems such as non-parallel lines and distorted cell proportions after correction; Third, general repair methods usually lack prior structural constraints on table row and column lines, making it difficult to guarantee the continuity of lines and the topological consistency of cells after completion, thus affecting the reliability of subsequent structural inference.

[0004] Furthermore, with the increasing application of multimodal large models in table recognition and structural understanding, the regularity and structural consistency of the front-end input image have a more significant impact on the overall recognition effect. If the front-end enhancement and correction cannot adaptively adjust according to image quality, or cannot stably output a unified table layout with "approximately parallel lines, regular cells, and recoverable missing structures," it will be difficult to fully leverage the structural reasoning capabilities of the back-end model. Therefore, there is an urgent need for a front-end processing method for engineering configuration table scenarios that can adaptively select enhancement strategies based on image quality and combine key structural elements to achieve robust perspective correction and structural awareness completion, so as to output a unified, regular, and high-quality configuration table image, providing reliable input for subsequent table structure parsing and field recognition. Summary of the Invention

[0005] Based on the above description, the present invention provides an adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables, which realizes adaptive enhancement, geometric reconstruction and structural completion of engineering configuration table images, significantly improving readability, structural regularity and recognition accuracy and reducing labor costs.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: An adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables, comprising the following steps:

[0007] S01 acquires a photographic image of the project configuration table;

[0008] S02 inputs the captured image into the Image Quality Assessment Network (IQAN) and outputs a quality vector representing the quality state of the captured image.

[0009] S03 inputs the quality vector into the adaptive enhancement strategy selection network Policy-Net, outputs at least one enhancement operation type and its corresponding operation parameters, and performs adaptive enhancement on the captured image according to the enhancement operation type and the operation parameters to obtain an enhanced image;

[0010] S04 performs table boundary detection on the enhanced image and extracts the four corner points of the table to obtain the coordinates of the four corner points of the table.

[0011] S05 calculates the perspective transformation matrix based on the coordinates of the four corner points of the table, and uses the perspective transformation matrix to perform perspective correction on the enhanced image to obtain a geometrically reconstructed image of the page layout;

[0012] S06 performs edge defect repair and local structure completion on the geometric reconstruction image of the layout, and outputs a normalized engineering configuration table image.

[0013] Through the aforementioned technical solution, a closed-loop process of "quality assessment → adaptive strategy enhancement → table positioning and four-corner point determination → homography perspective correction → defect completion" is implemented to achieve end-to-end normalization processing of photographed images of engineering configuration tables. Compared to solutions that only perform fixed parameter enhancement or perspective correction, this method can still output more readable and structurally regular table images even when uneven lighting, noise, blur, tilt, and perspective distortion coexist. This significantly reduces the cost of reshooting and manual image retouching, and provides a unified input for subsequent table structure analysis / field recognition, thereby improving recognition stability and engineering deployability at the system level.

[0014] Based on the above technical solution, the present invention can be further improved as follows.

[0015] Furthermore, the image quality assessment network IQAN performs quality assessment on the images taken from the input engineering configuration table and outputs a quality vector. The quality vector includes at least one or more of the following dimensions: sharpness index, noise level, illumination uniformity, tilt angle estimate, and an indicator of whether there is significant perspective distortion.

[0016] The IQAN is a CNN or lightweight Transformer-based network that uses a multi-head output method to output sharpness, noise, illumination uniformity, and tilt angle using regression heads, and to output perspective distortion indicators using a binary classification head.

[0017] Before inputting into IQAN, at least one preprocessing step is performed on the captured image: size normalization, luminance / grayscale channel extraction, and pixel value normalization.

[0018] The above technical solution explicitly encodes dimensions such as sharpness, noise, illumination uniformity, tilt angle, and perspective distortion indicators into quality vectors. A multi-head output (regression head + binary classification head) is employed to achieve "interpretable and decomposable" quality assessment, more accurately distinguishing "what to enhance and to what extent." Combined with size normalization and channel / pixel normalization preprocessing, distribution drift caused by different shooting devices and resolutions is reduced, making quality estimation more stable. This reduces problems such as over-sharpening, over-denoising, or over-brightness exaggeration caused by misjudgments in the strategy network, improving the consistency and controllability of the enhancement results.

[0019] Furthermore, the Policy-Net network takes the quality vector as input and outputs the enhancement operation type and corresponding operation parameters; wherein:

[0020] The enhancement operation types include at least one or more of the following: adaptive brightness / contrast adjustment, Retinex illumination equalization, adaptive Gamma correction, bilateral filtering noise reduction, and sharpening enhancement;

[0021] The operating parameters include at least one or more of the following: Gamma value, filter radius, and weighting coefficient.

[0022] The preferred output format of Policy-Net is "discrete code of operation type + continuous value of parameter". The corresponding enhancement operator is selected according to the discrete code, and the enhancement is performed after configuring the parameters of the enhancement operator using the continuous value.

[0023] The enhancement process preferably supports multi-operator cascading: multiple enhancement operators and their parameters are output by Policy-Net in sequence and applied to the input image in turn to obtain the enhanced image;

[0024] Policy-Net is jointly trained with the large backend table recognition model or trained in two stages, using "recognition accuracy" as a reinforcement learning reward signal to make the output enhancement strategy more friendly to the backend recognition.

[0025] Through the above technical solutions, Policy-Net maps quality vectors to "discrete operator types + continuous parameters," enabling adaptive configuration of enhancement operators and parameters, avoiding the failure of traditional manual parameter tuning in complex scenes. It supports multi-operator cascading, allowing the system to process images in a sequence such as "illumination equalization, denoising, and sharpening," improving its ability to repair composite degraded images. Furthermore, it uses recognition accuracy as a reward for reinforcement learning or conducts joint / two-stage training with the backend, aligning the front-end enhancement target with the back-end recognition target, reducing the bias of "visually improved but recognition worsened," thereby increasing the upper limit of field F1 and structural parsing accuracy.

[0026] Furthermore, the table boundary detection includes: using a table region detection network based on FCN and / or YOLO to detect the enhanced image and output at least one table region candidate box; and preferably performing candidate box filtering on multiple candidate boxes to determine the target candidate box, wherein the candidate box filtering includes at least one of the following: selecting according to the principle of maximum confidence, selecting according to the principle of maximum candidate box area, or performing non-maximum suppression on the candidate boxes before selection.

[0027] The above technical solution employs detection networks such as FCN / YOLO to first determine candidate bounding boxes for the table region. Then, target bounding boxes are filtered using confidence / area principles or NMS, effectively eliminating interfering areas such as background text, binding holes, and shadow edges. This significantly reduces the search space and false detection rate for subsequent corner detection and homography estimation. This segmented design of "localization first, geometry second" is beneficial for real-time performance and stability in scenarios with limited computing power: on the one hand, it reduces artifacts caused by irrelevant regions participating in enhancement and reconstruction; on the other hand, it improves the success rate of extracting target table boundaries and corners, making perspective correction more reliable and the output more regular.

[0028] Furthermore, the extraction of the four corner points of the table includes: constructing a key point detection sub-network to output probability heatmaps for the four corner points of the table, wherein the probability heatmaps are four-channel heatmaps, corresponding to the top left, top right, bottom right, and bottom left corner points respectively; performing non-maximum suppression on each channel heatmap within the target candidate box area to obtain the coordinates of the four corner points, and outputting them in the order of "top left - top right - bottom right - bottom left";

[0029] Set the corner confidence threshold When the maximum confidence of any corner channel within the target candidate box is less than Or, after sorting, the distance between any two adjacent corner points is less than a preset minimum distance threshold. Or, the area of ​​the quadrilateral formed by the four corner points is less than a preset ratio threshold of the area of ​​the target candidate box. When this happens, a backoff strategy is triggered: the four vertices of the target candidate box are used as the coordinates of the four corner points; where, , , These are configurable engineering parameters.

[0030] Through the above technical solution, the four-channel corner heatmap locates the top-left, top-right, bottom-right, and bottom-left corners respectively, and performs NMS within the candidate bounding box, which improves the corner location accuracy and noise resistance. Simultaneously, a joint verification is introduced using a corner confidence threshold τ, a minimum distance threshold between adjacent corners d_min, and a quadrilateral area ratio threshold η. When corner occlusion, reflection, edge defects, or broken grid lines cause the heatmap to be unreliable, a backtracking mechanism is triggered, replacing the corner output with the four vertices of the candidate bounding box. This mechanism significantly reduces the risk of "corner anomalies → homography matrix divergence → overall reconstruction failure," improves the availability of the end-to-end link, and provides configurable engineering parameters to adapt to different shooting distances and table formats.

[0031] Furthermore, the perspective transformation matrix is ​​a homography matrix. It is determined by the correspondence between the four corner points and the four vertices of the target rectangle, and satisfies the following mapping relationship: ;in To enhance pixel coordinates in an image, These are the pixel coordinates in the geometrically reconstructed image;

[0032] The output resolution of the target rectangle is determined according to at least one of the following rules:

[0033] Aspect Ratio Constraint Rule: The aspect ratio of the target candidate box is used as the aspect ratio of the target rectangle, and the maximum side length is within the preset limit. The width and height of the target rectangle are determined under constraints such that the target rectangle maintains the same or approximately the same aspect ratio as the target candidate box;

[0034] Line density / minimum cell pixel constraint rule: Before or during geometric reconstruction, perform line detection on the table's row and column lines. Estimate the number of rows and columns based on the number of detected horizontal and vertical grid lines, and set the target rectangle resolution to satisfy the condition that "the average cell width after reconstruction is not less than a preset pixel threshold". The average cell height is not less than the preset pixel threshold. The minimum resolution;

[0035] in, , , These are configurable engineering parameters.

[0036] By utilizing the above technical solution to solve the homography matrix H by matching the four corner points with the four vertices of the target rectangle, perspective distortion can be geometrically eliminated, making table lines more parallel and cell shapes closer to regular rectangles. This creates a more "regular" input for subsequent row and column line detection and structural analysis. Furthermore, aspect ratio constraints and maximum side length L_max limits prevent stretching deformation and resolution loss. Line density / minimum cell pixel thresholds w_min and h_min constraints ensure that reconstructed cells have sufficient pixels to carry character and line details, reducing interpolation blur and jagged edges, thereby improving readability and recognition accuracy, and meeting the computing power and memory limits of mobile / edge devices.

[0037] Furthermore, the edge defect repair and local structure completion are achieved through a structure-aware completion network. The input of the structure-aware completion network includes at least a geometrically reconstructed image and further includes a defect mask; wherein:

[0038] The defect mask is used to indicate edge-trimmed areas or locally missing areas, and the defect mask is generated by at least one of the following methods:

[0039] Invalid pixel region generation based on perspective-corrected image: mark the holes / invalid regions generated by perspective resampling as defective regions;

[0040] Based on the relationship between the main body of the table and the image boundary: when the main body of the table intersects with the image boundary or the distance is less than the preset boundary distance threshold, the preset width strip area adjacent to the boundary is marked as the defect area;

[0041] Morphological expansion or lateral expansion is performed on the initial defect area to cover the broken table lines and table header edges.

[0042] Set a defect trigger threshold. When the proportion of the defect mask coverage area to the geometric reconstruction image area is greater than a preset proportion threshold, or when the defect mask is connected to the outer boundary of the image, start the completion network; otherwise, skip the completion step and directly output the geometric reconstruction image.

[0043] The completion network learns the geometric priors of the table rows and columns, and combines local texture and global layout information to complete the missing edge lines and / or table header areas, outputting a completed project configuration table image. The completion network adopts an encoder-decoder structure, and preferably only performs cropping and inference on the areas containing the missing mask before pasting it back to the whole image, so as to reduce the amount of computation and reduce unnecessary modifications to the complete area.

[0044] The above technical solution generates defect masks based on invalid pixel holes, table-boundary relationships, and morphological expansion, which can more accurately locate problem areas such as "perspective resampling holes," "edge clipping bands," and "broken table lines / missing headers." Then, using the proportion of the defect area or its connection to the outer boundary as a trigger threshold, "complete only when necessary" is achieved, avoiding excessive rewriting of the complete image. The structure-aware completion network learns the geometric priors of rows and columns and integrates the global layout, making the completion result more consistent with the table topology. Simultaneously, defect region clipping inference and re-pasting are used to reduce computational load, decrease artifacts in non-defective areas, and improve engineering robustness and maintainability.

[0045] Furthermore, the training of the structure-aware completion network employs a loss function constrained by pixel reconstruction, structure awareness, and perceptual consistency, and:

[0046] The pixel reconstruction loss is used to constrain the pixel consistency within the completed area, and is preferably L1 or L2 reconstruction loss;

[0047] The structure-aware loss is constructed based on the table line detection results. Specifically, the table row and column line detection is performed on the completed output image to obtain a line structure map, and consistency constraints are applied between the labeled line structure map or the reference line structure map generated from the complete table to ensure that the completed row and column lines are continuous, the intersection points are stable, and the cell topology is consistent.

[0048] The perceptual loss is used to constrain the high-level semantic and texture consistency of the completed region, and is preferably obtained by calculating the difference in a multi-layer feature space by a pre-trained feature extraction network.

[0049] The total loss is the weighted sum of the losses mentioned above, and the weights of each item are configurable engineering parameters.

[0050] The above technical solution employs pixel reconstruction loss to ensure basic color / brightness consistency in the completed region; it introduces structure-aware loss based on table line detection results to ensure continuous row and column lines, stable intersections, and consistent cell topology after completion, directly addressing the key objective of "correct table structure"; furthermore, it constrains high-level semantics and texture consistency through perceptual loss, reducing unnatural phenomena such as "blurring" and "texture drift" in the completed region. The weighted sum of the three types of losses can be adjusted through engineering parameters to maintain a controllable convergence direction under different data noise and defect patterns, thereby improving the generalization ability and output reliability of the completion network and enhancing the gain for backend structure parsing.

[0051] Furthermore, the adaptive enhancement and layout geometry reconstruction module is collaboratively optimized with the backend multimodal table recognition model: the regularized engineering configuration table image output from the front end is used as the input of the multimodal table recognition model, and the combination of structure recognition accuracy and field recognition F1 value is used as the optimization target. The parameters of at least one module of the front end are updated through joint training or two-stage training so that the enhancement strategy and geometry reconstruction results are optimal for backend recognition.

[0052] The above technical solution allows the front-end standardized engineering configuration table image to directly serve the back-end multimodal table recognition model. Using a combination of structural recognition accuracy and field recognition F1 score as the optimization objective, front-end enhancement and geometric reconstruction are no longer solely driven by "visual appeal," but rather by "recognition usability." By updating front-end module parameters through joint training or two-stage training, the system can automatically learn which enhancement and reconstruction details best improve back-end parsing (e.g., suppressing reflections, preserving fine lines, and unifying cell scale), thereby improving the final deliverables from an overall pipeline perspective and reducing the manpower required for repetitive parameter tuning in different project / site environments.

[0053] Furthermore, the collaborative optimization includes reinforcement learning optimization of the policy selection network Policy-Net: the type of enhancement operator and operation parameters output by Policy-Net are regarded as actions, and the recognition accuracy corresponding to the back-end recognition result is used as the reward signal to update the policy of Policy-Net to achieve end-to-end optimization.

[0054] The reinforcement learning optimization and supervised learning optimization can be combined in at least one of the following ways: supervised pre-training followed by reinforcement learning fine-tuning, or alternating updates during joint training.

[0055] By treating the enhancement operator type and parameters as actions and using recognition accuracy as the reward signal for reinforcement learning updates, the causal chain of "enhancement strategy → recognition result" can be directly optimized. This approach is particularly suitable for solving strategy combination problems (operator order, parameter coupling, scene differences) that are difficult to cover with traditional supervision signals. Through "supervised pre-training followed by reinforcement fine-tuning" or "joint training and alternating updates," stable convergence in the early stages of training is ensured, while the strategy can continue to adaptively adjust itself when the distribution of real data shifts (device changes, lighting changes, paper material differences). This improves the system's long-term robustness and iterative efficiency in complex scenarios and reduces performance degradation caused by strategy solidification.

[0056] Compared with the prior art, the technical solution of this application has the following beneficial technical effects:

[0057] 1. The system uses quality assessment as a pre-driven mechanism, explicitly quantifying degradation factors such as sharpness, noise, illumination uniformity, and tilt / perspective, and adaptively selecting enhancement operators and continuous parameters (supporting cascaded multi-operator operations) accordingly. Therefore, under complex shooting conditions, it avoids common problems with fixed-parameter solutions such as "over-sharpening, noise reduction, overexposure, and loss of fine lines," outputting clearer, more balanced, and more text / line-fidelity-preserving table images. Simultaneously, it reduces sensitivity to device resolution and environmental changes, significantly lowering the cost of reshoots and manual retouching, providing consistent and controllable high-quality input for subsequent table structure analysis and field recognition.

[0058] 2. Through a geometric chain of "table detection and positioning → corner point estimation → homography matrix perspective correction," the tilt and perspective distortion caused by shooting are normalized into a regular rectangular layout, making the row and column lines more parallel and the cell shape more regular, fundamentally improving the feasibility and accuracy of structural analysis. Furthermore, verification and backoff strategies such as corner point confidence, minimum corner point distance, and quadrilateral area ratio are introduced, along with rigid engineering constraints such as aspect ratio, maximum side length, and minimum cell pixel size. This prevents homography divergence caused by abnormal corner points and avoids stretching deformation and resolution loss after correction, ensuring stable reproduction and scalable deployment of the output on mobile / edge devices.

[0059] 3. For defective areas such as perspective resampling holes, edge clipping bands, and broken lines, a structure-aware mask generation and threshold triggering mechanism is adopted to achieve "completeness only when necessary." Furthermore, through joint constraints of pixel consistency, table line structure consistency, and perceptual consistency, the completed results are made more consistent with the table topology (continuous lines, stable intersections, and reliable cell boundaries), while reducing artifact contamination in non-defective areas. Combined with joint / two-stage optimization with the backend table recognition model, and reinforcement learning strategy updates that reward recognition accuracy, the front-end processing no longer pursues subjective perception but directly maximizes structural accuracy and field F1 score, significantly improving the final deliverable recognition performance and cross-scene generalization ability. Attached Figure Description

[0060] Figure 1 This is a flowchart of an adaptive image enhancement and layout geometry reconstruction method for an engineering configuration table provided in Embodiment 1 of the present invention.

[0061] Figure 2 This is a general structural block diagram of an adaptive image enhancement and layout geometry reconstruction system for an engineering configuration table provided in Embodiment 2 of the present invention;

[0062] Figure 3 This is a schematic diagram of the IQAN image quality assessment network structure of the present invention;

[0063] Figure 4 This is a schematic diagram illustrating the cascaded execution of the Policy-Net enhancement strategy and operators of the present invention;

[0064] Figure 5 This is a schematic diagram of the table detection + ROI clipping + four-corner point heatmap and backoff constraint of the present invention;

[0065] Figure 6 This is a flowchart of the defect mask generation and trigger determination process of the present invention;

[0066] Figure 7 This is an overview diagram of the structure-aware completion network and training / cooperative optimization of the present invention. Detailed Implementation

[0067] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0069] Example 1:

[0070] In this embodiment, the object of processing is a photograph of the engineering configuration table. This photograph may contain issues such as uneven lighting, shadows, reflections, noise, blur, rotational tilt, perspective distortion, and edge cropping defects. For ease of description, the original input image is denoted as... The image after adaptive enhancement is denoted as The geometric reconstruction image of the page obtained after perspective correction is denoted as The image after edge defect repair and local structure completion is denoted as The final output image of the standardized project configuration table is denoted as... Meanwhile, the quality vector output by the image quality assessment network is denoted as... The target candidate boxes obtained by table detection are denoted as The set of coordinates of the four corner points of the table is denoted as The perspective transformation matrix is ​​denoted as the homography matrix. In engineering implementation, the corner confidence threshold... Minimum distance threshold Area ratio threshold Maximum side length Minimum cell pixel threshold and the threshold for the proportion of defects All parameters are set as configurable project parameters.

[0071] refer to Figure 1-7 An adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables includes the following steps:

[0072] S01 acquires a photographic image of the project configuration table;

[0073] S02 inputs the captured image into the Image Quality Assessment Network (IQAN) and outputs a quality vector representing the quality state of the captured image.

[0074] S03 inputs the quality vector into the adaptive enhancement strategy selection network Policy-Net, outputs at least one enhancement operation type and its corresponding operation parameters, and performs adaptive enhancement on the captured image according to the enhancement operation type and the operation parameters to obtain an enhanced image;

[0075] S04 performs table boundary detection on the enhanced image and extracts the four corner points of the table to obtain the coordinates of the four corner points of the table.

[0076] S05 calculates the perspective transformation matrix based on the coordinates of the four corner points of the table, and uses the perspective transformation matrix to perform perspective correction on the enhanced image to obtain a geometrically reconstructed image of the page layout;

[0077] S06 performs edge defect repair and local structure completion on the geometric reconstruction image of the layout, and outputs a normalized engineering configuration table image.

[0078] Preferably, step S1 involves acquiring a photographic image of the engineering configuration table and performing basic preprocessing. In step S1, the system acquires a photographic image of the engineering configuration table as input. To reduce the sensitivity of subsequent network inference to direction, scale, and numerical distribution, this step... Perform basic preprocessing to form standardized input. Specifically, prioritize orientation correction of the image based on image metadata (e.g., EXIF ​​orientation) to ensure that the table is visually in a conventional orientation with the top as the table header and the bottom as the table footer. If there is no metadata or the metadata is unreliable, a lightweight orientation discrimination strategy (e.g., projection features based on text line orientation) can be used for rotation correction or the original state can be maintained for subsequent steps.

[0079] Subsequently, the image size is normalized. Let the long side of the input image be in pixels. ,when Greater than the preset input long side threshold When (e.g., taking a value within the range of 2048-4096 pixels), the image is scaled proportionally to make the longer side meet the specified value. This controls computational load and memory usage; when If the image size is too small (e.g., less than 960-1280 pixels) and will significantly affect the resolution of the corner heatmap, the input branch of the network can be scaled up proportionally or super-resolution enhanced without changing the semantic content of the original image. To adapt to subsequent network inputs, this step further performs channel preparation and numerical normalization: firstly, the image can be converted from RGB to YCrCb / HSV and the luminance channel can be extracted. or Firstly, it serves as the preferred input for quality assessment and corner location; secondly, it simultaneously preserves the RGB image for subsequent direct processing of the color image by enhancement operators such as Retinex, Gamma, contrast, and sharpening. Finally, the pixels are mapped to... or The interval is then processed and normalized according to the training configuration to obtain the preprocessed tensor. It serves as the input basis for network modules such as S2 and S5.

[0080] Preferably, in step S2, the captured image is input into IQAN for quality assessment and a quality vector is output. In step S2, the preprocessed tensor obtained in step S1 is... Input the image quality assessment network IQAN to output a quality vector representing the quality state of the captured image. In this embodiment, IQAN can employ a network structure based on CNN or a lightweight Transformer, and use a multi-head output method to simultaneously cover multiple quality factors. Specifically, the backbone network of IQAN is used to extract global quality-related features, and then multiple output heads are set: regression heads are set to output continuous values ​​for sharpness, noise level, illumination uniformity, and tilt angle, respectively; a binary classification head is set to output perspective distortion probability or category indication for perspective distortion indicator, thereby forming a quality vector. .

[0081] To facilitate direct use of the policy network, this embodiment calibrates and normalizes each output dimension to ensure they uniformly fall within a certain range. Intervals: For example, the tilt angle regression value is normalized according to the maximum allowable angle, and the perspective distortion output is represented in the form of sigmoid probability. IQAN training can be completed offline, with training samples including real-world configuration table images and degraded samples generated through synthesis; these degradations include, but are not limited to, Gaussian noise and compression artifacts, motion / Gaussian blur, non-uniform lighting (shadow overlap, vignetting), brightness shift, rotational tilt, and perspective projection, thus enabling the network to be robust to complex on-site photography conditions. During the online inference phase, IQAN only outputs... This serves as the conditional input for subsequent adaptive reinforcement strategy decisions.

[0082] Preferably, in step S3, the mass vector is input into Policy-Net, which outputs the enhancement operator type and parameters, and adaptive enhancement is performed. In step S3, the mass vector obtained in step S2 is... The Policy-Net adaptive enhancement strategy selection network is input to output at least one enhancement operation type and its corresponding operation parameters, and adaptive enhancement is performed on the captured image accordingly to obtain the enhanced image. In this embodiment, the output of Policy-Net adopts the form of "discrete encoding + continuous parameters": the discrete encoding is used to indicate the type of enhancement operator, and the continuous parameters are used to indicate the specific parameter values ​​of that type of operator; the enhancement operator library includes at least one or more of the following: brightness / contrast adaptive adjustment, Retinex illumination equalization, adaptive Gamma correction, bilateral filtering denoising and sharpening enhancement; the corresponding parameters include at least Gamma value, filtering radius and weight coefficient, etc., and other parameters (such as Retinex scale / weight, sharpening intensity and threshold, etc.) can be extended according to the operator type.

[0083] To enhance adaptability and avoid the limitations of a single operator in complex degradation cases, this embodiment allows Policy-Net to output multi-operator cascade sequences, i.e., output length is... operator sequence And apply them sequentially to the input image to obtain .in, Set upper limit (For example, 3-5) to limit processing latency and suppress the risk of over-enhancement. To ensure controllability, continuous parameters... Before execution, the interval is pruned to fall within a preset valid range, for example, limiting Gamma to a certain value. Limit the radius of the bilateral filter to Limit the sharpening intensity to This avoids issues such as noticeable artifacts, broken lines due to over-sharpening, or blunted table lines due to excessive denoising. In a more robust implementation, the enhanced IQAN can be called again to obtain a new quality vector. If key quality indicators deteriorate significantly compared to before enhancement (e.g., reduced sharpness or increased noise exceeding the threshold), the last one or more operators are degraded or rolled back, thus forming a closed loop of "enhancement-evaluation-correction".

[0084] Preferably, step S4 performs table boundary detection on the enhanced image and filters target candidate boxes. In step S4, the enhanced image... Perform table boundary detection to obtain a set of one or more candidate boxes for table regions. And further determine the target candidate box through candidate box filtering. In this embodiment, table boundary detection can be implemented using a segmentation detection network based on FCN or a target detection network based on YOLO; the former outputs a probability map of the table region and obtains candidate boxes through connected component analysis, while the latter directly outputs the candidate boxes and their confidence scores. .

[0085] When multiple candidate boxes exist, this embodiment determines the candidate box selection based on candidate box filtering rules. The selection rules include at least one of the following: selecting based on the highest confidence level, selecting based on the largest candidate box area, or first performing non-maximum suppression (NMS) on the candidate boxes to remove highly overlapping duplicate boxes before selection; to improve the coverage of corner detection on edge lines, this embodiment can optionally select... Obtained by scaling That is, while keeping the center unchanged, expand outwards by a preset proportion (e.g., 2%–5%), and then from... Cropping to obtain the ROI image This is used by the corner network in step S5; if the table detection fails to output valid candidate boxes under extreme conditions, the entire image can be used as... Proceed to the next step, relying on the constraint rollback mechanism of step S5 to ensure that the process can continue to execute.

[0086] Preferably, in step S5, the four corner points are extracted within the target candidate bounding box region, and a backtracking is triggered when the constraints are not met. In step S5, the ROI image obtained in step S4 is processed. The four corner points of the table are extracted to obtain their coordinates. To ensure the stability and interpretability of corner point localization, this embodiment constructs a keypoint detection subnetwork to output a four-channel probability heatmap, corresponding to the top-left, top-right, bottom-right, and bottom-left corner point channels, respectively. During inference, non-maximum suppression (NMS) is performed on the heatmap of each channel within the target candidate box region, and the peak position of each channel is taken as the corresponding corner point coordinate. Simultaneously, the peak confidence is recorded to form a confidence set. To improve the stability of the perspective matrix solution, a weighted centroid approach can be used for sub-pixel thinning within the peak neighborhood, and the corner coordinates can be mapped back to the enhanced image from the ROI coordinate system. The full-map coordinate system.

[0087] To avoid perspective correction distortion caused by false corner detections, this embodiment sets three types of rigid engineering constraints and triggers a backoff strategy when the constraints are not met. Specifically, a corner confidence threshold is set. When the maximum confidence of any corner channel within the target candidate box is less than A rollback is triggered when the distance between any two adjacent corner points is less than a preset minimum distance threshold after corner point sorting. A rollback is triggered when the area of ​​the quadrilateral formed by the four corner points is less than a preset threshold proportion of the target candidate box area. A rollback is triggered when any of the above conditions are met. In this embodiment, the target candidate box is... The four vertices are directly used as the coordinates of the four corner points, thus ensuring that the subsequent homography matrix solution has a definite input and maintains the robustness of the process. The final output of the corner points is sorted in the order of "top left - top right - bottom right - bottom left" to unify the point correspondence in downstream perspective correction.

[0088] Preferably, S6 solves the homography matrix based on the four corner points and performs perspective correction, while simultaneously determining the output resolution according to rules.

[0089] In step S6, the coordinates of the four corner points are determined based on the set of coordinates obtained in step S5. Solve for the perspective transformation matrix and use this matrix to enhance the image. Perform perspective correction to obtain a geometrically reconstructed image of the page layout. In this embodiment, the perspective transformation matrix is ​​a homography matrix. It is determined by the correspondence between the four corner points and the four vertices of the target rectangle, and satisfies the following mapping relationship: ;

[0090] in To enhance the image Mid-pixel coordinates For geometric reconstruction of images Pixel coordinates. In engineering implementation, DLT or a mature vision library can be used to solve the problem. Normalization of the input points can be selected to improve numerical stability. When corner points come from non-backtracking detection and may be noisy, a solution method with RANSAC can be selected to reduce the influence of outliers.

[0091] Furthermore, to ensure that the output image meets the minimum detail resolution requirements for backend recognition without being excessively large and causing computational strain, this embodiment determines the output resolution of the target rectangle according to at least one of the following rules. One is the aspect ratio constraint rule: the aspect ratio of the target candidate box is used as the aspect ratio of the target rectangle, and within a preset maximum side length... The first constraint determines the width and height of the target rectangle, ensuring that the target rectangle maintains the same or approximately the same aspect ratio as the target candidate box. The second constraint is the line density / minimum cell pixel constraint rule: before or during geometric reconstruction, line detection is performed on the table's row and column lines. Based on the number of detected horizontal and vertical grid lines, the number of rows and columns is estimated, and the target rectangle resolution is set to satisfy the condition that "the average cell width after reconstruction is not less than a preset pixel threshold." The average cell height is not less than the preset pixel threshold. The minimum resolution; if not satisfied, then within a range not exceeding The output size is increased under the premise of perspective correction. Bilinear or bicubic interpolation is used for resampling during perspective correction, and invalid pixel areas generated by perspective resampling are identified in the output. This is used for subsequent defect mask generation.

[0092] Preferably, in step S7, a defect mask is generated and a completion network is activated based on a defect trigger threshold. In step S7, the geometrically reconstructed image is processed. Generate defect mask The defect mask is used to indicate edge clipping areas or locally missing areas, and to determine whether to activate the completion network accordingly. Specifically, the defect mask can be generated in at least one of the following ways: First, it can be generated based on invalid pixel areas of the perspective-corrected image, that is, holes / invalid areas generated by perspective resampling (by...). First, the table is marked as a missing area. Second, it is generated based on the relationship between the main body of the table and the image boundary. When the main body of the table intersects with the image boundary or the distance is less than the preset boundary distance threshold, the preset width strip area adjacent to the boundary is marked as a missing area. Third, morphological dilation or expansion is performed on the initial missing area to cover the broken table lines and table header edges, thereby avoiding the problem that the completion network only repairs holes and fails to extend the line structure.

[0093] Regarding the defect trigger determination, this embodiment sets a defect trigger threshold: calculating the ratio of the defect mask coverage area to the geometrically reconstructed image area. ,when Greater than the preset ratio threshold If the missing mask is connected to the outer boundary of the image, the completion network is activated; otherwise, the completion step is skipped and the geometrically reconstructed image is directly output. This mechanism is designed to avoid introducing unnecessary generative repairs in cases of minor edge interpolation gaps or when they do not affect recognition, thereby improving engineering reliability.

[0094] Preferably, step S8 uses a structure-aware completion network to repair edge defects and complete local structures, outputting a regularized image. In step S8, when step S7 determines that completion is needed, the structure-aware completion network performs edge defect repair and local structure completion on the defective area. The input to the completion network includes at least a geometrically reconstructed image. And further includes defect masks The output is the completed image. In this embodiment, the completion network adopts an encoder-decoder structure. By learning the geometric priors of the table's row and column lines and combining local texture and global layout information, it completes missing edge lines and / or header areas, ensuring that the completed row and column lines are continuous, the intersections are stable, and the cell topology is not unreasonably altered. To further enhance the "structure awareness" attribute, it is optional to introduce a network at the input end... The calculated table line structure guide map (e.g., the horizontal and vertical line probability map obtained using morphological line extraction or lightweight line detection networks) and compared with... and Channel splicing inputs are used to complete the network, thereby strengthening the network's constraints on the extension of the linear structure.

[0095] To reduce computational load and minimize unnecessary modifications to the complete region, this embodiment preferably performs clipping and inference only on the region containing the missing mask before re-attaching it to the entire image: that is, the missing mask is calculated first. The minimum bounding rectangle is used to expand outwards with a preset padding, and the resulting completed sub-image is input into the network for inference. The completed sub-image is then pasted back to its original position on the full image. Finally, light feathering or smooth blending can be used at the mask boundaries to avoid stitching artifacts. The output is then complete. In the engineering configuration table scenario, to avoid introducing incorrect field information from "generated content", this embodiment focuses on the structural continuity of table border lines, row and column line extensions, and table header outlines for completion. For text / numerical content areas, the original content is kept unredrawn, with only weak texture extensions or whitespace added to the missing areas, thereby reducing the risk of accidental modification while ensuring the layout is neat.

[0096] Example 2:

[0097] refer to Figure 1-7An adaptive image enhancement and layout geometry reconstruction system for engineering configuration tables is disclosed. This system performs quality assessment, adaptive enhancement, table boundary detection, corner point localization, perspective correction, edge defect repair, and local structure completion on photographed images of engineering configuration tables, thereby outputting a regularized image of the engineering configuration table for subsequent table structure recognition and field extraction. The system's input is a photographed image of the engineering configuration table. The output is a regularized project configuration table image. For ease of description, the enhanced image will be denoted as... The geometric reconstruction image of the page obtained by perspective correction is denoted as The completed image is denoted as The system includes at least the following modules: image acquisition and preprocessing module, image quality assessment module, enhancement strategy selection module, enhancement execution module, table boundary detection and candidate box filtering module, four-corner point extraction and backtracking module, perspective transformation and layout geometry reconstruction module, defect mask generation and trigger determination module, structure-aware completion module, and output and interface module. These modules can be implemented programmatically using the same processor, distributed by multiple processing units, or collaboratively by a general-purpose processor and a dedicated accelerator (such as a GPU / NPU).

[0098] In another embodiment, the system may also optionally include a training and co-optimization module, used to perform supervised learning training on the completion network and policy network, co-optimize with the back-end recognition model, or perform reinforcement learning optimization on the enhanced policy selection network during offline training or online iteration phases, so that the front-end output is more friendly to the back-end recognition.

[0099] In this embodiment, the image acquisition and preprocessing module is used to acquire photographic images of the engineering configuration table. The image is then subjected to basic preprocessing to generate standardized input for subsequent network module inference. This preprocessing includes at least one or more of the following: orientation correction, size normalization, luminance / grayscale channel extraction, and pixel value normalization. Specifically, when image metadata exists, rotation correction can be performed based on the EXIF ​​orientation; when reliable metadata is unavailable, a default orientation can be used for subsequent steps, with stability ensured by the fallback mechanism of subsequent modules. For size normalization, let the long side pixels of the input image be... ,when Greater than the preset input threshold When the image is scaled proportionally to limit computational load, when... If the image is too small, the input branch of the network can be scaled up proportionally to improve the resolution of the corner heatmap. Subsequently, the image can be converted to YCrCb / HSV and the luminance channel extracted as the preferred input for quality assessment and corner localization, while the RGB image is retained for subsequent enhancement operators to perform illumination equalization and contrast enhancement on the color image. After preprocessing, the preprocessed tensor is output. It can optionally output RGB preserved image. .

[0100] In this embodiment, the Image Quality Assessment Module (IQAN) is used to assess the quality of images taken from the input engineering configuration table and output a quality vector. The image quality assessment module is used to characterize the image quality status. It includes an Image Quality Assessment Network (IQAN), which can be a CNN-based or lightweight Transformer network, and preferably employs a multi-head output approach. Specifically, the IQAN backbone network extracts global features, then sets a regression head to output sharpness indicators, noise levels, illumination uniformity, and tilt angle estimates, and a binary classification head to output an indicator (or probability) of the presence of significant perspective distortion. Based on this, a quality vector is formed. It includes at least one or more of the following: sharpness, noise, illumination uniformity, tilt angle, and perspective distortion indicator. For ease of direct use by the subsequent policy network, each dimension of the quality vector can be calibrated and normalized to... Output the interval.

[0101] In this embodiment, the policy selection module (Policy-Net) is used to receive quality vectors, while the policy execution module is used to enhance the policy selection module. The enhancement execution module outputs the enhancement operation type and corresponding operation parameters; it performs adaptive enhancement on the captured image according to the enhancement operation type and operation parameters to obtain an enhanced image. The enhancement strategy selection module includes a Policy-Net network, whose output format is preferably "discrete encoding of operation type + continuous value of parameter". The enhancement operation type includes at least one or more of the following: adaptive brightness / contrast adjustment, Retinex illumination equalization, adaptive Gamma correction, bilateral filtering denoising, and sharpening enhancement. The operation parameters include at least one or more of the following: Gamma value, filter radius, and weight coefficient, and other parameters can be extended according to the operator type. To adapt to complex degradation scenarios, the Policy-Net preferably also supports multi-operator cascading, that is, outputting multiple enhancement operators and their parameters arranged in sequence, which are then applied sequentially to the input image by the enhancement execution module to obtain the enhanced image. In engineering implementation, the system can perform interval pruning on continuous parameters and set an upper limit on the cascade length. This is to avoid over-enhancement and excessive latency; and optionally, IQAN can be called again after enhancement for quality review, and the last operator can be degraded or rolled back if the quality deteriorates, in order to improve the controllability and stability of the enhancement.

[0102] In this embodiment, the table boundary detection and candidate box filtering module is used to process the enhanced image. Perform table region detection, outputting at least one table region candidate box, and further filter to determine the target candidate box $B^\*$. The table boundary detection can be implemented using a table region detection network based on FCN and / or YOLO. If multiple candidate boxes are output, it is preferable to perform candidate box filtering on these multiple candidate boxes to determine the target candidate box. The candidate box filtering includes at least one of the following: selection based on the maximum confidence principle, selection based on the maximum candidate box area principle, or selection after performing non-maximum suppression (NMS) on the candidate boxes. To improve the coverage of edge lines by corner point extraction, the system can optionally perform a preset ratio expansion on the target candidate boxes and then crop them to obtain the ROI image. This is used by the corner extraction module; if the detection fails, the entire image can be used as a target candidate box for subsequent steps, and the process can be kept executable through the corner backoff mechanism.

[0103] In this embodiment, the four-corner point extraction and backtracking module is used to extract the coordinates of the four corner points of the table within the target candidate box area and trigger a backtracking strategy when a corner point is deemed unreliable. This module includes a keypoint detection sub-network, which outputs probability heatmaps for each of the four corner points of the table. These probability heatmaps are four-channel heatmaps, corresponding to the top-left, top-right, bottom-right, and bottom-left corner points, respectively. During inference, non-maximum suppression is performed on each channel heatmap within the target candidate box area to obtain the coordinates of the four corner points, and these coordinates are output in the order of "top-left—top-right—bottom-right—bottom-left".

[0104] To improve the robustness of geometric reconstruction, this embodiment sets a corner confidence threshold. and minimum distance threshold Area ratio threshold When the maximum confidence score of any corner channel within the target candidate box is less than... Or, after sorting, the distance between any two adjacent corner points is less than Or, the area of ​​the quadrilateral formed by the four corner points is less than a preset ratio of the area of ​​the target candidate box. When this happens, a backtracking strategy is triggered: the four vertices of the target candidate box are used as the coordinates of the four corner points for output. Wherein, , , These are configurable engineering parameters. Through the above constraints and rollbacks, the system can still output a definite corner input even in the case of false corner detection or missing edges, ensuring that the downstream homography solution can be executed.

[0105] In this embodiment, the perspective transformation and layout geometry reconstruction module is used to solve for the perspective transformation matrix based on the coordinates of the four corner points, and then use this matrix to enhance the image. Perform perspective correction to obtain a geometrically reconstructed image of the page layout. The perspective transformation matrix is ​​a homography matrix. It is determined by the correspondence between the four corner points and the four vertices of the target rectangle, and satisfies the following mapping relationship: ;

[0106] in To enhance pixel coordinates in an image, These are the pixel coordinates in the geometrically reconstructed image. The system can use DLT to solve the problem or call a mature vision library, and can optionally use a solution method with Random Consistent Sampling (RANSAC) to suppress the influence of corner noise.

[0107] Furthermore, to ensure that the output meets the minimum detail resolution requirements for subsequent recognition while controlling the computational load, the target rectangle output resolution in this embodiment is determined according to at least one of the following rules: one is the aspect ratio constraint rule, that is, the aspect ratio of the target candidate box is used as the aspect ratio of the target rectangle, and within a preset maximum side length The first constraint determines the width and height of the target rectangle, ensuring that the target rectangle maintains the same or approximately the same aspect ratio as the target candidate box. The second constraint is the line density / minimum cell pixel constraint rule, which involves detecting the row and column lines of the table before or during geometric reconstruction, estimating the number of rows and columns based on the number of detected horizontal and vertical grid lines, and setting the resolution of the target rectangle to satisfy the condition that "the average cell width after reconstruction is not less than a preset pixel threshold". The average cell height is not less than the preset pixel threshold. The minimum resolution of ""; where , , These are configurable engineering parameters. During perspective resampling, the system can also output invalid pixel area identifiers to indicate holes / invalid areas caused by perspective correction.

[0108] In this embodiment, the defect mask generation and trigger determination module is used to generate defect masks. And determine whether to activate the completion network. The defect mask is used to indicate the edge clipping area or local missing area. The defect mask can be generated in at least one of the following ways: First, it is generated based on the invalid pixel area of ​​the perspective-corrected image, that is, the hole / invalid area generated by perspective resampling is marked as the defect area; Second, it is generated based on the relationship between the table body area and the image boundary. When the table body area intersects with the image boundary or the distance is less than a preset boundary distance threshold, the preset width strip area adjacent to the boundary is marked as the defect area; Third, morphological dilation or expansion is performed on the initial defect area to cover the broken table lines and table header edges.

[0109] Regarding trigger determination, this embodiment sets a defect trigger threshold: when the proportion of the defect mask coverage area to the geometrically reconstructed image area is greater than a preset proportion threshold. If the missing mask is connected to the outer boundary of the image, the completion network is activated; otherwise, the completion step is skipped and the geometrically reconstructed image is directly output. Through the aforementioned judgment mechanism, the system avoids introducing unnecessary generative repairs when the damage is minor and does not affect subsequent identification, thereby improving engineering stability.

[0110] In this embodiment, the structure-aware completion module is used to reconstruct the geometric image of the layout when completion is triggered. The missing areas are repaired for edge defects and local structures are completed to output a completed engineering configuration table image. The input of the structure-aware completion network includes at least a geometrically reconstructed image and further includes a defect mask. The completion network learns geometric priors about the table's row and column lines, combining local texture and global layout information to complete missing edge lines and / or header regions. The completion network employs an encoder-decoder structure, and preferably only performs cropping and inference on regions containing missing masks before pasting them back onto the entire image, reducing computational load and minimizing unnecessary modifications to complete regions. After completion, the output and interface module merges the completion result with the non-missing regions to form the final output image. It can optionally output intermediate products such as quality vectors, candidate boxes, corner points, homography matrices, and defect masks for debugging, auditing, and tracing.

[0111] The system executes in the following order: Image acquisition and preprocessing module output. → Image quality assessment module outputs quality vector → Enhancement strategy selection module outputs operator type and parameters (or operator sequence) → Enhancement execution module outputs enhanced image → The table boundary detection and candidate box filtering module outputs the target candidate boxes. With ROI image → The four-corner point extraction and rollback module outputs the coordinates of the four corner points. (Includes optional rollback flag) → Solve the homography matrix using the perspective transformation and layout geometry reconstruction module. And output the geometrically reconstructed image. Invalid pixel region identifier → Defect mask generation and trigger determination module outputs defect mask And decide whether to complete → The structure-aware completion module outputs the completion result and forms the final output when triggered. Data between modules can be transferred in the form of image matrices, mask matrices, coordinate arrays, parameter vectors, etc. The system can achieve inter-module connections through local function calls, SDK interfaces, or network service interfaces.

[0112] The system further includes a training and collaborative optimization module for training or iteratively optimizing the structure-aware completion network and the enhancement strategy selection network. Specifically, the training of the structure-aware completion network can employ a loss function constrained by pixel reconstruction, structure awareness, and perceptual consistency. The pixel reconstruction loss constrains pixel consistency within the completion region, the structure awareness loss is constructed based on table line detection results to constrain row and column line continuity, intersection stability, and cell topology consistency, and the perceptual loss constrains high-level semantic and texture consistency within the completion region. The total loss is a weighted sum of the above losses, and each weight is a configurable engineering parameter.

[0113] The system and the backend multimodal table recognition large model are collaboratively optimized: the normalized engineering configuration table image output from the front end is used as the backend input, and the combination of structure recognition accuracy and field recognition F1 score is used as the optimization objective. The parameters of at least one module in the front end are updated through joint training or two-stage training, making the enhancement strategy and geometric reconstruction results better for backend recognition. Furthermore, collaborative optimization can also include reinforcement learning optimization of Policy-Net: the enhancement operator types and operation parameters output by Policy-Net are treated as actions, and the recognition accuracy corresponding to the backend recognition results is used as the reward signal to update the Policy-Net policy; and reinforcement learning optimization and supervised learning optimization can be combined by first supervising pre-training and then fine-tuning through reinforcement learning, or by alternating updates during joint training.

[0114] The system can be deployed on mobile terminals or industrial handheld devices, utilizing local processors and NPU / GPUs for inference; it can also be deployed on edge servers or cloud servers, with the terminal uploading captured images and receiving the normalized results; or it can adopt an edge-cloud collaborative approach, performing preprocessing and lightweight enhancements on the edge, and corner localization, geometric reconstruction, and completion inference on the cloud, to balance latency and computing power. The system can also employ pipelined parallel processing to improve throughput, for example, performing quality assessment and policy decisions for the next image in parallel while processing corner localization and geometric reconstruction of the current image.

[0115] Example 3:

[0116] In an embodiment of an electronic device, the electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor; when the processor executes the program, it implements the functions of the aforementioned modules and outputs a standardized engineering configuration table image. In an embodiment of a computer-readable storage medium, the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the aforementioned system functions.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables, characterized in that, Includes the following steps: S01 acquires a photograph of the project configuration table; S02 inputs the captured image into the Image Quality Assessment Network (IQAN) and outputs a quality vector representing the quality state of the captured image. S03 inputs the quality vector into the adaptive enhancement strategy selection network Policy-Net, outputs at least one enhancement operation type and its corresponding operation parameters, and performs adaptive enhancement on the captured image according to the enhancement operation type and the operation parameters to obtain an enhanced image; S04 performs table boundary detection on the enhanced image and extracts the four corner points of the table to obtain the coordinates of the four corner points of the table. S05 calculates the perspective transformation matrix based on the coordinates of the four corner points of the table, and uses the perspective transformation matrix to perform perspective correction on the enhanced image to obtain a geometrically reconstructed image of the page layout; S06 performs edge defect repair and local structure completion on the geometric reconstruction image of the layout, and outputs a normalized engineering configuration table image.

2. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 1, characterized in that, The image quality assessment network IQAN assesses the quality of the images taken from the input engineering configuration table and outputs a quality vector. The quality vector includes at least one or more of the following dimensions: sharpness index, noise level, illumination uniformity, tilt angle estimate, and an indication of whether there is significant perspective distortion. The IQAN is a CNN or lightweight Transformer-based network that uses a multi-head output method to output sharpness, noise, illumination uniformity, and tilt angle using regression heads, and to output perspective distortion indicators using a binary classification head. Before inputting into IQAN, at least one preprocessing step is performed on the captured image: size normalization, luminance / grayscale channel extraction, and pixel value normalization.

3. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 1 or 2, characterized in that, The Policy-Net network takes the quality vector as input and outputs the enhancement operation type and corresponding operation parameters; wherein: The enhancement operation types include at least one or more of the following: adaptive brightness / contrast adjustment, Retinex illumination equalization, adaptive Gamma correction, bilateral filtering noise reduction, and sharpening enhancement; The operating parameters include at least one or more of the following: Gamma value, filter radius, and weighting coefficient. The preferred output format of Policy-Net is "discrete code of operation type + continuous value of parameter". The corresponding enhancement operator is selected according to the discrete code, and the enhancement is performed after configuring the parameters of the enhancement operator using the continuous value. The enhancement process preferably supports multi-operator cascading: multiple enhancement operators and their parameters are output by Policy-Net in sequence and applied to the input image in turn to obtain the enhanced image; Policy-Net is jointly trained with the large backend table recognition model or trained in two stages, using "recognition accuracy" as a reinforcement learning reward signal to make the output enhancement strategy more friendly to the backend recognition.

4. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 1, characterized in that, The table boundary detection includes: using a table region detection network based on FCN and / or YOLO to detect the enhanced image and output at least one table region candidate box; and preferably performing candidate box filtering on multiple candidate boxes to determine the target candidate box, wherein the candidate box filtering includes at least one of the following: selecting according to the principle of maximum confidence, selecting according to the principle of maximum candidate box area, or performing non-maximum suppression on the candidate boxes before selection.

5. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 4, characterized in that, The extraction of the four corner points of the table includes: constructing a key point detection sub-network to output probability heatmaps for the four corner points of the table, wherein the probability heatmaps are four-channel heatmaps, corresponding to the top left, top right, bottom right, and bottom left corner points respectively; performing non-maximum suppression on each channel heatmap within the defined area of ​​the target candidate box to obtain the coordinates of the four corner points, and outputting them in the order of "top left - top right - bottom right - bottom left"; Set the corner confidence threshold When the maximum confidence of any corner channel within the target candidate box is less than Or, after sorting, the distance between any two adjacent corner points is less than a preset minimum distance threshold. Or, the area of ​​the quadrilateral formed by the four corner points is less than a preset ratio threshold of the area of ​​the target candidate box. When this happens, a backoff strategy is triggered: the four vertices of the target candidate box are used as the coordinates of the four corner points for output; where, , , These are configurable engineering parameters.

6. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 1 or 5, characterized in that, The perspective transformation matrix is ​​a homography matrix. It is determined by the correspondence between the four corner points and the four vertices of the target rectangle, and satisfies the following mapping relationship: ; in To enhance pixel coordinates in an image, These are the pixel coordinates in the geometrically reconstructed image; The output resolution of the target rectangle is determined according to at least one of the following rules: Aspect Ratio Constraint Rule: The aspect ratio of the target candidate box is used as the aspect ratio of the target rectangle, and the maximum side length is within a preset range. The width and height of the target rectangle are determined under constraints such that the target rectangle maintains the same or approximately the same aspect ratio as the target candidate box; Line density / minimum cell pixel constraint rule: Before or during geometric reconstruction, perform line detection on the table's row and column lines. Estimate the number of rows and columns based on the number of detected horizontal and vertical grid lines, and set the target rectangle resolution to satisfy the condition that "the average cell width after reconstruction is not less than a preset pixel threshold". The average cell height is not less than the preset pixel threshold. The minimum resolution; in, , , These are configurable engineering parameters.

7. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 6, characterized in that, The edge defect repair and local structure completion are achieved through a structure-aware completion network. The input of the structure-aware completion network includes at least a geometrically reconstructed image and further includes a defect mask; wherein: The defect mask is used to indicate edge-trimmed areas or locally missing areas, and the defect mask is generated by at least one of the following methods: Invalid pixel region generation based on perspective-corrected image: mark the holes / invalid regions generated by perspective resampling as defective regions; Based on the relationship between the main body of the table and the image boundary: when the main body of the table intersects with the image boundary or the distance is less than the preset boundary distance threshold, the preset width strip area adjacent to the boundary is marked as the defect area; Perform morphological dilation or expansion on the initial defect area to cover the broken table lines and table header edges; Set a defect trigger threshold. When the proportion of the defect mask coverage area to the geometric reconstruction image area is greater than a preset proportion threshold, or when the defect mask is connected to the outer boundary of the image, start the completion network; otherwise, skip the completion step and directly output the geometric reconstruction image. The completion network learns the geometric priors of the table rows and columns, and combines local texture and global layout information to complete the missing edge lines and / or table header areas, outputting a completed project configuration table image. The completion network adopts an encoder-decoder structure, and preferably only performs cropping and inference on the areas containing the missing mask before pasting it back to the whole image, so as to reduce the amount of computation and reduce unnecessary modifications to the complete area.

8. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to claim 7, characterized in that, The structure-aware completion network is trained using a loss function constrained by pixel reconstruction, structure awareness, and perceptual consistency, and: The pixel reconstruction loss is used to constrain the pixel consistency within the completed area, and is preferably L1 or L2 reconstruction loss; The structure-aware loss is constructed based on the table line detection results. Specifically, the table row and column line detection is performed on the completed output image to obtain a line structure map, and consistency constraints are applied between the labeled line structure map or the reference line structure map generated from the complete table to ensure that the completed row and column lines are continuous, the intersection points are stable, and the cell topology is consistent. The perceptual loss is used to constrain the high-level semantic and texture consistency of the completed region, and is preferably obtained by calculating the difference in a multi-layer feature space by a pre-trained feature extraction network. The total loss is the weighted sum of the losses mentioned above, and the weights of each item are configurable engineering parameters.

9. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to any one of claims 1-8, characterized in that, The adaptive enhancement and layout geometry reconstruction module is collaboratively optimized with the backend multimodal table recognition model: the normalized engineering configuration table image output from the front end is used as the input of the multimodal table recognition model, and the combination of structure recognition accuracy and field recognition F1 value is used as the optimization target. The parameters of at least one module of the front end are updated through joint training or two-stage training so that the enhancement strategy and geometry reconstruction results are optimal for backend recognition.

10. The adaptive image enhancement and layout geometry reconstruction method for engineering configuration tables according to any one of claims 1-8, characterized in that, The collaborative optimization includes reinforcement learning optimization of the policy selection network Policy-Net: the type of enhancement operator and operation parameters output by Policy-Net are regarded as actions, and the recognition accuracy corresponding to the back-end recognition result is used as the reward signal to update the policy of Policy-Net to achieve end-to-end optimization. The reinforcement learning optimization and supervised learning optimization can be combined in at least one of the following ways: supervised pre-training followed by reinforcement learning fine-tuning, or alternating updates during joint training.

Images