Deep learning combined with LLM engineering material intelligent auditing method and system

Abstract

The application discloses a deep learning and LLM combined engineering material intelligent auditing method and system, and relates to the technical field of engineering construction material digitization. The method comprises the following steps: signature region identification and extraction are performed on an engineering material document image to be audited to obtain a signature region image to be audited. A signature feature encoder is used to process the signature region image to be audited to obtain a signature similarity coefficient and a visual evaluation confidence. The adaptive analysis depth parameters of a large language model are configured based on the visual evaluation confidence, and signature semantic verification is performed in combination with context information to obtain a signature semantic consistency coefficient. The signature similarity coefficient and the signature semantic consistency coefficient are fused to obtain a signature confidence by setting adaptive visual weights and adaptive semantic weights, and auditing and ruling are completed. The application realizes adaptive deep cooperation of visual evidence and semantic logic, and effectively solves the problems of accuracy, robustness and explainability of engineering material signature automatic auditing under low samples and complex backgrounds.

Description

Technical Field

[0001] This invention relates to the field of digital engineering construction data technology, specifically to an intelligent review method and system for engineering data that combines deep learning and LLM. Background Technology

[0002] In the engineering and construction field, construction logs, acceptance reports, and change orders are crucial documents for project compliance and quality traceability. These documents are typically circulated as scanned copies or electronic files, often containing handwritten signatures, seals, and complex forms. Currently, the review process relies heavily on manual labor, requiring verification of signature authenticity, signature placement, date logic, and page consistency. With the increasing scale of projects and the accelerated digitization of documents, the inefficiency, high cost, inconsistent standards, and susceptibility to errors due to fatigue associated with traditional manual review are becoming increasingly prominent, posing a major bottleneck to management effectiveness.

[0003] To address the shortcomings of manual review, existing technologies attempt to introduce automated methods. One type of method is primarily based on deep learning visual models, such as using object detection to locate the signature region and then performing handwriting comparison through metric learning. This method is effective in processing clear and standardized signatures, but its performance depends on a large number of high-quality training samples. In engineering practice, where handwritten signature samples are scarce, image quality varies, and background interference is complex, its recognition reliability and the accuracy of confidence assessment decrease significantly. Another type of method attempts to leverage the powerful semantic understanding and reasoning capabilities of large language models to directly analyze document content to determine the compliance of the signature. However, large language models have inherent limitations when processing unstructured image information, especially visual elements such as handwriting, and are prone to misreading or illusions. Furthermore, their decision-making process lacks quantifiable and auditable visual evidence, making it difficult to meet the stringent requirements of accuracy and traceability in engineering reviews. Summary of the Invention

[0004] This invention addresses the technical problem of existing technologies being unable to reliably and interpretably automate the review of handwritten signatures in engineering documents under low-sample conditions, and provides an intelligent review method and system for engineering documents that combines deep learning and LLM.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, this invention provides an intelligent review method for engineering documents that combines deep learning and LLM, including: The signature area of ​​the engineering documents to be reviewed is identified and extracted to obtain the signature area image to be reviewed. The signature region image to be reviewed is processed using a signature feature encoder to calculate the signature similarity coefficient and visual evaluation confidence. Based on the visual evaluation confidence level, the adaptation analysis depth parameters of the large language model are configured, and the signature semantic consistency coefficient is obtained by performing signature semantic verification in combination with context information. Based on the visual assessment confidence level, adaptive visual weights and adaptive semantic weights are set, and the signature similarity coefficient and signature semantic consistency coefficient are weighted and calculated to obtain the signature confidence level. The review decision is then completed based on the signature confidence level.

[0006] Secondly, this invention provides an intelligent engineering data review system combining deep learning and LLM, comprising: The signature area recognition and extraction module is used to recognize and extract the signature area of ​​the engineering documents to be reviewed, and obtain the signature area image to be reviewed. The signature feature encoding and calculation module is used to process the image of the signature region to be audited using the signature feature encoder, and calculate the signature similarity coefficient and visual evaluation confidence. The semantic verification module is used to configure the adaptation analysis depth parameters of the large language model based on the visual evaluation confidence level, and to perform signature semantic verification in combination with context information to obtain the signature semantic consistency coefficient. The review and adjudication module is used to set adaptive visual weights and adaptive semantic weights based on the visual assessment confidence level, calculate the signature confidence level by weighting the signature similarity coefficient and the signature semantic consistency coefficient, and complete the review and adjudication based on the signature confidence level.

[0007] The beneficial effects of this invention are: Compared to existing technologies, this invention first generates signature similarity coefficients and visual assessment confidence scores by extracting visual features, quantifying the reliability of signature matching at the image level. Secondly, it innovatively uses the visual assessment confidence score as a dynamically adjustable parameter, adaptively configuring the analysis depth and scope of the large language model to achieve deep synergy between visual cues and semantic analysis. Thirdly, it performs semantic verification by combining the large language model with document context, generating a signature semantic consistency coefficient to verify the rationality of the signature. Finally, it dynamically allocates visual and semantic weights based on the same visual assessment confidence score, integrating multi-dimensional evidence to form a comprehensive signature confidence score. This ensures that the review decision possesses both objective quantitative evidence and semantic interpretability, improving the accuracy, robustness, and interpretability of engineering document signature review in low-sample, complex contexts. Attached Figure Description

[0008] Figure 1 A flowchart illustrating the intelligent review method for engineering data combining deep learning and LLM provided by this invention; Figure 2 A schematic diagram of the structure of the intelligent engineering data review system combining deep learning and LLM provided by the present invention.

[0009] In the attached diagram, the components represented by each number are as follows: The module includes: a signature area identification and extraction module 11, a signature feature encoding and calculation module 12, a semantic verification module 13, and an audit and adjudication module 14. Detailed Implementation

[0010] Example 1, as Figure 1 As shown, this embodiment of the invention provides an intelligent review method for engineering data combining deep learning and LLM, including: S10: Perform signature area recognition and extraction on the engineering data document image to be reviewed, and obtain the signature area image to be reviewed; First, the signature area is identified and extracted from the input images of the engineering documents to be reviewed. The images of the engineering documents to be reviewed are digital forms of forms, reports, records and other documents generated during various engineering construction processes. They are complex layout images containing various visual elements such as printed text, tables, handwritten signatures, and seals, such as construction acceptance forms, design change orders, and material arrival approval forms.

[0011] The signature region recognition process is essentially a preprocessing step that separates the signature as a key audit element from the complex document background. By using deep learning-based object detection or semantic segmentation technology, it accurately identifies all possible locations of handwritten signatures in the document image and extracts the corresponding image blocks from the original image. This provides structured input for subsequent specialized signature handwriting comparison and semantic verification, ensuring that the audit process can focus on signed information with clear validity and responsibility.

[0012] In the review of engineering documents, signatures are the core visual evidence for confirming responsibility, authority and process compliance. Their location, handwriting and contextual logic must be strictly verified. Therefore, accurate signature area recognition is the primary technical foundation for the automated review process to be launched and for ensuring the accuracy of subsequent analysis.

[0013] Optionally, the signature region recognition process can be implemented using a target detection model based on a deep convolutional neural network. For example, using a model that has been trained on a large number of engineering document images and signature box annotation data, forward inference is performed on the input document image. The model will output several predicted bounding boxes and their corresponding class confidence scores, where the bounding box with the class of signature region is the recognition result.

[0014] Next, the identified candidate signature regions are extracted. Based on the bounding box coordinates output from the recognition step, for each candidate box identified as a signature region, a cropping operation is performed at the corresponding pixel position in the original image to obtain an independent image of the signature region to be reviewed. This image of the signature region to be reviewed is a local image containing the handwritten signature and possibly some adjacent background, representing a specific review target entity separated from the original complex document. It is used for subsequent specialized signature feature encoding and visual analysis to ensure that the review process can accurately focus on core evidence.

[0015] S20: The image of the signature region to be audited is processed using a signature feature encoder to calculate the signature similarity coefficient and visual evaluation confidence. Secondly, the obtained signature region image to be reviewed is processed using a signature feature encoder. This signature feature encoder is a neural network model trained based on deep metric learning, used to extract high-dimensional feature vectors with strong discriminativeness and robustness from the input handwritten signature image, so as to map the signature image to a feature space in which different signature samples of the same signer are close to each other while samples of different signers are far apart. Finally, the signature similarity coefficient and visual evaluation confidence are calculated.

[0016] The signature similarity coefficient is a quantitative value that represents the degree of matching between the signature to be reviewed and the corresponding signer's signature in the reference signature database in terms of handwriting features. It is used to objectively reflect the similarity between the two in terms of visual form. The visual evaluation confidence is an evaluation value that comprehensively reflects the impact of factors such as the quality of the current signature image, the degree of background interference, and the sufficiency of the reference samples on the credibility of the visual feature matching results. It is used to measure the reliability and uncertainty of identity determination based solely on visual information.

[0017] Specifically, the image of the signature region to be audited is processed using a signature feature encoder to calculate the signature similarity coefficient and visual evaluation confidence level, including: A signature feature encoder is generated by jointly training a deep convolutional neural network based on the ArcFace loss function and the triplet loss function. Using the signature feature encoder, the signature feature vector to be audited in the signature region image to be audited is extracted, and the reference signature feature vector set of the valid reference signature of the corresponding signer is extracted from the low sample reference signature library. Calculate the cosine similarity between the signature feature vector to be audited and each reference signature feature vector in the reference signature feature vector set, and use the maximum cosine similarity value as the signature similarity coefficient; The first confidence factor is obtained by evaluating the actual number of valid reference signature samples of the corresponding signer in the low-sample reference signature library, wherein the ratio of the actual number of valid reference signature samples to the number of preset benchmark samples is used as the first confidence factor. The second confidence factor is obtained based on the image quality and background complexity of the image of the signature area to be reviewed; The first confidence factor and the second confidence factor are weighted and calculated to obtain the visual assessment confidence.

[0018] First, the signature feature encoder is a specialized model generated through supervised training using a joint loss function. This step involves jointly training a deep convolutional neural network based on the ArcFace loss function and the triplet loss function. Specifically, the ArcFace loss function increases the angular spacing between samples of different classes in the feature space, thereby enhancing the model's ability to distinguish the signer's identity. The triplet loss function further optimizes the relative distribution relationships in the feature space by bringing positive sample pairs closer together and distancing negative sample pairs further apart.

[0019] By combining the ArcFace loss function and the triplet loss function, the trained deep convolutional neural network can map the signature image to a feature space with good metric properties. The signature feature vectors of the same signer are highly clustered, while the feature vectors of different signers are significantly separated. The model generated in this way is the signature feature encoder.

[0020] Specifically, a deep convolutional neural network is jointly trained based on the ArcFace loss function and the triplet loss function to generate a signature feature encoder, including: The basic network structure for constructing a signature feature encoder is based on a deep convolutional neural network; Prepare a sample training dataset containing signature image samples from multiple different signers, where each signer corresponds to at least one signature image sample; Construct a joint loss function, wherein the joint loss function is a weighted combination of the ArcFace loss function and the triplet loss function, used to simultaneously constrain the inter-class separability of feature vectors in the angle space and the relative distance relationship between sample pairs during training; Using the sample training dataset and the joint loss function, end-to-end supervised training is performed on the deep convolutional neural network until the model converges, and the trained deep convolutional neural network is used as the signature feature encoder.

[0021] First, the basic network architecture of the signature feature encoder is constructed. This basic network architecture uses a deep convolutional neural network as its core, which acts as the backbone of feature extraction, gradually extracting abstract visual features from the input signature image from low to high levels. Specifically, the design of the deep convolutional neural network needs to ensure that it has sufficient expressive power to characterize the complex and varied handwriting styles, pen pressure, and structural layout details in handwritten signatures.

[0022] Secondly, prepare a sample training dataset for model training. This dataset should contain signature image samples from multiple different signers, with each signer corresponding to at least one signature image sample. Specifically, the sample training dataset should be constructed to meet the low-sample requirement, allowing each signer to provide only a small number of signature samples, such as one to five samples, to simulate the real-world scenario where reference signatures are difficult to obtain in large quantities in engineering practice. Each signature image sample should be labeled with its corresponding signer identity tag.

[0023] Furthermore, a joint loss function is constructed to guide model training. This joint loss function is a composite supervision signal formed by combining the ArcFace loss function and the triplet loss function with preset weight coefficients. Specifically, the core function of the ArcFace loss function is to impose constraints on the angular dimension of the feature space. Its optimization objective is to increase the angular interval between the signature feature vectors of different signers, thereby enabling the model to learn highly discriminative features, making the feature distributions of different categories far apart in the angular space. The core function of the triplet loss function is to impose constraints on the Euclidean distance dimension of the feature space. Its optimization objective is to minimize the distance between the feature vectors of different signature samples of the same signer, while maximizing the distance between the feature vectors of signature samples of different signers, thereby finely adjusting the relative proximity between feature vectors.

[0024] By combining the two weighted loss functions, the training process is guided by two strong constraints: angular separability and relative distance relationships. This results in learning a feature embedding space with superior metric properties. The weights of the ArcFace loss function and the triplet loss function are dynamically set based on specific task requirements and model convergence performance. For example, in the early stages of training, the triplet loss function is given a higher weight to quickly build the basic structure of the feature space. In the later stages of training, the weight of the ArcFace loss function is gradually increased to refine the inter-class boundaries. Finally, a fixed weighting ratio is determined through cross-validation. For instance, setting the weight coefficient λ to 0.7 results in the ArcFace loss accounting for 70% and the triplet loss accounting for 30% in the joint loss function.

[0025] Finally, using the prepared training dataset and the aforementioned joint loss function as the optimization objective, the deep convolutional neural network is trained end-to-end. During training, the network parameters are iteratively updated using the backpropagation algorithm to minimize the value of the joint loss function until the model converges. Specifically, the convergence condition is set according to a preset training stopping criterion, such as the model's signature recognition accuracy improvement on the independent validation set being less than 0.5% for ten consecutive training epochs, or the joint loss function value decreasing by less than 0.01% for twenty consecutive training epochs. At this point, the trained deep convolutional neural network possesses the ability to map signature images to high-quality feature vectors. This model is formally defined as a signature feature encoder, used for subsequent inference and feature extraction tasks.

[0026] For example, the signature feature encoder can consist of a feature extraction backbone network and a global feature embedding layer. The feature extraction backbone network employs an improved residual network structure, and its input layer receives a signature region image that has undergone size normalization and pixel normalization. The backbone network contains multiple cascaded convolutional modules, each consisting of a convolutional layer, a batch normalization layer, and a ReLU activation function, used to progressively extract multi-scale signature visual features from edges, textures, to the overall structure. The global feature embedding layer is connected after the backbone network and typically consists of a global average pooling layer and a fully connected layer. Its function is to aggregate and map the two-dimensional feature maps extracted by the backbone network into a fixed-length, high-dimensional signature feature vector. During the training phase, to suppress model overfitting, a Dropout layer is introduced before the fully connected layer, with a dropout rate set to 0.3.

[0027] During training, key hyperparameters included an initial learning rate of 0.01, a total of 100 training epochs, and a batch size of 32. The learning rate was set using a warm-up and cosine annealing strategy to balance stability in the early stages of training with convergence accuracy in later stages. The number of training epochs ensured the model had sufficient iterations to fully learn the metric space distribution of the signature samples. The batch size was chosen to balance gradient update stability with hardware memory limitations. Specifically, metric learning was used for training, with the aforementioned training dataset containing multiple signers and their corresponding signature image samples as input. Several signature samples from each signer in the training dataset constituted a category.

[0028] Furthermore, the signature image samples from the training dataset are input into a deep convolutional neural network, with the joint loss function, weighted by the ArcFace loss and triplet loss, used as the optimization objective. The training process employs stochastic gradient descent as the optimizer, supplemented by a momentum term to accelerate convergence. In each iteration, the feature vectors output by the network and the joint loss value are calculated via forward propagation, and then the gradient is calculated and all network weight parameters are updated via backpropagation. Through optimization of the joint loss function, the model simultaneously learns to enhance the inter-class angular separability and intra-class clustering of the feature vectors.

[0029] Specifically, the entire training process is monitored by a validation set, and convergence is determined according to preset criteria. For example, if the mean accuracy used to evaluate signature verification performance on the validation set improves by less than 0.5% over fifteen consecutive training epochs, or the joint loss function value decreases by less than 0.1% over ten consecutive training epochs, the model is considered converged, and training is terminated. The resulting converged deep convolutional neural network model serves as the signature feature encoder, stably mapping the input handwritten signature image to a high-dimensional feature space with good metric properties for subsequent similarity calculation and confidence assessment. Furthermore, the generated signature feature encoder is used for feature extraction. This signature feature encoder processes the image of the signature region to be audited and the image from the low-sample reference signature library respectively. For the signature region image to be audited, the signature feature encoder outputs a high-dimensional signature feature vector to be audited. Simultaneously, all valid reference signature samples corresponding to the signer claimed by the current signature to be audited are retrieved from the low-sample reference signature library, and the feature vector of each reference signature sample is extracted one by one using the same encoder, forming a reference signature feature vector set. The reference signature feature vector set contains at least one reference signature feature vector.

[0030] Specifically, the signature feature vector to be verified is a numerical representation that has been abstracted and compressed through a deep network. It represents the measurement information extracted from the current signature image to be verified, containing core visual attributes such as handwriting style, structural layout, and penmanship habits. The reference signature feature vector set is a collection of several feature vectors. Each reference signature feature vector in the set represents similar measurement information extracted from confirmed valid historical signature samples belonging to the same signer. Together, they constitute an authoritative and certified handwriting feature reference benchmark for comparison in the feature space.

[0031] Secondly, the cosine similarity between the feature vector of the signature to be reviewed and each reference signature feature vector in the reference signature feature vector set is calculated. Cosine similarity is a similarity measurement method based on a vector space model, which measures the degree of proximity of two feature vectors in a direction. Its value range is usually between -1 and +1, with higher values ​​indicating more consistent directions and more similar features. The maximum value is selected from all calculated cosine similarity values ​​and formally determined as the signature similarity coefficient. This signature similarity coefficient directly quantifies the degree of matching between the signature to be reviewed and the most similar sample in the reference signature database in terms of handwriting visual features.

[0032] Next, the calculation process for visual assessment confidence is initiated. This process requires a comprehensive evaluation of multiple factors affecting the reliability of visual recognition. Specifically, the calculation process first generates a first confidence factor from the dimension of sample sufficiency. Due to the low-sample nature of engineering practice, the number of valid signature samples for a specific signer in the low-sample reference signature library is often limited. The first confidence factor is calculated by dividing the actual number of valid reference signature samples for that signer by a preset baseline sample size. The baseline sample size represents the ideal sample size expected for reliable visual comparison and is set based on domain expert experience and historical data statistical analysis; for example, the baseline sample size is set to 5. The closer the actual number of valid reference signature samples is to or exceeds the preset baseline sample size, the higher the first confidence factor, indicating that the reference basis for comparison is more sufficient.

[0033] Simultaneously, a second confidence factor is generated from the image quality dimension. This step quantitatively evaluates the image quality and background complexity of the image of the signature area to be reviewed.

[0034] Specifically, a second confidence factor is obtained based on the image quality and background complexity assessment of the image of the signature area to be reviewed, including: Image quality analysis is performed on the image of the signature area to be reviewed to obtain image sharpness score and noise level score; Background complexity analysis is performed on the image of the signature area to be reviewed to evaluate the degree of interference between the signature handwriting and the background pattern, text, and table lines, and to obtain a background interference score. The ratio of the image sharpness score to the preset benchmark image sharpness score is used as the first image confidence coefficient; The ratio of the preset baseline noise level score to the noise level score is used as the second image confidence coefficient; The ratio of the preset baseline background interference score to the background interference score is used as the third image confidence coefficient. The second confidence factor is obtained by weighting the confidence coefficients of the first image, the second image, and the third image.

[0035] First, image quality analysis is performed on the image of the signature area to be reviewed. This analysis aims to assess the physical imaging conditions of the image itself, generating two key scores: image sharpness score and noise level score. Specifically, the image sharpness score quantifies the focus state by analyzing the edge sharpness and detail retention of the signature area image; the noise level score quantifies the signal-to-noise ratio by assessing the abnormal fluctuations of random pixels in the signature area image. The image sharpness score and noise level score together reflect the potential difficulty in extracting stable and accurate visual features from the signature area image.

[0036] Secondly, background complexity analysis is performed on the image of the signature area to be reviewed. This background complexity analysis aims to assess the degree of visual interference from the environment in which the signature is located. By calculating the overlap and confusion between the signature handwriting and possible background patterns, text, table lines, and other elements in the image background in terms of texture, color, and spatial distribution, a background interference score can be generated. This background interference score quantifies the intensity of interference caused by background information to the correct segmentation and feature extraction of the signature handwriting.

[0037] Then, the raw scores obtained from the above analysis are converted into standardized confidence coefficients. Specifically, the image sharpness score is divided by a preset baseline image sharpness score to obtain the first image confidence coefficient. The preset baseline image sharpness score represents the level of sharpness achievable under ideal imaging conditions. It is set according to the typical sharpness value of a high-resolution scanning device imaging a clean document under standard lighting conditions, for example, the baseline score is set to 95 points.

[0038] Simultaneously, a second image confidence coefficient is obtained by dividing a preset baseline noise level score by the analyzed noise level score. This calculation method ensures that the lower the noise, the higher the second image confidence coefficient value. The preset baseline noise level score represents the theoretical lower limit of the score under ideal conditions without noise interference, set according to the minimum quantifiable threshold of the noise component in the image quality assessment model; for example, this baseline score is set to 5 points. Simultaneously, a third image confidence coefficient is obtained by dividing a preset baseline background interference score by the analyzed background interference score. This third image confidence coefficient reflects the cleanliness of the current background relative to an ideal interference-free background. The preset baseline background interference score represents the ideal score when the background is pure white or a single uniform color, completely free of interference patterns or text, set according to the theoretical optimal score for a clean background in the image complexity assessment model; for example, this baseline score is set to 10 points.

[0039] Finally, the three confidence coefficients reflecting image conditions from different dimensions are synthesized. Specifically, according to preset weight ratios, the confidence coefficients of the first, second, and third images are weighted and calculated, and the weighted sum is the final second confidence factor. Second confidence factor = w1 × first image confidence coefficient + w2 × second image confidence coefficient + w3 × third image confidence coefficient, where w1, w2, and w3 are weight coefficients, set based on prior knowledge of the impact of each image quality dimension on the signature feature extraction task; for example, w1 is set to 0.5, w2 to 0.3, and w3 to 0.2. This second confidence factor is a comprehensive evaluation value, and its value directly reflects the quality and reliability of the image of the signature area to be reviewed as a source of visual evidence. High-quality images with simple backgrounds will produce a second confidence factor approaching 1, while low-quality, high-noise, or complex background images will cause the second confidence factor to approach 0.

[0040] Finally, the evaluation results of the two dimensions are combined. The calculated first confidence factor and second confidence factor are weighted according to preset weights, and the weighted sum is the final visual evaluation confidence score. Specifically, visual evaluation confidence score = α × first confidence factor + β × second confidence factor, where α and β are weight coefficients, set according to the relative importance of sample sufficiency and image quality to the final visual judgment. For example, α is set to 0.6 and β to 0.4. This visual evaluation confidence score is a value between 0 and 1, and its value comprehensively reflects the overall reliability and confidence level of the task of signature verification based on the current visual information. When the reference sample is sufficient and the image quality is good, the visual evaluation confidence score approaches 1; when the reference sample is scarce or the image quality is poor, the visual evaluation confidence score approaches zero, indicating insufficient reliability of visual evidence.

[0041] S30: Configure the adaptation analysis depth parameters of the large language model based on the visual evaluation confidence, and perform signature semantic verification in combination with context information to obtain the signature semantic consistency coefficient; Furthermore, the adaptation analysis depth parameters of the large language model are configured based on the obtained visual evaluation confidence scores. A large language model is an artificial intelligence model with powerful semantic understanding and logical reasoning capabilities, such as the GPT series or ChatGLM series models, which are pre-trained on massive amounts of text data. Adaptation analysis depth parameters are configured for this large language model; these parameters are a set of dynamically adjustable control variables used to characterize the breadth, depth, and complexity of semantic analysis that the large language model should employ when handling the current signature verification task.

[0042] Specifically, the adaptation analysis depth parameters of the large language model are configured based on the visual evaluation confidence, including: The visual evaluation confidence is input into a preset parameter calculation function, and the output is an adaptive analysis depth parameter, wherein the analysis depth parameter includes at least the range ratio coefficient of context text extraction, the task identifier of associated text analysis, and the maximum inference chain length of logical reasoning. The smaller the visual evaluation confidence value input to the parameter calculation function, the larger the range ratio coefficient of the calculation output, the more complex the analysis task indicated by the task identifier, and the longer the maximum inference chain length.

[0043] The process of configuring adaptation analysis depth parameters for a large language model based on visual evaluation confidence involves a pre-defined mapping mechanism. The core of this mechanism is a pre-defined parameter calculation function, which takes the visual evaluation confidence as input and, through internal calculation, outputs a set of adaptation analysis depth parameters. These parameters include at least three key control variables: the scope ratio coefficient for contextual text extraction, the task identifier for related text analysis, and the maximum inference chain length for logical reasoning.

[0044] The contextual text extraction range ratio determines the proportion of page space around the signature area from which text information is extracted, serving as the contextual basis for semantic analysis. The task identifier for correlated text analysis is a code used to index a specific set of instructions, which clarifies the type and steps of semantic analysis tasks that the large language model needs to perform. The maximum length of the logical reasoning chain limits the maximum number of reasoning steps that the large language model is allowed to undertake during logical deduction; more steps generally indicate greater depth and complexity of analysis.

[0045] Specifically, the parameter calculation function is designed according to a core principle: the lower the confidence value of the input visual assessment, the lower the reliability of judgment based solely on visual evidence. To compensate for the lack of visual information, the parameter calculation function increases its reliance on semantic analysis. Therefore, the scope ratio of the extracted contextual text in the calculated output is larger, aiming to incorporate broader surrounding textual information to find supporting evidence. Simultaneously, the task identifier for correlated text analysis points to more complex analytical tasks, such as not only determining whether the signatory's name appears, but also verifying their position, the logical correlation of the signing date, and the completeness of multiple signing requirements. Furthermore, the maximum length of the logical reasoning chain is set longer to allow the model to perform deeper, multi-step causal or compliance reasoning, thereby making fuller use of the textual context for comprehensive judgment. This adaptive configuration mechanism ensures that the analytical depth and complexity of the large language model dynamically complement the strength of visual evidence, optimizing the robustness of the overall review.

[0046] Furthermore, semantic verification of the signature is performed in conjunction with contextual information to obtain a semantic consistency coefficient. Contextual information refers to the text content and structured data extracted from the entire page or related pages of the project document containing the signature to be reviewed, based on adaptation analysis depth parameters, and relevant to the current signature review task. This may include the signatory's name, position, date, description of the project matter, relevant clauses, and other related signature information. Semantic verification of the signature is a process of logical analysis and semantic reasoning based on the aforementioned contextual information, using a large language model to assess the rationality of the signature's existence, the matching of the signatory's identity, and the compliance of the signing behavior.

[0047] The final signature semantic consistency coefficient is a quantitative score that characterizes the comprehensive degree of semantic and logical consistency, identity matching, and behavioral compliance of the signature to be reviewed in a given document context.

[0048] Specifically, the signature semantic consistency coefficient is obtained by performing a signature semantic verification in conjunction with contextual information, including: The full-page document information containing the signature area to be reviewed, the coordinate information of the signature area, and the signature element information are input into the large language model running according to the analysis depth parameters. Using the large language model, the contextual text surrounding the signature area is extracted based on the range ratio coefficient, and it is determined whether the signer identity represented in the contextual text is consistent with the signer metadata, thus obtaining a semantic consistency judgment result. Using the large language model, based on the type of engineering documents and the preset signing rules, it is inferred whether it is reasonable for the current signature position to be signed by the current signer, and a logical rationality judgment result is obtained; The semantic consistency judgment result and the logical rationality judgment result are quantitatively scored and comprehensively weighted, and the weighted result is normalized to obtain the signature semantic consistency coefficient with a value between 0 and 1.

[0049] First, the entire page document information, including the signature area to be reviewed, the coordinate information of the signature area, and the signer's information are input into the large language model that has been configured according to the aforementioned adaptation analysis depth parameters. Specifically, the entire page document information is the complete image data of the page where the signature to be reviewed is located and its corresponding full-page structured text content obtained after optical character recognition processing; the coordinate information of the signature area is the bounding box data used to uniquely determine the location and range of the signature area to be reviewed in the page image, usually represented by pixel coordinates; the signer's information is the personal identification that the signature to be reviewed claims to represent, which at least includes the signer's name and can be extended to auxiliary identification information such as their organization, position, or employee number.

[0050] Secondly, a large language model is used to extract context-related text and determine identity consistency.

[0051] Specifically, the large language model performs the task of extracting context-related text, including: A rectangular analysis area is determined based on the coordinates of the signature area and the range scaling factor. Optical character recognition is performed on the image within the rectangular analysis area, and all recognized text blocks are used as a candidate text set. The large language model is used to filter out text lines or semantically coherent paragraphs containing preset identity tag keywords from the candidate text set, which are then used as context-related text.

[0052] First, using the center point of the bounding box defined by the signature area coordinates as the reference point, a rectangular analysis region is calculated and determined based on the range scaling factor. The range scaling factor directly controls the proportional relationship between this rectangular region and the original page image size. The larger the range scaling factor value, the larger the area of ​​the defined rectangular region, indicating that potential contextual semantic information is collected from a wider range of page space.

[0053] Secondly, optical character recognition (OCR) processing is performed on the portion of the page image covered by the rectangular analysis area. OCR is a computer vision technique that automatically converts images containing characters into editable text character sequences, aiming to accurately extract the text content carried by printed or neatly handwritten characters in an image. This OCR process outputs all successfully recognized independent text units within the rectangular area and their corresponding spatial location information, collectively forming a candidate text set. The final candidate text set contains all text content within the defined area, but has not yet been filtered based on its relevance to the signature verification task.

[0054] Finally, semantic relevance analysis and filtering are performed using a large language model. Specifically, the deep natural language understanding capabilities of the large language model are utilized to perform intelligent semantic analysis on the candidate text set. Based on its internalized knowledge system related to the engineering document signing scenario, the large language model identifies and extracts key semantic indications in the text related to the signatory's identity attributes, scope of responsibilities, or signing behavior. Specifically, the large language model filters out text lines containing preset identity tag keywords from the candidate text set, such as contextual statements that explicitly contain identifying fields such as signatory, reviewer, approver, and name. At the same time, the large language model identifies and extracts text paragraphs that, although not containing explicit keywords, are semantically coherent and clearly point to the description of the current signing matter or the explanation of the relevant responsible parties.

[0055] Through the aforementioned semantic understanding-based filtering process, the final determined text content is defined as context-dependent text. This context-dependent text is highly relevant to the signature to be reviewed in terms of business processes and logic, providing direct and reliable semantic basis for subsequent verification of the signer's identity and logical reasoning regarding the signing behavior.

[0056] Subsequently, the large language model analyzes the contextual text, identifying the signatory's identity information, such as name, position, or organization, whether explicitly mentioned or implicitly indicated. This identified signatory identity is then compared with the input signatory metadata. The comparison outputs a semantic consistency judgment result regarding identity matching. This semantic consistency judgment result is a discrete classification conclusion indicating the degree of identity matching, used to initially verify the veracity of the signatory's statement at the semantic level. For example, the conclusion can be "yes," "no," or "uncertain."

[0057] Furthermore, a deeper level of logical reasoning is performed using a large language model. Specifically, based on its understanding of the entire document's content, the large language model automatically determines the specific type of the current engineering document, such as a construction log, acceptance form, or change order record. Combining this with pre-defined signing rules related to that document type, it infers whether it is logical and compliant for the claimed signatory to sign at that specific location in the document. For example, it determines whether the signatory has the appropriate authority and whether the signing order conforms to process requirements. Through this reasoning, the large language model can output a logically sound judgment result regarding the reasonableness of the signing behavior.

[0058] Specifically, the result of this logical reasonableness judgment is a qualitative conclusion based on compliance and logical analysis, used to assess the procedural correctness and business appropriateness of the signature to be audited in the context of the document. For example, the conclusion can be "reasonable", "unreasonable" or "uncertain".

[0059] Finally, the judgment results of the above two dimensions are comprehensively quantified. The semantic consistency judgment result and the logical rationality judgment result are converted into numerical scores respectively, and then weighted and summed.

[0060] Specifically, the semantic consistency judgment result and the logical rationality judgment result are quantitatively scored and comprehensively weighted, including: Assign a first weight to the semantic consistency judgment result and a second weight to the logical rationality judgment result; Calculate the scores for semantic consistency and logical rationality separately. A full score is given when the judgment result is affirmative, a zero score is given when the judgment result is negative, and an intermediate score between the full score and the zero score is given when the judgment result is uncertain. The score of the semantic consistency judgment result is weighted using the first weight, and the score of the logical rationality judgment result is weighted using the second weight. The two weighted results are added together and then normalized to obtain the signature semantic consistency coefficient.

[0061] First, a first weight is assigned to the semantic consistency judgment result, and a second weight is assigned to the logical rationality judgment result. The specific values ​​of the first and second weights are pre-set based on the relative importance of the two judgments in the final semantic verification conclusion. For example, the first weight can be set to 0.6 and the second weight to 0.4 to reflect the fundamental role of direct identity matching in semantic verification.

[0062] Secondly, regarding the semantic consistency judgment results, a full score (e.g., 1.0) is assigned when the judgment result is affirmative (identity match); a zero score (0.0) is assigned when the judgment result is negative (identity mismatch); and an intermediate score (e.g., 0.5) is assigned when the judgment result is uncertain. Similarly, the same quantification rules are applied to the logical rationality judgment results: a full score is assigned when the judgment result is reasonable, a zero score is assigned when it is unreasonable, and an intermediate score is assigned when it is uncertain.

[0063] Finally, the scores for semantic consistency are weighted using the first weight, and the scores for logical rationality are weighted using the second weight. These weighted scores are then summed to obtain a raw total score. Since the maximum score and weights may cause the theoretical range of the raw total score to exceed 0 to 1, it needs to be normalized. For example, it can be divided by the theoretical maximum value of the product of the maximum score and the total weights, thus mapping the final result to a standard numerical range of 0 to 1. This normalized final value is the signature semantic consistency coefficient. This coefficient comprehensively reflects the strength of the semantic verification conclusion; the closer the value is to 1, the higher the certainty of the semantic verification.

[0064] S40: Based on the visual assessment confidence level, set the adaptive visual weight and the adaptive semantic weight, perform a weighted calculation on the signature similarity coefficient and the signature semantic consistency coefficient to obtain the signature confidence level, and complete the review decision based on the signature confidence level.

[0065] Finally, based on the obtained visual assessment confidence level, adaptive visual weights and adaptive semantic weights are set. Specifically, the adaptive visual weight is a dynamically calculated numerical coefficient representing the proportion of importance that the signature similarity coefficient obtained based on handwriting visual feature similarity should occupy in the final comprehensive decision in the current review task; the adaptive semantic weight is also a dynamically calculated numerical coefficient representing the proportion of importance that the signature semantic consistency coefficient obtained based on contextual semantic logic analysis should occupy in the final comprehensive decision.

[0066] The sum of the visual weight and the semantic weight is 1. Together, they form a dynamic weight allocation scheme to balance the impact of visual evidence and semantic evidence on the final review conclusion.

[0067] Specifically, the adaptation visual weight and adaptation semantic weight are set according to the visual evaluation confidence level, including: The adaptive visual weight is obtained by multiplying the ratio of the visual evaluation confidence level to the preset benchmark visual evaluation confidence level by the initial visual weight, wherein the initial visual weight is 0.5, and if the adaptive visual weight is greater than 0.65, it is taken as 0.65, and if the adaptive visual weight is less than 0.35, it is taken as 0.35. The adaptation semantic weight is obtained by subtracting the adaptation visual weight from 1.

[0068] First, the adaptive visual weights are calculated. This calculation is based on the visual assessment confidence level, which is compared to a preset baseline visual assessment confidence level. The preset baseline visual assessment confidence level is a predefined empirical threshold representing the typical confidence level corresponding to sufficient reliability of visual evidence in a typical engineering document review scenario. It is set based on the median of the confidence level interval where the visual model performs stably in historical review data; for example, this baseline value is set to 0.75. The visual assessment confidence level is divided by this preset baseline visual assessment confidence level to obtain a ratio. This ratio is then multiplied by an initial visual weight, which is fixed at 0.5. The resulting product is the initial adaptive visual weight value.

[0069] To ensure the stability and robustness of the final decision and to avoid excessively biased weight allocation due to extreme fluctuations in visual assessment confidence, upper and lower limits were imposed on the initial calculated value. If the initially calculated adaptive visual weight is greater than 0.65, it is forcibly set to 0.65; if it is less than 0.35, it is forcibly set to 0.35. The value obtained after this constraint processing is the final adaptive visual weight.

[0070] Secondly, the adaptation semantic weight is calculated. Since the adaptation visual weight and the adaptation semantic weight together constitute the entire basis of the decision, and their sum should be 1, the adaptation semantic weight is obtained by subtracting the previously determined adaptation visual weight from 1. That is, adaptation semantic weight = 1 - adaptation visual weight.

[0071] Through the above steps, a set of dynamic weight pairs closely related to the reliability of the current visual evidence is obtained. When the confidence of the visual evaluation is high, the adaptive visual weights tend to take higher values, while the adaptive semantic weights decrease accordingly, and the decision relies more on visual feature matching. When the confidence of the visual evaluation is low, the adaptive visual weights tend to take lower values, while the adaptive semantic weights increase accordingly, and the decision relies more on the semantic logic analysis of the large language model.

[0072] Furthermore, based on the calculated adaptive visual weights and adaptive semantic weights, the signature similarity coefficient and signature semantic consistency coefficient are weighted to obtain the signature confidence level, and the review decision is completed based on the signature confidence level.

[0073] Adaptive visual weight and adaptive semantic weight are dynamic adjustment coefficients used to fuse multimodal verification evidence, affecting the signature similarity coefficient and signature semantic consistency coefficient, respectively. Specifically, the signature confidence score is calculated using the following weighted formula: Signature Confidence Score = (Adaptive Visual Weight × Signature Similarity Coefficient) + (Adaptive Semantic Weight × Signature Semantic Consistency Coefficient). This calculation process organically fuses the signature similarity coefficient, which characterizes the visual matching degree of handwriting, with the signature semantic consistency coefficient, which characterizes the logical rationality of the context, based on the reliability of the current visual evidence. The sum of the adaptive visual weight and the adaptive semantic weight is always 1, ensuring the normalization of the fusion result.

[0074] The system makes a review decision based on the calculated signature confidence level. For example, the system may pre-set one or more signature confidence level decision thresholds. For instance, it may set an approval threshold and a rejection threshold. If the calculated signature confidence level is greater than or equal to the approval threshold, the system automatically approves the review; if the signature confidence level is less than or equal to the rejection threshold, the system automatically rejects the review; if the signature confidence level is between the rejection threshold and the approval threshold, the system decides that manual review is required.

[0075] In this way, the final review conclusion not only deeply integrates the quantitative strength of both visual features and semantic logic evidence, but also intuitively reflects the degree of control of automated decision-making through the continuous value of signature confidence. At the same time, uncertain cases in the confidence gray area are automatically screened and diverted to the manual processing stage, thereby improving the efficiency of batch review processing while effectively ensuring the overall reliability and risk controllability of the review conclusion.

[0076] In summary, the embodiments of this application have at least the following technical effects: This invention innovatively constructs a dynamic and collaborative auditing architecture by introducing visual evaluation confidence as a core regulatory variable. First, it overcomes the performance instability of traditional visual models under low-sample conditions by using a feature encoder trained with joint loss and multi-factor confidence evaluation to achieve a quantitative measurement of the reliability of visual signature matching. Second, it breaks the isolated working mode of visual and language models by adaptively configuring the analysis depth of the large language model using visual evaluation confidence, making the resource investment in semantic analysis complementary to the strength of visual evidence, and initiating deeper semantic reasoning to strengthen the visual evidence when it is weak. Third, it proposes a mechanism for dynamically allocating fusion weights based on the same visual evaluation confidence, enabling the final comprehensive signature confidence to objectively reflect the relative credibility of visual and semantic evidence in different scenarios, avoiding the rigidity of fixed-weight fusion.

[0077] Finally, by introducing a multi-threshold adjudication mechanism, a tiered output of review conclusions was achieved. This improved automation efficiency while accurately diverting uncertain cases to manual review, effectively balancing review speed and accuracy. Overall, this invention enhances the adaptability, accuracy, and decision interpretability of automated engineering document signature review in complex real-world scenarios.

[0078] Example 2, as Figure 2 As shown, based on the same inventive concept as the intelligent engineering document review method combining deep learning and LLM provided in Embodiment 1, this embodiment of the invention also provides an intelligent engineering document review system combining deep learning and LLM, including: The signature area recognition and extraction module 11 is used to recognize and extract the signature area of ​​the engineering data document image to be reviewed, and obtain the signature area image to be reviewed. The signature feature encoding and calculation module 12 is used to process the image of the signature region to be audited using the signature feature encoder, and calculate the signature similarity coefficient and visual evaluation confidence. The semantic verification module 13 is used to configure the adaptation analysis depth parameters of the large language model based on the visual evaluation confidence, and to perform signature semantic verification in combination with context information to obtain the signature semantic consistency coefficient. The review and adjudication module 14 is used to set the adaptive visual weight and the adaptive semantic weight according to the visual evaluation confidence level, calculate the signature confidence level by weighting the signature similarity coefficient and the signature semantic consistency coefficient, and complete the review and adjudication based on the signature confidence level.

[0079] The signature region identification and extraction module 11 is specifically used for: The signature area of ​​the engineering documents to be reviewed is identified and extracted to obtain the signature area image to be reviewed.

[0080] Specifically, the signature feature encoding and calculation module 12 is used for: Specifically, the image of the signature region to be audited is processed using a signature feature encoder to calculate the signature similarity coefficient and visual evaluation confidence level, including: A signature feature encoder is generated by jointly training a deep convolutional neural network based on the ArcFace loss function and the triplet loss function. Using the signature feature encoder, the signature feature vector to be audited in the signature region image to be audited is extracted, and the reference signature feature vector set of the valid reference signature of the corresponding signer is extracted from the low sample reference signature library. Calculate the cosine similarity between the signature feature vector to be audited and each reference signature feature vector in the reference signature feature vector set, and use the maximum cosine similarity value as the signature similarity coefficient; The first confidence factor is obtained by evaluating the actual number of valid reference signature samples of the corresponding signer in the low-sample reference signature library, wherein the ratio of the actual number of valid reference signature samples to the number of preset benchmark samples is used as the first confidence factor. The second confidence factor is obtained based on the image quality and background complexity of the image of the signature area to be reviewed; The first confidence factor and the second confidence factor are weighted and calculated to obtain the visual assessment confidence.

[0081] Specifically, a deep convolutional neural network is jointly trained based on the ArcFace loss function and the triplet loss function to generate a signature feature encoder, including: The basic network structure for constructing a signature feature encoder is based on a deep convolutional neural network; Prepare a sample training dataset containing signature image samples from multiple different signers, where each signer corresponds to at least one signature image sample; Construct a joint loss function, wherein the joint loss function is a weighted combination of the ArcFace loss function and the triplet loss function, used to simultaneously constrain the inter-class separability of feature vectors in the angle space and the relative distance relationship between sample pairs during training; Using the sample training dataset and the joint loss function, end-to-end supervised training is performed on the deep convolutional neural network until the model converges, and the trained deep convolutional neural network is used as the signature feature encoder.

[0082] Specifically, a second confidence factor is obtained based on the image quality and background complexity assessment of the image of the signature area to be reviewed, including: Image quality analysis is performed on the image of the signature area to be reviewed to obtain image sharpness score and noise level score; Background complexity analysis is performed on the image of the signature area to be reviewed to evaluate the degree of interference between the signature handwriting and the background pattern, text, and table lines, and to obtain a background interference score. The ratio of the image sharpness score to the preset benchmark image sharpness score is used as the first image confidence coefficient; The ratio of the preset baseline noise level score to the noise level score is used as the second image confidence coefficient; The ratio of the preset baseline background interference score to the background interference score is used as the third image confidence coefficient. The second confidence factor is obtained by weighting the confidence coefficients of the first image, the second image, and the third image.

[0083] The semantic verification module 13 is specifically used for: Based on the aforementioned visual evaluation confidence level, the adaptation analysis depth parameters of the large language model are configured, including: The visual evaluation confidence is input into a preset parameter calculation function, and the output is an adaptive analysis depth parameter, wherein the analysis depth parameter includes at least the range ratio coefficient of context text extraction, the task identifier of associated text analysis, and the maximum inference chain length of logical reasoning. The smaller the visual evaluation confidence value input to the parameter calculation function, the larger the range ratio coefficient of the calculation output, the more complex the analysis task indicated by the task identifier, and the longer the maximum inference chain length.

[0084] The signature semantic consistency coefficient is obtained by performing a signature semantic verification in conjunction with contextual information, including: The full-page document information containing the signature area to be reviewed, the coordinate information of the signature area, and the signature element information are input into the large language model running according to the analysis depth parameters. Using the large language model, the contextual text surrounding the signature area is extracted based on the range ratio coefficient, and it is determined whether the signer identity represented in the contextual text is consistent with the signer metadata, thus obtaining a semantic consistency judgment result. Using the large language model, based on the type of engineering documents and the preset signing rules, it is inferred whether it is reasonable for the current signature position to be signed by the current signer, and a logical rationality judgment result is obtained; The semantic consistency judgment result and the logical rationality judgment result are quantitatively scored and comprehensively weighted, and the weighted result is normalized to obtain the signature semantic consistency coefficient with a value between 0 and 1.

[0085] The large language model performs the task of extracting context-related text, including: A rectangular analysis area is determined based on the coordinates of the signature area and the range scaling factor. Optical character recognition is performed on the image within the rectangular analysis area, and all recognized text blocks are used as a candidate text set. The large language model is used to filter out text lines or semantically coherent paragraphs containing preset identity tag keywords from the candidate text set, which are then used as context-related text.

[0086] Specifically, the semantic consistency judgment result and the logical rationality judgment result are quantitatively scored and comprehensively weighted, including: Assign a first weight to the semantic consistency judgment result and a second weight to the logical rationality judgment result; Calculate the scores for semantic consistency and logical rationality separately. A full score is given when the judgment result is affirmative, a zero score is given when the judgment result is negative, and an intermediate score between the full score and the zero score is given when the judgment result is uncertain. The score of the semantic consistency judgment result is weighted using the first weight, and the score of the logical rationality judgment result is weighted using the second weight. The two weighted results are added together and then normalized to obtain the signature semantic consistency coefficient.

[0087] Specifically, the review and adjudication module 14 is used for: Based on the aforementioned visual evaluation confidence level, adaptive visual weights and adaptive semantic weights are set, including: The adaptive visual weight is obtained by multiplying the ratio of the visual evaluation confidence level to the preset benchmark visual evaluation confidence level by the initial visual weight, wherein the initial visual weight is 0.5, and if the adaptive visual weight is greater than 0.65, it is taken as 0.65, and if the adaptive visual weight is less than 0.35, it is taken as 0.35. The adaptation semantic weight is obtained by subtracting the adaptation visual weight from 1.

Claims

1. A method for intelligent review of engineering documents combining deep learning and LLM, characterized in that, The methods include: The signature area of ​​the engineering documents to be reviewed is identified and extracted to obtain the signature area image to be reviewed. The image of the signature region to be audited is processed using a signature feature encoder to calculate the signature similarity coefficient and visual evaluation confidence level, including: A signature feature encoder is generated by jointly training a deep convolutional neural network based on the ArcFace loss function and the triplet loss function. Using the signature feature encoder, the signature feature vector to be audited in the signature region image to be audited is extracted, and the reference signature feature vector set of the valid reference signature of the corresponding signer is extracted from the low sample reference signature library. Calculate the cosine similarity between the signature feature vector to be audited and each reference signature feature vector in the reference signature feature vector set, and use the maximum cosine similarity value as the signature similarity coefficient; The first confidence factor is obtained by evaluating the actual number of valid reference signature samples of the corresponding signer in the low-sample reference signature library, wherein the ratio of the actual number of valid reference signature samples to the number of preset benchmark samples is used as the first confidence factor. The second confidence factor is obtained based on the image quality and background complexity of the image of the signature area to be reviewed; The first confidence factor and the second confidence factor are weighted and calculated to obtain the visual assessment confidence level; Based on the visual evaluation confidence level, the adaptation analysis depth parameters of the large language model are configured, and the signature semantic consistency coefficient is obtained by performing signature semantic verification in combination with context information. Based on the visual assessment confidence level, adaptive visual weights and adaptive semantic weights are set, and the signature similarity coefficient and signature semantic consistency coefficient are weighted and calculated to obtain the signature confidence level. The review decision is then completed based on the signature confidence level. The adaptation analysis depth parameters of the large language model are configured based on the visual evaluation confidence level, including: The visual evaluation confidence is input into a preset parameter calculation function, and the adaptation analysis depth parameter is output. The adaptation analysis depth parameter includes at least the range ratio coefficient of context text extraction, the task identifier of associated text analysis, and the maximum inference chain length of logical reasoning. The smaller the visual evaluation confidence value input to the parameter calculation function, the larger the range ratio coefficient of the calculation output, the more complex the analysis task indicated by the task identifier, and the longer the maximum inference chain length.

2. The intelligent review method for engineering data combining deep learning and LLM as described in claim 1, characterized in that, A deep convolutional neural network is jointly trained using the ArcFace loss function and the triplet loss function to generate a signature feature encoder, including: The basic network structure for constructing a signature feature encoder is based on a deep convolutional neural network; Prepare a sample training dataset containing signature image samples from multiple different signers, where each signer corresponds to at least one signature image sample; Construct a joint loss function, wherein the joint loss function is a weighted combination of the ArcFace loss function and the triplet loss function, used to simultaneously constrain the inter-class separability of feature vectors in the angle space and the relative distance relationship between sample pairs during training; Using the sample training dataset and the joint loss function, end-to-end supervised training is performed on the deep convolutional neural network until the model converges, and the trained deep convolutional neural network is used as the signature feature encoder.

3. The intelligent review method for engineering data combining deep learning and LLM as described in claim 1, characterized in that, The second confidence factor is obtained based on the image quality and background complexity assessment of the signature area image to be reviewed, including: Image quality analysis is performed on the image of the signature area to be reviewed to obtain image sharpness score and noise level score; Background complexity analysis is performed on the image of the signature area to be reviewed to evaluate the degree of interference between the signature handwriting and the background pattern, text, and table lines, and to obtain a background interference score. The ratio of the image sharpness score to the preset benchmark image sharpness score is used as the first image confidence coefficient; The ratio of the preset baseline noise level score to the noise level score is used as the second image confidence coefficient; The ratio of the preset baseline background interference score to the background interference score is used as the third image confidence coefficient. The second confidence factor is obtained by weighting the confidence coefficients of the first image, the second image, and the third image.

4. The intelligent review method for engineering data combining deep learning and LLM as described in claim 1, characterized in that, The signature semantic consistency coefficient is obtained by performing a signature semantic verification in conjunction with contextual information, including: The full-page document information containing the signature area to be reviewed, the coordinate information of the signature area, and the signature element information are input into the large language model running according to the adaptive analysis depth parameters. Using the large language model, the contextual text surrounding the signature area is extracted based on the range ratio coefficient, and it is determined whether the signer identity represented in the contextual text is consistent with the signer metadata, thus obtaining a semantic consistency judgment result. Using the large language model, based on the type of engineering documents and the preset signing rules, it is inferred whether it is reasonable for the current signature position to be signed by the current signer, and a logical rationality judgment result is obtained; The semantic consistency judgment result and the logical rationality judgment result are quantitatively scored and comprehensively weighted, and the weighted result is normalized to obtain the signature semantic consistency coefficient with a value between 0 and 1.

5. The intelligent review method for engineering data combining deep learning and LLM as described in claim 4, characterized in that, The large language model performs the task of extracting context-related text, including: A rectangular analysis area is determined based on the coordinates of the signature area and the range scaling factor. Optical character recognition is performed on the image within the rectangular analysis area, and all recognized text blocks are used as a candidate text set. The large language model is used to filter out text lines or semantically coherent paragraphs containing preset identity tag keywords from the candidate text set, which are then used as context-related text.

6. The intelligent review method for engineering data combining deep learning and LLM as described in claim 4, characterized in that, The semantic consistency judgment results and logical rationality judgment results are quantitatively scored and comprehensively weighted, including: Assign a first weight to the semantic consistency judgment result and a second weight to the logical rationality judgment result; Calculate the scores for semantic consistency and logical rationality separately. A full score is given when the judgment result is affirmative, a zero score is given when the judgment result is negative, and an intermediate score between the full score and the zero score is given when the judgment result is uncertain. The score of the semantic consistency judgment result is weighted using the first weight, and the score of the logical rationality judgment result is weighted using the second weight. The two weighted results are added together and then normalized to obtain the signature semantic consistency coefficient.

7. The intelligent review method for engineering data combining deep learning and LLM as described in claim 1, characterized in that, Based on the aforementioned visual evaluation confidence level, adaptive visual weights and adaptive semantic weights are set, including: The adaptive visual weight is obtained by multiplying the ratio of the visual evaluation confidence level to the preset benchmark visual evaluation confidence level by the initial visual weight, wherein the initial visual weight is 0.5, and if the adaptive visual weight is greater than 0.65, it is taken as 0.65, and if the adaptive visual weight is less than 0.35, it is taken as 0.

35. The adaptation semantic weight is obtained by subtracting the adaptation visual weight from 1.

8. An intelligent engineering document review system combining deep learning and LLM, characterized in that, The method for intelligent review of engineering documents combining deep learning and LLM as described in any one of claims 1-7 includes: The signature area recognition and extraction module is used to recognize and extract the signature area of ​​the engineering documents to be reviewed, and obtain the signature area image to be reviewed. The signature feature encoding and calculation module is used to process the image of the signature region to be audited using a signature feature encoder, and to calculate the signature similarity coefficient and visual evaluation confidence level, including: A signature feature encoder is generated by jointly training a deep convolutional neural network based on the ArcFace loss function and the triplet loss function. Using the signature feature encoder, the signature feature vector to be audited in the signature region image to be audited is extracted, and the reference signature feature vector set of the valid reference signature of the corresponding signer is extracted from the low sample reference signature library. Calculate the cosine similarity between the signature feature vector to be audited and each reference signature feature vector in the reference signature feature vector set, and use the maximum cosine similarity value as the signature similarity coefficient; The first confidence factor is obtained by evaluating the actual number of valid reference signature samples of the corresponding signer in the low-sample reference signature library, wherein the ratio of the actual number of valid reference signature samples to the number of preset benchmark samples is used as the first confidence factor. The second confidence factor is obtained based on the image quality and background complexity of the image of the signature area to be reviewed; The first confidence factor and the second confidence factor are weighted and calculated to obtain the visual assessment confidence level; The semantic verification module is used to configure the adaptation analysis depth parameters of the large language model based on the visual evaluation confidence level, and to perform signature semantic verification in combination with context information to obtain the signature semantic consistency coefficient. The review and adjudication module is used to set the adaptive visual weight and the adaptive semantic weight according to the visual evaluation confidence level, calculate the signature confidence level by weighting the signature similarity coefficient and the signature semantic consistency coefficient, and complete the review and adjudication based on the signature confidence level. The adaptation analysis depth parameters of the large language model are configured based on the visual evaluation confidence level, including: The visual evaluation confidence is input into a preset parameter calculation function, and the adaptation analysis depth parameter is output. The adaptation analysis depth parameter includes at least the range ratio coefficient of context text extraction, the task identifier of associated text analysis, and the maximum inference chain length of logical reasoning. The smaller the visual evaluation confidence value input to the parameter calculation function, the larger the range ratio coefficient of the calculation output, the more complex the analysis task indicated by the task identifier, and the longer the maximum inference chain length.

Images