Papermaking process fiber visual language auxiliary quality analysis method and system

Abstract

The invention discloses a papermaking process fiber visual language assisted quality analysis method and system. The method comprises the following steps: collecting a microscopic image of a paper pulp sample in a papermaking process; a semantic segmentation mask is generated through a semantic segmentation model, and structured parameters of fibers are calculated and converted into standardized text description; extracting texture features of the microscopic image and topological structure features of a semantic segmentation mask, converting standardized text description into a high-dimensional parameter embedding vector, and generating a multi-modal query vector through cross attention fusion and semantic alignment; by taking the multi-modal query vector as a retrieval condition, executing mixed retrieval in the papermaking fiber knowledge base, and recalling Top-K related knowledge entries to form a retrieval context; and the standardized text description, the image texture features, the topological structure features of the segmentation mask and the retrieval context are spliced according to a preset cue word template and input into a large language model decoder, and a quality analysis result is generated. According to the invention, the accuracy of fiber anomaly identification can be obviously improved.

Description

Technical Field

[0001] This invention relates to the field of fiber quality analysis technology, specifically to a method and system for fiber visual language-assisted quality analysis in the papermaking process. Background Technology

[0002] In the papermaking industry, the microscopic morphological characteristics of pulp fibers (such as length, width, thickness, kink index, crimp index, and fibrillation degree) are key factors determining the physical properties of paper (such as tensile strength, tear strength, and folding endurance) and processing properties (such as water permeability and forming uniformity). Therefore, accurate and efficient detection and analysis of fiber quality are important prerequisites for achieving stable control of the papermaking process and improving product quality.

[0003] Currently, mainstream fiber quality testing relies primarily on commercial fiber analyzers. These devices typically combine optical microscopy with image processing algorithms to automatically measure fiber geometric parameters and output structured numerical reports. However, such systems only provide raw data and lack the ability to semantically interpret the meaning of the parameters. Operators must rely on their accumulated process experience to correlate discrete parameters with specific production problems (such as over-beating or abnormal raw materials) to make effective process adjustment decisions. This process is highly dependent on manual labor, inefficient, and prone to subjective judgment bias, easily leading to delays in process adjustments, resulting in raw material waste and product quality fluctuations.

[0004] In recent years, some studies have attempted to apply general visual language models to industrial image analysis scenarios. However, these general models face significant limitations in paper fiber analysis tasks: on the one hand, their training data do not cover the professional terminology and fine-grained morphological features of the paper industry (such as "fiber brooming," "cell contamination," and "fiber brittle fracture"), making it difficult to accurately identify and describe key defects; on the other hand, general models lack knowledge of paper industry standards (such as GB / T and TAPPI series standards), cannot determine whether parameters are compliant, and cannot establish a causal relationship between fiber morphology and paper performance or process parameters, resulting in output results lacking engineering guidance value. Furthermore, some studies have explored the use of Large Language Models (LLMs) for generating industrial diagnostic reports. However, due to the lack of external knowledge constraints, such generative models are prone to generating illusory content that does not conform to reality. For example, they may misclassify crimp within the normal range as abnormal and make incorrect process recommendations accordingly. Industry tests show that without the introduction of domain knowledge verification mechanisms, generative models have an excessively high rate of illusory results in fiber quality diagnosis, making the results unreliable and failing to meet the stringent reliability and safety requirements of industrial production.

[0005] In summary, existing technologies for fiber quality analysis generally suffer from three major bottlenecks: first, traditional detection systems lack semantic interpretation and decision support capabilities; second, general AI models lack expertise in the papermaking field, making accurate understanding difficult; and third, generative models lack reliability without knowledge constraints, posing a high risk of illusion. Therefore, there is an urgent need for an intelligent analysis method that integrates high-precision visual perception, structured parameter modeling, and domain knowledge enhancement to achieve end-to-end, interpretable, and low-illusion automated reasoning from image data to actionable process recommendations, supporting the intelligent upgrading and closed-loop process optimization of the papermaking industry. Summary of the Invention

[0006] To overcome the defects and shortcomings of existing technologies, this invention provides a method and system for fiber visual language-assisted quality analysis in the papermaking process. It can realize end-to-end automated, interpretable, and highly reliable intelligent analysis from microscopic images of pulp fibers to executable process suggestions. It effectively solves key technical problems in existing technologies, such as the lack of semantic interpretation capabilities of image parameters, the lack of professional knowledge in the papermaking field in general AI models, and the tendency of generative models to produce illusions.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a method for fiber visual language-assisted quality analysis in the papermaking process, comprising the following steps: Microscopic images of pulp samples collected during the papermaking process; The microscopic images are used to generate semantic segmentation masks through a semantic segmentation model. The structural parameters of the fibers are calculated based on image morphology analysis methods and then converted into standardized text descriptions. The visual encoder extracts the texture features of the microscopic image and the topological features of the semantic segmentation mask. The structural parameter embedding layer transforms the standardized text description into a high-dimensional parameter embedding vector. The image texture features, the topological features of the segmentation mask, and the high-dimensional parameter embedding vector are fused by cross-attention and semantic alignment through a cross-modal projector to generate a multimodal query vector. Using multimodal query vectors as retrieval criteria, a hybrid retrieval is performed in a pre-built paper fiber knowledge base to recall the Top-K relevant knowledge entries to form a retrieval context; Standardized text descriptions, image texture features, topological features of segmentation masks, and retrieval context are concatenated according to preset prompt word templates, input into a large language model decoder, and quality analysis results are generated.

[0008] As a preferred technical solution, the visual encoder includes a first sub-encoder and a second sub-encoder; The first sub-encoder uses ViT-L / 14 as the backbone network to extract the texture features of the microscopic image, performs pixel cropping and bilinear interpolation scaling on the microscopic image, performs normalization processing, subtracts the ImageNet mean and divides it by its standard deviation to generate a Float32 tensor. The second sub-encoder adopts the SigLIP architecture to extract the topological features of the semantic segmentation mask, convert the mask into a one-hot encoded form, and generate a multi-channel Float32 tensor.

[0009] As a preferred technical solution, the structural parameter embedding layer uses a fully connected neural network to embed the normalized structural parameters and generate a multi-dimensional parameter embedding vector.

[0010] As a preferred technical solution, the cross-modal projector adopts a Q-Former structure, which performs cross-attention fusion on image texture features, topological features of segmentation masks and high-dimensional parameter embedding vectors, and projects the fusion result onto the input semantic space of a large language model.

[0011] As a preferred technical solution, the cross-modal projector concatenates image texture features, topological features of the segmentation mask, and high-dimensional parameter embedding vectors into context key-value pairs, and maps them to the latent space of Q-Former through a linear projection layer. The query vector can be learned as the query vector for cross-attention. Cross-attention is calculated in each Transformer layer, gradually aggregating texture, topology, and numerical semantic information. Finally, the query vector is output after average pooling.

[0012] As a preferred technical solution, the cross-modal projector adopts a multilayer perceptron, which concatenates image texture features, topological structure features of segmentation masks and high-dimensional parameter embedding vectors along the channel dimension, and sequentially passes them through multiple fully connected layers. Each layer is followed by LayerNorm and GELU activation, and the last layer has no activation function.

[0013] As a preferred technical solution, a hybrid search is performed in a pre-constructed paper fiber knowledge base to retrieve the Top-K relevant knowledge entries to form a search context, specifically including: Structured filtering is performed based on the metadata of the current production scenario to select a subset of candidate entries that meet the criteria. The cosine similarity matching algorithm is used to recall the Top-K most relevant knowledge entries in the dense vector index as the retrieval context.

[0014] As a preferred technical solution, a cosine similarity matching algorithm is used to recall the Top-K most relevant knowledge entries in a dense vector index as the retrieval context, specifically including: The normalized multimodal query vector and the subset of candidate entries are used to calculate cosine similarity. The IVF_FLAT index is used to accelerate the approximate nearest neighbor search and recall the Top-K entries. For the recalled Top-K items, calculate the cosine similarity between each pair. If the similarity between any two items is greater than a set threshold, it is considered semantic redundancy. The item with the higher score is retained, and the corresponding knowledge item is output as the retrieval context.

[0015] As a preferred technical solution, the paper fiber knowledge base includes unstructured text data and structured index data; The unstructured text data is transformed into a multidimensional dense vector using the BGE-M3 model and a dense index is constructed. The structured index data is used to construct a keyword index using the BM25 algorithm, and metadata fields such as raw material type, paper type, and production process are added for filtering and retrieval.

[0016] The present invention also provides a fiber visual language-assisted quality analysis system for papermaking process, used to implement the above-mentioned fiber visual language-assisted quality analysis method for papermaking process, including: a data acquisition module, a data preprocessing module, a knowledge base construction module, a multimodal joint coding module, a retrieval-enhanced reasoning module, and a quality analysis generation module; The data acquisition module is used to collect microscopic images of pulp samples during the papermaking process; The data preprocessing module is used to input the microscopic image into the semantic segmentation model to generate a semantic segmentation mask, calculate the structural parameters of the fiber based on the image morphology analysis method, and convert it into a standardized text description. The knowledge base construction module is used to construct a paper fiber knowledge base; The multimodal joint coding module is used to extract image texture features and topological structure features of the segmentation mask from the microscopic image and semantic segmentation mask through a visual encoder. The standardized text description is transformed into a high-dimensional parameter embedding vector through a structural parameter embedding layer. The image texture features, topological structure features of the segmentation mask and high-dimensional parameter embedding vector are cross-attention fusion and semantic alignment through a cross-modal projector to generate a multimodal query vector. The retrieval enhancement reasoning module is used to perform hybrid retrieval in a pre-built paper fiber knowledge base using multimodal query vectors as retrieval conditions, and recall the Top-K relevant knowledge entries to form a retrieval context; The quality analysis generation module is used to concatenate standardized text descriptions, image texture features, topological features of segmentation masks, and retrieval context according to preset prompt word templates, input them into the large language model decoder, and generate quality analysis results.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) This invention uses a visual-structure-language ternary joint coding architecture to deeply integrate microscopic image texture, segmentation topology and structured parameters in a unified semantic space, which effectively improves the model’s recognition accuracy of key abnormal morphologies such as brittle fracture, insufficient brooming and contamination by miscellaneous cells. It effectively solves the technical bottleneck that pure numerical analysis cannot capture visual semantic associations and significantly improves the accuracy of fiber abnormality recognition.

[0018] (2) The present invention shortens the end-to-end processing time from image acquisition to process suggestion generation. The output report directly contains executable operation instructions, forming a closed loop of "detection-analysis-decision-adjustment". It can significantly improve the efficiency of the papermaking process, reduce the waste of raw materials caused by fiber quality problems, and improve the paper qualification rate. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the fiber visual language-assisted quality analysis method for the papermaking process of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] Example 1 like Figure 1 As shown, this embodiment provides a fiber visual language-assisted quality analysis method for the papermaking process, including the following steps: S1: Acquire microscopic images of pulp samples during the papermaking process using an industrial microscopic imaging system. The resolution of the microscopic images is no less than 1920×1080 pixels, and the magnification is no less than 200 times. A transmission light source is used to ensure clear fiber edges. The microscopic images are input into a semantic segmentation model finely tuned by a papermaking fiber dataset. The fiber body, fine fibers, miscellaneous cells, and background areas in the images are segmented at the pixel level to generate a semantic segmentation mask containing the fiber body, fine fibers, miscellaneous cells, and background areas. In this embodiment, the semantic segmentation model is U-Net or Segment Anything Model (SAM), and it has been fine-tuned using a paper fiber dataset containing three types of annotations: fiber, fine fiber, and miscellaneous cells. S2: Based on semantic segmentation mask, image morphology analysis method is used to calculate the structural parameters of the fiber. The structural parameters include number average length, length-weight average length, width, thickness, kink index, crimp index and fibrillation degree, and the parameters are serialized into standardized text descriptions. The average length of the quantity is expressed as:

[0022] The length-weight average length is expressed as:

[0023] in, Indicates the first The projected length of the root fiber, Total effective fiber count; S3: Input the original microscopic image, the generated semantic segmentation mask, and the standardized text description into the multimodal joint encoding module. The original microscopic image and the semantic segmentation mask are processed by the visual encoder to extract image texture features and topological structure features of the segmentation mask. Then, the standardized text description is transformed into a high-dimensional parameter embedding vector through the structural parameter embedding layer. The visual features and the high-dimensional parameter embedding vector are cross-attention fusion and semantic alignment are performed through the cross-modal projector to generate a unified-dimensional multimodal query vector. This vector retains texture, topology, and numerical semantic information. In this embodiment, the multimodal joint coding module adopts a phased, heterogeneous feature extraction and cross-modal alignment strategy to fuse the original microscopic image, semantic segmentation mask and structured parameters into a unified 768-dimensional query vector to support subsequent vector database retrieval and large language model inference. This module consists of three sub-components: a visual encoder, a structured parameter embedding layer and a cross-modal projector. In this embodiment, the visual encoder includes a first sub-encoder and a second sub-encoder, which process the original microscopic image and the semantic segmentation mask respectively to capture complementary texture and topological information. The first sub-encoder uses ViT-L / 14 as the backbone network to extract texture features from microscopic images. Specifically, the input image is first cropped from 2448×2048 pixels and scaled to 224×224 pixels using bilinear interpolation to match the pre-trained input size of the model. Then, it is normalized, the pixel values ​​are mapped to intervals, and the ImageNet mean is subtracted and divided by its standard deviation to generate a Float32 tensor. The ViT-L / 14 model contains 24 Transformer encoder layers with an output dimension of 768. The feature vectors corresponding to its tokens are used as global image representations. The model is fine-tuned end-to-end on a paper fiber-specific dataset containing 1250 expert-annotated microscopic images covering the morphological changes of six types of raw materials, including softwood pulp and hardwood pulp, under different beating degrees. Cross-entropy loss is used to supervise downstream classification tasks during fine-tuning. The second sub-encoder employs the SigLIP architecture to extract topological features from the semantic segmentation mask. Specifically, the input mask is an 8-bit indexed PNG image, which is first converted to one-hot encoding to generate a 4-channel (corresponding to four categories: long fibers, fine fibers, clutter cells, and background) Float32 tensor, with the size also adjusted to 224×224. The SigLIP backbone uses the ViT-B / 16 architecture, but in this embodiment, it is replaced with a variant of the same depth as ViT-L / 14 to maintain consistent feature dimensions, outputting a 768-dimensional vector. This model is fine-tuned on a paper fiber dataset containing 10,200 paired mask-text descriptions, with the optimization goal of enhancing sensitivity to topological properties such as fiber connectivity, breakpoints, and region proportions. During fine-tuning, the bottom 12 Transformer layers are frozen, and only the top 12 layers and the projection head are updated to preserve general visual priors and adapt to domain characteristics. The two sub-encoders of the visual encoder share the same output space dimension (768 dimensions) and are both deployed on edge computing servers with the TensorRT engine. The inference latency is controlled within 45 ms (ViT-L / 14) and 38 ms (SigLIP), respectively.

[0024] In this embodiment, the structural parameter embedding layer employs a fully connected neural network to embed the structured parameters after Z-Score normalization, outputting a 768-dimensional parameter embedding vector. Before embedding, all parameters are first Z-Score normalized to eliminate dimensional differences and improve model convergence stability. The normalized 12-dimensional vector is input into a two-layer fully connected multilayer perceptron (MLP) structure. The MLP structure is as follows: the first layer has 512 hidden units and uses GELU as the activation function; the second layer has an output dimension of 768 and no activation function (i.e., linear projection). The weights are initialized using a uniform Xavier distribution, and the bias term is initialized to zero. This MLP structure is jointly fine-tuned with the visual encoder during the system training phase, learning to map the normalized parameters to a vector space semantically aligned with the image texture feature vector and the topological feature vector of the segmentation mask, ultimately outputting the parameter embedding vector.

[0025] In this embodiment, the cross-modal projector receives image texture feature vectors, topological structure feature vectors of segmentation masks, and parameter embedding vectors from the structural parameter embedding layer from the visual coding submodule, and generates a unified query vector through a learnable fusion mechanism. The cross-modal projector adopts a Q-Former structure, which includes 12 Transformer encoder layers and learnable query vectors. It achieves deep fusion of visual features and parameter embedding vectors through a cross-attention mechanism, and projects the fusion result onto the input semantic space of a large language model. Specifically, the three input features are concatenated into context key-value pairs and mapped to the latent space of the Q-Former through a linear projection layer. The learnable query vectors serve as query vectors for cross-attention. Cross-attention is calculated in each Transformer layer, gradually aggregating texture, topology, and numerical semantic information. Finally, the 12 query vectors are averaged and then output as a single 768-dimensional vector. Alternatively, a lightweight 3-layer MLP can be used to replace Q-Former. The image texture feature vector from the visual encoding submodule, the topological structure feature vector from the segmentation mask, and the parameter embedding vector from the structural parameter embedding layer are concatenated along the channel dimension to form a 2304-dimensional vector. This vector is then passed sequentially through three fully connected layers (dimensions: 2304→1024→768→768), each followed by LayerNorm and GELU activation, with the last layer having no activation function. This approach is suitable for resource-constrained scenarios, improving inference speed by approximately 3 times, but slightly reducing semantic fusion capabilities.

[0026] Regardless of the fusion mechanism used, the output vector resides in the same semantic space as the 768-dimensional entry vectors stored in the knowledge base, and is compatible with the context embedding interface of subsequent large language models. In this embodiment, Q-Former is the default configuration, and its output Q is sent to the vector database server via the gRPC communication protocol to trigger the hybrid retrieval process.

[0027] S4: Using multimodal query vectors as retrieval conditions, perform hybrid retrieval in the pre-built paper fiber knowledge base. Hybrid retrieval includes metadata-based filtering retrieval and dense vector-based similarity matching, recalling the Top-K relevant knowledge entries to form a retrieval context. In this embodiment, the search scope is first narrowed by structured filtering based on the metadata of the current production scenario (such as raw material type, paper type, and process). Then, the cosine similarity matching algorithm is used to recall the Top-K most relevant knowledge entries in the dense vector index. In this embodiment, K in Top-K is 3 to 10, and only knowledge entries with a cosine similarity of not less than 0.7 with the multimodal query vector are retained as the search context.

[0028] In this embodiment, the hybrid retrieval strategy first performs an approximate nearest neighbor search based on the IVF_FLAT index to recall the Top-20 candidates; then it combines the keyword inverted index to perform secondary filtering on the metadata; finally, it removes redundancy through a re-ranking model and returns the Top-3 highly relevant knowledge entries. Each result includes text content, source type (standard / case / rule), and associated metadata.

[0029] In this embodiment, the paper fiber knowledge base is constructed using Milvus or Faiss, integrating dense vector retrieval and sparse keyword retrieval, covering industry standards, historical defect cases, and expert rules to form a high-confidence retrieval context. The paper fiber knowledge base contains unstructured text data and structured indicator data. The unstructured text data is transformed into a 768-dimensional dense vector using the BGE-M3 model and a dense index is constructed. The structured indicator data is indexed using the BM25 algorithm and additionally includes metadata fields for raw material type, paper type, and production process for filtering retrieval. In this embodiment, the paper fiber knowledge base consists of three types of heterogeneous knowledge sources: standard documents, historical defect cases, and expert rule base. To support subsequent multimodal vectorization and efficient retrieval, all knowledge sources have undergone unified structural processing to form a set of knowledge entries with standardized fields and metadata tags. The standard documents primarily originate from Chinese national standards (GB / T series) and the TAPPI (Technical Association of Pulp and Paper Industries) standards. The data was collected by downloading the original PDF documents, extracting the text content using existing OCR recognition tools, and matching key clauses and paragraphs using regular expressions. During the cleaning process, charts, headers, footers, and non-technical descriptions were removed, retaining only numerical thresholds, detection methods, and judgment rules that provide process guidance. Each standard segment was encapsulated as a structured entry. Historical defect cases are derived from the paper mill's production quality traceability system over the past five years. Each record includes: (1) a microscopic image of the time when the abnormality occurred; (2) the process log text; (3) the diagnosis conclusion and handling measures. The cleaning process includes: image denoising and size normalization; text desensitization and removal of colloquial expressions; and automatic labeling through the named entity recognition model. The expert rule base is stored in the enterprise knowledge management system, covering typical problem-solving strategies for core processes such as pulping, pulp preparation, and molding.

[0030] All three types of knowledge entries are stored uniformly in the central knowledge repository. Each entry is required to include metadata fields: raw material type, process, and exception tag. This structured design provides basic support for subsequent metadata-based filtering and retrieval. To achieve cross-modal semantic alignment, this embodiment employs a unified 768-dimensional vector space for embedding representation of three types of knowledge. Specifically, for text-based knowledge (including standard clauses, case summaries, and rule descriptions), the BGE-M3 multilingual embedding model is used for vectorization. Based on the original version, the BGE-M3 model undergoes further pre-training using a corpus from the papermaking domain to improve its domain terminology understanding. Before text input, it is segmented according to semantic integrity, with the segment length strictly controlled between 250 and 350 Chinese characters to ensure information integrity and avoid truncation. Each text segment is concatenated with the metadata of its corresponding entry to form a prompt prefix, in the format: The text content is then input into the BGE-M3 encoder, which outputs a 768-dimensional dense vector. For abnormal microscopic images from historical defect cases, the SigLIP visual encoder is used to extract features. The SigLIP model backbone is a ViT-B / 16 architecture, fine-tuned on a collected dataset of paper fiber images. The optimization goal is to enhance the ability to distinguish abnormal patterns such as fiber breakage, brooming, and impurities. The input image is centered and cropped to 512×512 pixels, normalized to the specified interval, and then fed into the model, outputting a 768-dimensional image feature vector. This image vector is in the same semantic space as the aforementioned text vector. Because SigLIP and BGE-M3 share the same contrastive learning objective function and are fine-tuned using cross-modal alignment loss during joint training, to ensure comparability of different modal vectors during retrieval, the system performs L2 norm normalization during the knowledge entry stage: calculating the L2 norm and normalizing it to a unit vector. This operation makes cosine similarity calculation equivalent to vector dot product, improving retrieval efficiency and accuracy.

[0031] In this embodiment, the vector database server constructs a hybrid index structure to support efficient and accurate knowledge retrieval. Specifically, under the Milvus 2.3 framework, an IVF_FLAT index is built for all normalized 768-dimensional dense vectors, where the number of cluster centers nlist is set to 1024. At the same time, a sparse inverted index is built for the text content and metadata fields of each knowledge entry, and the BM25 algorithm is used to calculate the relevance score. A three-level hybrid search process is executed during the actual query process: Level 1: Metadata Filtering. When a query request is received from the edge server, the accompanying metadata conditions are first parsed, such as "raw material type = softwood pulp AND process = pulping". The inverted index is then used to quickly filter out a subset of candidate entries that meet the conditions.

[0032] Level 2: Dense Vector Top-K Recall. The normalized multimodal query vector is compared with the candidate vector set after Level 1 screening using cosine similarity calculation. The IVF_FLAT index is used to accelerate the approximate nearest neighbor search and recall Top-K entries, where K is dynamically adjusted according to the application scenario, with a value ranging from 3 to 10.

[0033] Level 3: Redundancy Filtering and Re-ranking. For the recalled Top-K results, the cosine similarity between each pair of entries is calculated. If the similarity between any two entries is ≥0.7, it is considered semantic redundancy, and the higher-scoring entry is retained. Furthermore, a weighted fusion of BM25 keyword scores and dense vector scores is performed, resulting in the final scoring formula:

[0034] in For density similarity, This is the raw score for BM25. This is the weighting coefficient, with a value of 0.7. This process returns the top-3 highly relevant and low-redundancy knowledge entries, which serve as the context for the large language model's report generation.

[0035] S5: The standardized text description, the key visual feature description of the original microscopic image, and the retrieval context are concatenated according to the preset prompt word template, and input into the large language model decoder that has been fine-tuned by instructions in the papermaking field to generate a structured natural language report that includes fiber morphology evaluation, quality anomaly diagnosis, cause analysis, and process adjustment suggestions.

[0036] In this embodiment, the large language model is Llama3 70B or ChatGLM 4, and it is fine-tuned on a papermaking instruction dataset containing no less than 5,000 "fiber image-structured parameter-expert report" triples using LoRA low-rank adaptation technology. The rank of the LoRA adapter is 8.

[0037] In this embodiment, according to the preset prompt word template, the original image description, structured parameter text and retrieval context are integrated into a standardized prompt, which is input into the large language model decoder finely tuned by instructions in the papermaking field to generate a natural language report containing three parts: (1) quantitative evaluation of fiber morphology (whether it meets the standard threshold); (2) accurate diagnosis of quality anomalies (combining visual features and parameter anomalies); (3) targeted process adjustment suggestions (such as adjusting pulping concentration, grinding gap and other specific operations). The report has operability for direct production guidance.

[0038] Specifically, the following content is integrated into a prompt word template: the structured parameters of the current sample; the three contexts returned by the retrieval; and the preset instruction: "You are a papermaking process expert. Based on the above data, please generate a structured report containing three parts: [Quality Analysis Conclusion], [Potential Causes], and [Process Recommendations]." This prompt word is input into a large language model fine-tuned with instructions from the papermaking field, and the output format is Markdown-compatible text, which is pushed to the production monitoring center and the corresponding workstation terminals via the MQTT protocol.

[0039] In this embodiment, the large language model used in the report generation module is customized and trained based on a fine-tuning strategy for instructions in the papermaking field to ensure its professionalism, accuracy, and operability in fiber morphology analysis scenarios. The model is deployed on an NVIDIA A100 GPU on an edge computing server and runs in GGUF quantization format, balancing inference speed and generation quality.

[0040] The following details its implementation from three aspects: dataset construction, low-rank adaptation training, and prompt word engineering. First, the domain instruction fine-tuning dataset is constructed. The domain instruction fine-tuning dataset consists of 5,217 sets of "image-JSON parameter-expert report" triples, covering three major raw material types: softwood pulp, hardwood pulp and straw pulp, and covering seven typical abnormal states, including excessive fiber cutting, insufficient buffing, uneven fiber splitting, impurity contamination, missing curl, excessive kinking and normal samples. Each set of triples is strictly aligned with the multimodal information at the same microscopic image acquisition time. Specifically, the image part is a 2448×2048 pixel 16-bit grayscale TIFF image acquired and preprocessed by an industrial camera; the JSON parameter part is output by the parameter extraction module, containing 10 standardized fields, namely: raw material type, quantity average length, length-weight average length, fine fiber content, width, kink index, crimp index, fibrillation degree, beating degree, filtration time and process stage. All numerical fields retain three significant digits. The expert reports are written in accordance with TAPPI standards and the fiber standards of the papermaking process in the factory. Each report strictly follows a three-part structural template: Quality analysis conclusion: Briefly describe whether the current fiber morphology meets the process requirements; Potential causes: Inferring possible equipment or operational factors by combining parameter deviations with image features; Process recommendations: Propose 1–4 specific and actionable adjustments, such as adjusting the grinding disc gap from 0.1 mm to 0.25 mm or increasing the feed concentration to 39%. All triplets, after cross-validation and consistency checks, are divided into training, validation, and test sets in an 8:1:1 ratio for subsequent model fine-tuning and evaluation.

[0041] The second part is the low-rank adaptation training strategy. This part focuses on efficiently adapting a general-purpose large language model to the papermaking field under limited computing power. In this embodiment, low-rank adaptive LoRA technology is used to efficiently fine-tune the parameters of the base model. Specifically, the LoRA adapter is inserted after the query and key projection matrices of all attention layers in the base model. Let the original weight matrix be... LoRA represents its update as ,in , The model is a trainable low-rank matrix with a fixed rank r of 8. All original parameters of the backbone model are frozen, and only A and B are trained, thereby reducing the number of trainable parameters to less than 0.3% of the original model. During fine-tuning, the input sequence is dynamically generated by standardized prompt word templates, and the target output is the full text of the corresponding expert report.

[0042] Finally, the prompt word engineering and report generation section is completed. This part uses a structured template design to ensure that the large language model fully integrates multi-source contextual information during the generation process. The final prompt word input into the large language model is composed of five parts concatenated in a fixed order: The first part is the task instruction: "You are a papermaking process expert. Based on the following data, please generate a structured report containing three parts: [Quality Analysis Conclusion], [Potential Causes], and [Process Recommendations]". The second part is the raw material and process identification: "The current sample raw material type is {raw material type}, and it is in the {process stage} process"; The third part is a summary of the structured parameters: "Key morphological parameters are as follows: Quantity, Average Length = mm, fine fiber content = %””; The fourth part is the visual feature description (optional): "Microscopic images show that the fibers exhibit the features of {visual description keywords}", where {visual description keywords} are automatically extracted from the semantic segmentation results, such as a large number of short and broken fibers, sparse broom-like ends, and dense mixed cells, etc., and at most two of the most representative phrases can be selected.

[0043] The fifth part is the search context: "Refer to the following three pieces of information: summary of item 1, summary of item 2, and summary of item 3"; The placeholders in the above fields are dynamically filled by the preceding modules to ensure that each inference input is unique and the information is complete. The generation stage adopts a greedy decoding strategy, that is, the token with the highest probability is selected at each step, and sampling or beam search is not enabled to ensure the stability and repeatability of the output.

[0044] The final output strictly follows a three-part format, with paragraph titles marked in Chinese square brackets and content using concise and unambiguous technical language. After the generated results are formatted and validated, they are pushed to the production monitoring center and corresponding workstation terminals via the MQTT protocol for operators to view and execute in real time.

[0045] Example 2 This embodiment provides a fiber visual language-assisted quality analysis system for the papermaking process, used to implement the fiber visual language-assisted quality analysis method for the papermaking process in Embodiment 1, including: The data acquisition module is used to acquire microscopic images of pulp fibers; The data preprocessing module is used to input microscopic images into the semantic segmentation model to generate a semantic segmentation mask, calculate the structural parameters of fibers based on image morphology analysis methods, and convert them into standardized text descriptions. The knowledge base building module is used to store vectorized industry standards, historical defect cases, and expert experience rules, and supports hybrid retrieval based on metadata filtering and dense vector similarity; In this embodiment, the knowledge base construction module uses Milvus or Faiss vector databases to store dense vectors and metadata, supporting simultaneous execution of Boolean filtering based on metadata and approximate nearest neighbor retrieval based on cosine similarity.

[0046] The multimodal joint coding module is used to map microscopic images, semantic segmentation masks, and structured parameters into multimodal query vectors of a unified dimension; In this embodiment, the multimodal joint coding module includes: The visual encoder uses ViT-L / 14 and SigLIP to extract texture features from microscopic images and topological features from segmentation masks, respectively. The structural parameter embedding layer uses a fully connected network to transform the Z-Score-normalized structural parameters into 768-dimensional embedding vectors; The cross-modal projector adopts a Q-Former structure, which achieves multimodal feature fusion through a 12-layer Transformer encoder and learnable query vectors, and outputs a 768-dimensional visual language feature vector. The retrieval-enhanced reasoning module is used to retrieve the Top-K relevant knowledge entries from the paper fiber knowledge base using multimodal query vectors as query conditions to form a retrieval context; The quality analysis generation module integrates raw data descriptions, structured parameters, and retrieval context into standardized prompts, inputs them into a large language model fine-tuned by instructions from the papermaking industry, and generates a natural language report that includes morphological evaluation, anomaly diagnosis, and process recommendations.

[0047] In this embodiment, the large language model used by the quality analysis generation module is fine-tuned using LoRA technology, the weights of the basic model are frozen, and only a low-rank adaptation matrix with a rank of 8 is trained to reduce computational overhead and improve domain adaptability.

[0048] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A fiber visual language-assisted quality analysis method for papermaking process, characterized in that, Includes the following steps: Microscopic images of pulp samples collected during the papermaking process; The microscopic images are used to generate semantic segmentation masks through a semantic segmentation model. The structural parameters of the fibers are calculated based on image morphology analysis methods and then converted into standardized text descriptions. The visual encoder extracts the texture features of the microscopic image and the topological features of the semantic segmentation mask. The structural parameter embedding layer transforms the standardized text description into a high-dimensional parameter embedding vector. The image texture features, the topological features of the segmentation mask, and the high-dimensional parameter embedding vector are fused by cross-attention and semantic alignment through a cross-modal projector to generate a multimodal query vector. Using multimodal query vectors as retrieval criteria, a hybrid retrieval is performed in a pre-built paper fiber knowledge base to recall the Top-K relevant knowledge entries to form a retrieval context; Standardized text descriptions, image texture features, topological features of segmentation masks, and retrieval context are concatenated according to preset prompt word templates, input into a large language model decoder, and quality analysis results are generated.

2. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 1, characterized in that, The visual encoder includes a first sub-encoder and a second sub-encoder; The first sub-encoder uses ViT-L / 14 as the backbone network to extract the texture features of the microscopic image, performs pixel cropping and bilinear interpolation scaling on the microscopic image, performs normalization processing, subtracts the ImageNet mean and divides it by its standard deviation to generate a Float32 tensor. The second sub-encoder adopts the SigLIP architecture to extract the topological features of the semantic segmentation mask, convert the mask into a one-hot encoded form, and generate a multi-channel Float32 tensor.

3. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 1, characterized in that, The structural parameter embedding layer uses a fully connected neural network to embed the normalized structural parameters and generate a multi-dimensional parameter embedding vector.

4. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 1, characterized in that, The cross-modal projector adopts a Q-Former structure, which performs cross-attention fusion on image texture features, topological features of segmentation masks, and high-dimensional parameter embedding vectors, and projects the fusion result onto the input semantic space of the large language model.

5. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 4, characterized in that, The cross-modal projector concatenates image texture features, topological features of the segmentation mask, and high-dimensional parameter embedding vectors into context key-value pairs, and maps them to the latent space of the Q-Former through a linear projection layer. It can learn query vectors as query vectors for cross-attention, calculate cross-attention in each Transformer layer, and gradually aggregate texture, topology, and numerical semantic information. Finally, the query vector is output after average pooling.

6. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 1, characterized in that, The cross-modal projector employs a multilayer perceptron, which concatenates image texture features, topological features of the segmentation mask, and high-dimensional parameter embedding vectors along the channel dimension, and sequentially passes them through multiple fully connected layers. Each layer is followed by LayerNorm and GELU activation, with the last layer having no activation function.

7. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 1, characterized in that, Perform a hybrid search within a pre-built knowledge base of papermaking fibers, retrieving the Top-K relevant knowledge entries to form a search context, specifically including: Structured filtering is performed based on the metadata of the current production scenario to select a subset of candidate entries that meet the criteria. The cosine similarity matching algorithm is used to recall the Top-K most relevant knowledge entries in the dense vector index as the retrieval context.

8. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 7, characterized in that, The cosine similarity matching algorithm is used to recall the Top-K most relevant knowledge entries from the dense vector index as the retrieval context, specifically including: The normalized multimodal query vector and the subset of candidate entries are used to calculate cosine similarity. The IVF_FLAT index is used to accelerate the approximate nearest neighbor search and recall the Top-K entries. For the recalled Top-K items, calculate the cosine similarity between each pair. If the similarity between any two items is greater than a set threshold, it is considered semantic redundancy. The item with the higher score is retained, and the corresponding knowledge item is output as the retrieval context.

9. The fiber visual language-assisted quality analysis method for the papermaking process according to claim 1, characterized in that, The paper fiber knowledge base includes unstructured text data and structured indicator data; The unstructured text data is transformed into a multidimensional dense vector using the BGE-M3 model and a dense index is constructed. The structured index data is used to construct a keyword index using the BM25 algorithm, and metadata fields such as raw material type, paper type, and production process are added for filtering and retrieval.

10. A fiber visual language-assisted quality analysis system for papermaking processes, characterized in that, The method for implementing the fiber visual language-assisted quality analysis method for the papermaking process according to any one of claims 1-9 includes: a data acquisition module, a data preprocessing module, a knowledge base construction module, a multimodal joint coding module, a retrieval-enhanced reasoning module, and a quality analysis generation module; The data acquisition module is used to collect microscopic images of pulp samples during the papermaking process; The data preprocessing module is used to input the microscopic image into the semantic segmentation model to generate a semantic segmentation mask, calculate the structural parameters of the fiber based on the image morphology analysis method, and convert it into a standardized text description. The knowledge base construction module is used to construct a paper fiber knowledge base; The multimodal joint coding module is used to extract image texture features and topological structure features of the segmentation mask from the microscopic image and semantic segmentation mask through a visual encoder. The standardized text description is transformed into a high-dimensional parameter embedding vector through a structural parameter embedding layer. The image texture features, topological structure features of the segmentation mask and high-dimensional parameter embedding vector are cross-attention fusion and semantic alignment through a cross-modal projector to generate a multimodal query vector. The retrieval enhancement reasoning module is used to perform hybrid retrieval in a pre-built paper fiber knowledge base using multimodal query vectors as retrieval conditions, and recall the Top-K relevant knowledge entries to form a retrieval context; The quality analysis generation module is used to concatenate standardized text descriptions, image texture features, topological features of segmentation masks, and retrieval context according to preset prompt word templates, input them into the large language model decoder, and generate quality analysis results.

Images