Professional knowledge base retrieval optimization method based on AI deep semantic matching

Abstract

The invention is suitable for the technical field of artificial intelligence, and relates to a professional knowledge base retrieval optimization method based on AI deep semantic matching, which comprises the following steps: S10, a system collects engineering project specifications and standards or company core management files, and forms a standardized knowledge base system through classification and arrangement; s20, vectorization processing is performed based on the personal knowledge base, a professional vector database is constructed, and fine tuning training is performed through specific domain knowledge on the basis of the pre-trained large language model; s30, performing similarity retrieval based on a vector database, searching for a vector most similar to the target vector, and extracting key information; and S40, through systematic collection, classifying and sorting the similar problems, summarizing and analyzing the retrieved solutions and previous recorded results, and optimizing and adjusting the solutions. According to the method, the problem that the collected information related to the problem has defects or is incomplete before deep semantic matching is solved, and the accuracy of a deep semantic matching result is ensured.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a professional knowledge base retrieval optimization method based on AI deep semantic matching. Background Technology

[0002] Deep semantic matching refers to the process of using deep learning technology to understand and match deep semantic relationships between text, images, or other data. It is an activity that achieves more accurate content relevance judgment by capturing information such as context, intent, and implicit semantics. In other words, the professional knowledge base is the foundation of deep semantic matching, while deep semantic matching is the prerequisite for supporting knowledge retrieval. Before the deep semantic matching process of knowledge retrieval, information related to the question is collected, and then deep semantic matching is performed on the collected relevant information. Finally, the output is based on the results of deep semantic matching in order to answer the corresponding question.

[0003] Existing technologies often suffer from deficiencies or incompleteness in the information collected before deep semantic matching, leading to errors or omissions during the process and negatively impacting the accuracy of the results. Patent CN111291069B provides a data processing method, apparatus, and electronic device. The method includes: receiving search information and extracting keywords; retrieving multiple search results from a database based on the keywords, wherein the database is established for a specific domain; calculating the text similarity between each search result and the search information; filtering the search results based on the text similarity; sorting the filtered search results using a deep semantic matching model; selecting a target search result based on the sorting result and returning the target search result; thereby providing users with accurate search results. However, this patent only filters search results through deep semantic matching and does not address the problem of incomplete collection of question-related information before deep semantic matching.

[0004] Therefore, how to overcome the defects or incompleteness of the information collected that is relevant to the problem before conducting deep semantic matching is a problem that urgently needs to be solved by people in this technical field. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a professional knowledge base retrieval optimization method based on AI deep semantic matching, in order to solve the problem that the information collected before deep semantic matching is defective or incomplete, resulting in low accuracy of deep semantic matching results.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] This invention provides a method for optimizing the retrieval of professional knowledge bases based on AI deep semantic matching, comprising the following steps:

[0008] S10. Collect engineering project specifications and standards or core management documents of the company, and form a standardized knowledge base system through classification and organization;

[0009] S20. Based on personal knowledge base, vectorization processing is performed and a professional vector database is constructed. On the basis of pre-trained large language model, fine-tuning training is carried out through specific domain knowledge.

[0010] S30. Based on the vector database, perform similarity retrieval to find the vector most similar to the target vector and extract key information;

[0011] S40. By systematically collecting and classifying similar problems, summarizing and analyzing the retrieved solutions and previously recorded results, the solutions are optimized and adjusted.

[0012] Furthermore, the specific steps of step S10 are as follows:

[0013] S101. Collect engineering project specifications and standards or the company's core management documents, including bidding documents, contract terms, project construction standards and quotas;

[0014] S102. Set up application scenarios for engineering project specifications and standards, and match the engineering project specifications and standards according to the application scenarios;

[0015] S103. Screen the matched engineering project specifications and standards, determine the engineering project specifications and standards that conform to the application scenario of the engineering project, and integrate the engineering project specifications and standards that conform to the application scenario of the engineering project into a personal knowledge base.

[0016] Furthermore, the vectorization processing and construction of the professional vector database in step S20 specifically includes:

[0017] Word segmentation is performed using Python for Chinese word segmentation. The default parameters for word segmentation are chunk_size = 512 and chunk_overlap = 20 for repeated content in each segment. For word segmentation of large files, a parent-child segmentation method is adopted, in which the child text is segmented in the normal mode and the content of the parent text is retained.

[0018] Vector transformation is performed using the encoding method of the BGE-M3 vector model, generating both dense and sparse vectors during the transformation.

[0019] The dense vector representation, sparse vector representation, and corresponding original text are associated and stored to construct a searchable multimodal vector database.

[0020] Furthermore, the input text is subjected to multi-level semantic segmentation through a recursive segmentation algorithm to generate a hierarchical text structure containing parent-child relationships. In this structure, child text paragraphs are overlapped within a set length range, while parent text retains complete contextual information.

[0021] A multi-encoding fusion model is used to vectorize text paragraphs, simultaneously generating dense vectors with dimensionality compression and sparse vectors with explicit feature representation. The dense vectors are used for semantic similarity calculation, and the sparse vectors are used for keyword matching and retrieval.

[0022] Construct a storage structure with a triple mapping relationship: a joint index of dense and sparse vectors; a bidirectional pointer association between vector data and original text paragraphs; and a hierarchical mapping between child paragraphs and parent context.

[0023] Furthermore, the specific steps of step S30 are as follows:

[0024] S301. Generate query vectors, receive user query text, and encode the text in parallel into dense vectors and sparse vectors using a pre-trained BGE-M3 model.

[0025] S302, Vector retrieval: Input the dense vector and sparse vector into the vector database, call the hybrid_search interface to perform hybrid similarity retrieval, and obtain a retrieval result set containing subtext content, similarity distance value distance and hierarchical index identifier;

[0026] S303. Based on the hierarchical index identifier, extract the parent text content corresponding to the child text from the preset parent-child related text library;

[0027] S304. Input the user's question, the retrieved subtext and parent text content into the AI ​​generation model to generate a natural language answer that combines hierarchical context.

[0028] Furthermore, in step S302, the comprehensive similarity score is calculated using the following formula:

[0029]

[0030] Where α and β are dynamic weighting coefficients, (α + β = 1), q dense d dense The dense vectors f represent the query and the document respectively. t，d Let k be the term frequency of term t in document d, and k1,b be the adjustment parameters of the BM25 algorithm. This is the formula for calculating inverse document frequency.

[0031] Furthermore, the hierarchical index identifier includes the parent text UUID and the paragraph position offset, and the parent text is located in the following way:

[0032] Based on the UUID, match the parent text file; based on the offset, extract the complete paragraph containing the associated child text. The expression is: Where λ is the preset paragraph length threshold and δ is the context window compensation coefficient.

[0033] Furthermore, in step S40, LoRa is used to fine-tune the model in order to optimize and adjust the solution.

[0034] Furthermore, the specific steps for fine-tuning the model are as follows:

[0035] Dataset and Data Preprocessing: The pandas library's read_json function is used to read data from a specified JSON file, resulting in a DataFrame object. The DataFrame is then converted into a Hugging Face Dataset object. The map function is used to apply the preprocessing function process_data to each sample in the Dataset object to process the data. The remove_columns parameter removes all columns from the original dataset, retaining only the processed data. A DataCollatorForSeq2Seq object is created to batch the data during model training.

[0036] Configure parameters, including model path parameters, dataset path, training hyperparameters, LoRA-specific parameters, distributed training parameters, and hardware-related parameters.

[0037] Furthermore, the training parameters include the training batch size on each device, the gradient accumulation steps, the learning rate, the number of training logs recorded every few steps, the number of warm-up steps in the learning rate scheduler, the rank of the matrix, the scaling factor of LoRA, and the dropout rate in the LoRA matrix.

[0038] The professional knowledge base retrieval optimization method based on AI deep semantic matching provided by this invention has at least the following advantages compared with the prior art:

[0039] Existing technologies often suffer from deficiencies or incompleteness in the information collected before deep semantic matching, leading to errors or omissions during the process and negatively impacting the accuracy of the results. This invention addresses this by collecting relevant standards and core company management documents to create a personal knowledge base for knowledge retrieval. Extensive training of a large model with basic neurons using engineering project expertise improves the quality and reliability of answers, preventing errors or delays in deep semantic matching due to knowledge base errors or anomalies, thus ensuring accuracy and timeliness. Rapid similarity retrieval based on a vector database identifies the most similar vector to the target vector and extracts key information, avoiding inaccurate or incomplete answers due to mismatched information. Furthermore, this invention systematically collects, categorizes, and organizes solutions to similar problems, summarizing and analyzing retrieved solutions against previously recorded results, continuously optimizing and adjusting solutions. This maximizes accuracy while maintaining efficiency, saving resources and costs, thereby achieving optimized and managed professional knowledge base retrieval based on AI deep semantic matching. Attached Figure Description

[0040] To more clearly illustrate the solution of the present invention, a brief introduction will be given to the drawings used in the description of the embodiments below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This invention provides a method for optimizing the retrieval of a professional knowledge base based on AI deep semantic matching. Detailed Implementation

[0042] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0044] This invention provides a professional knowledge base retrieval optimization method based on AI deep semantic matching, applied to solve the problem of how to quickly obtain standard answers when encountering difficulties in the progress of existing projects. The professional knowledge base retrieval optimization method based on AI deep semantic matching includes the following steps:

[0045] S10. Collect engineering project specifications and standards or core management documents of the company, and form a standardized knowledge base system through classification and organization;

[0046] S20. Based on personal knowledge base, vectorization processing is performed and a professional vector database is constructed. On the basis of pre-trained large language model, fine-tuning training is carried out through specific domain knowledge.

[0047] S30. Based on the vector database, perform similarity retrieval to find the vector most similar to the target vector and extract key information;

[0048] S40. By systematically collecting and classifying similar problems, summarizing and analyzing the retrieved solutions and previously recorded results, the solutions are optimized and adjusted.

[0049] This invention overcomes the problem that the information collected before deep semantic matching is defective or incomplete, thus ensuring the accuracy of the deep semantic matching results.

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0051] This invention provides a professional knowledge base retrieval optimization method based on AI deep semantic matching, applied to solve the problem of how to quickly obtain standard answers when encountering difficulties in the progress of existing projects, such as... Figure 1 As shown, in this embodiment, the professional knowledge base retrieval optimization method based on AI deep semantic matching includes the following steps:

[0052] S10. The system collects engineering project specifications and standards or core management documents of the company, and forms a standardized knowledge base system through classification and organization.

[0053] Specifically, in this embodiment, step S10 involves the following steps:

[0054] S101. Collect engineering project specifications and standards or core management documents from national and local standards platforms or the company's internal knowledge base. For example, engineering project specifications and standards include bidding documents, contract terms, project construction standards and quotas, etc.

[0055] S102. Set up application scenarios for engineering project specifications and standards in the system. The application scenarios for engineering project specifications and standards refer to the scope of questions and answers that need to be retrieved. These scenarios are pre-defined based on the actual situation of the company and are usually classified according to their application scope. For example, engineering project specifications and standards are classified by professional field into building engineering, transportation engineering, water conservancy engineering, power engineering, municipal engineering, etc.

[0056] S103. Based on the application scenarios of engineering project specifications and standards, match and classify the engineering project specifications and standards, screen the matched engineering project specifications and standards, determine the engineering project specifications and standards that conform to the application scenarios of the engineering projects, and integrate the engineering project specifications and standards that conform to the application scenarios of the engineering projects into a personal knowledge base.

[0057] It is worth noting that deep semantic matching refers to the process of understanding and matching deep semantic relationships between text, images, or other data using deep learning technology during relevant knowledge retrieval. This process includes question parsing, retrieval, and matching answers to determine the correct answers, then summarizing and outputting the results. In relevant knowledge retrieval, professional knowledge retrieval typically involves efficiently and accurately extracting the required knowledge from massive amounts of professional data. By acquiring the required knowledge, answers to user questions can be determined, and then summarized, analyzed, and output. Its application areas include academic research, engineering, medicine, law, and finance.

[0058] S20. Based on a personal knowledge base, vectorization processing is performed and a professional vector database is constructed. On the basis of a pre-trained large language model, fine-tuning training is carried out through specific domain knowledge, thereby training the model's knowledge understanding and generation capabilities in the vertical domain.

[0059] Specifically, in this embodiment, the personal knowledge base is summarized and stored in the system to form a vector database that can be used for retrieval. The vector database includes the storage of high-dimensional vector data. In the vector database, storage specifically refers to the persistent preservation and management mechanism of high-dimensional vector data and related metadata.

[0060] Furthermore, in this embodiment, the vectorization processing and construction of the professional vector database in step S20 specifically includes word segmentation and vector conversion, wherein:

[0061] For word segmentation, the RecursiveCharacterTextSplitter method of Python's `chatchat` is used for Chinese word segmentation. The `chunk_size` value is set according to the actual situation. A reasonable value can improve retrieval efficiency and reduce storage and computing costs. The default parameters for each word segmentation are: `chunk_size = 512` and `chunk_overlap = 20` for each segment with repeated content. For word segmentation of large files, a parent-child segmentation method is adopted. The child text is segmented according to the normal mode, and the content of the parent text is retained.

[0062] Specifically, a recursive segmentation algorithm is used to perform multi-level semantic segmentation on the input text, generating a hierarchical text structure containing parent-child relationships. Among them, child text paragraphs are segmented with overlap within a set length range, while parent text retains complete contextual information. The recursive segmentation algorithm adopts the following: a preset segmentation length threshold of 512 character units; an overlap region of 20 character units between adjacent text paragraphs; and hierarchical semantic preservation of long texts is achieved through recursive iteration.

[0063] For example: Suppose there is a long text (e.g., a 2000-word Chinese article). Parent segmentation: Divide the entire article into larger blocks by paragraphs or chapters (e.g., each paragraph is about 1000 words); Child segmentation: Further recursively segment each parent block by chunk_size=512 and chunk_overlap=20: First child block: characters [0-511] (512 characters in total). Second child block: starting from character 512-20=492, covering [492-1003] (512 characters in total, where the first 20 characters overlap with the first block). Repeat until the end of the parent block; Preserve parent block: each child block is stored in association with the original content of the parent block to ensure context integrity.

[0064] Vector transformation is performed using the encoding method of the BGE-M3 vector model, generating both dense and sparse vectors during the transformation.

[0065] Specifically, a multi-encoding fusion model is used to vectorize text paragraphs, simultaneously generating dense vectors with dimensional compression and sparse vectors with explicit feature representation. The dense vectors are used for semantic similarity calculation, and the sparse vectors are used for keyword matching and retrieval. The multi-encoding fusion model adopts: a dual-channel encoding architecture based on an attention mechanism; generating high-dimensional semantic representations of more than 768 dimensions through the dense vector channel; and generating interpretable feature weight distributions through the sparse vector channel.

[0066] For example: Input text: Each sub-block (e.g., "Bionic architecture is a design concept that imitates the external form of organisms") is encoded. Dense vector: Generated through the multi-layer attention mechanism of the Transformer model, such as a 768-dimensional floating-point array: [0.23, -0.45, 1.2, ...] (dimension = 768); Sparse vector: Generated through algorithms such as word frequency or BM25, such as: {(1234:0.8, 5678:0.5, ...)} (index-value pairs with non-zero weights).

[0067] Dense vector dimensions: Higher dimensions (e.g., 1024 dimensions) indicate stronger representation capabilities, but higher storage and computation costs; lower dimensions (e.g., 256 dimensions) indicate higher efficiency, but may result in the loss of semantic details.

[0068] Sparse vector weights: High sparsity (few non-zero values) indicates fast retrieval speed, but may ignore low-frequency important words; low sparsity (more non-zero values) indicates that more information is retained, but the computational complexity increases.

[0069] Furthermore, in this embodiment, dense vector representations, sparse vector representations, and corresponding original texts are stored together to construct a searchable multimodal vector database.

[0070] Construct a storage structure with a triple mapping relationship: a joint index of dense and sparse vectors; a bidirectional pointer association between vector data and original text paragraphs; and a hierarchical mapping between child paragraphs and parent context.

[0071] The storage logic is to use dense vectors for similarity search and sparse vectors for fast keyword filtering.

[0072] S30. Based on a vector database, fast similarity retrieval finds the vector most similar to the target vector and extracts key information to avoid inaccurate or incomplete answers to questions due to mismatched information.

[0073] Specifically, in this embodiment, based on fast similarity retrieval of the vector database, the vector most similar to the target vector is found, and key information is extracted to avoid the problem of inaccurate or incomplete output answers due to mismatch of extracted information. The specific steps of step S30 are as follows:

[0074] S301. Generate query vectors, receive user query text, and encode the text in parallel into dense vectors and sparse vectors using a pre-trained BGE-M3 model.

[0075] S302. Vector retrieval: Input dense vectors and sparse vectors into the vector database, call the hybrid_search interface to perform hybrid similarity retrieval, and obtain a retrieval result set containing subtext content, similarity distance value, and hierarchical index identifier.

[0076] S303. Based on the hierarchical index identifier, extract the parent text content corresponding to the child text from the preset parent-child related text library.

[0077] S304. Input the user's question, the retrieved subtext and parent text content into the AI ​​generation model to generate a natural language answer that combines hierarchical context.

[0078] Specifically, in this embodiment, dense vectors and sparse vectors are input into a vector database, and the hybrid_search interface is called to perform hybrid similarity retrieval. The comprehensive similarity score is calculated using the following formula:

[0079]

[0080] Where α and β are dynamic weighting coefficients (α + β = 1), q dense d dense Let f represent the dense vectors of the query and the document, respectively. t，d Let be the term frequency of term t in document d, and k1 and b be the BM25 algorithm adjustment parameters (1.2≤k1≤2.0, 0.5≤b≤0.8). This is the formula for calculating inverse document frequency.

[0081] Furthermore, in this embodiment, the hierarchical index identifier includes the parent text UUID and the paragraph position offset. The parent text is located in the following way: the parent text file is matched according to the UUID, and the complete paragraph containing the associated sub-text is extracted based on the offset. The expression is as follows: Where λ is the preset paragraph length threshold (λ≥512 tokens) and δ is the context window compensation coefficient (-50≤δ≤50).

[0082] For example, a construction company encounters a technical problem during highway asphalt pavement construction. This invention allows for the rapid retrieval of relevant specifications and the generation of a solution. Details are as follows:

[0083] A user asked: "How can the temperature of the asphalt mixture be controlled and rainwater infiltration prevented during asphalt pavement paving in the rainy season?"

[0084] Dual-modal coding:

[0085] Input text: User question text (length L = 28 words);

[0086] BGE-M3 model output: Dense vector: q dense ∈R 768 (L2 norm = 14.3); Sparse vector: contains keywords such as "rainy season", "asphalt paving", "temperature control", and "rainwater infiltration", with 10 dimensions. 5 .

[0087] Hybrid search: Call the hybrid_search interface with the following parameters: α = 14.3 / 14.3 + 8.2 = 0.64, β = 1 - α = 0.36 (due to the high norm of dense vectors, semantic matching is emphasized);

[0088] Search formula: Score = 0.64·cos(q) dense ,d dense )+0.36·BM25(q sparse ,d sparse BM25 parameters: k1 = 1.8, b = 0.65 (optimized for use in engineering documents);

[0089] Search results: Subtext fragment (Top-1): "During the rainy season, the temperature of the mixture at the factory should be ≥160℃, and a waterproof tarpaulin should be covered immediately after paving", Distance: 0.78 Hierarchical index: (JTG_F40-2023, offset=2456);

[0090] Based on subtext offset s =2456, locate the parent text using the formula:

[0091]

[0092] Parent text content: Section 4.3 "Construction Requirements for Special Climates" of "Technical Specification for Construction of Asphalt Pavement of Highway JTG_F40-2023": 4.3.2 Construction during the rainy season (1) The vehicles transporting the mixture must be equipped with an insulation layer, and the temperature upon arrival must not be lower than 155℃; (2) A mobile rain shelter should be provided at the paving site, and compaction should be completed within 30 minutes after paving; (3) In case of moderate or above rain, construction should be stopped immediately and the paved sections should be waterproofed.

[0093] AI-generated answer: According to Article 4.3.2 of JTG_F40-2023, the following measures must be taken for asphalt paving during the rainy season:

[0094] Temperature control: The temperature of the mixture leaving the plant is ≥160℃, and the temperature upon arrival at the site is ≥155℃ (using insulated transport vehicles); the paver is preheated to above 130℃ to reduce temperature loss.

[0095] Rain protection measures: Cover with waterproof tarpaulin immediately after paving, and complete compaction within 30 minutes; provide mobile rain shelters on site, and monitor radar images for real-time rain protection; suspend construction during moderate to heavy rain, and cover paved sections with waterproof geotextile. Note: For detailed operation, please refer to Appendix B of this specification, "Emergency Plan for Rainy Season Construction".

[0096] S40. By systematically collecting, classifying and organizing solutions to similar problems, summarizing and analyzing the retrieved solutions with previously recorded results, and continuously optimizing and adjusting the solutions.

[0097] Specifically, in this embodiment, optimizing and adjusting the answers to questions requires fine-tuning the model. Model fine-tuning involves using the input question-and-answer dataset and calling the model fine-tuning method to generate a new model.

[0098] Specifically, model fine-tuning is achieved using LoRA (Low-Rank Adaptation), a fine-tuning technique for large language models designed to reduce computational and memory requirements during the fine-tuning process.

[0099] The fine-tuning process mainly includes two steps: dataset and data preprocessing, and parameter setting.

[0100] Specifically, the dataset and data preprocessing are performed. The `read_json` function from the pandas library is used to read data from a specified JSON file (given by the `args.train_file` parameter), where `lines=True` indicates that each line in the file is an independent JSON object, resulting in a DataFrame object. This DataFrame is then converted to a Hugging Face Dataset object to utilize the features provided by the datasets library for further data manipulation. The `map` function applies a preprocessing function `process_data` to each sample in the Dataset object. This function processes the data, such as using a tokenizer to encode the text and ensuring that the sequence length does not exceed `max_seq_length` (2048 in this case). Simultaneously, the `remove_columns` parameter removes all columns from the original dataset, retaining only the processed data. A `DataCollatorForSeq2Seq` object is created to organize the data batches during model training, including using a tokenizer to encode the text, padding to ensure consistent sequence lengths across batches, and specifying the returned tensor type as a PyTorch tensor (`return_tensors="pt"`). Thus, the final train_dataset is a preprocessed Dataset object ready for training, while data_collator is a tool used to organize data batches during training.

[0101] Specifically, set the parameters. These mainly include model path-related parameters, dataset path, training hyperparameters, LoRA-specific parameters, distributed training parameters, and hardware-related parameters. Training parameters include:

[0102] `per_device_train_batch_size`: The training batch size on each device. The batch size determines the amount of data fed to the model during each training session. Batch sizes that are too small may lead to unstable or inefficient training, while batch sizes that are too large will increase GPU memory usage and may cause an OutOfMemoryError (OOM).

[0103] `gradient_accumulation_steps`: The number of steps to accumulate gradients. `global batch = num_gpus * per_device_train_batch_size * gradient_accumulation_steps`. Setting this parameter too high can lead to excessive gradient accumulation, affecting the model's learning efficiency and stability. This is because gradients are accumulated over multiple mini-batches, rather than being updated once per mini-batch, which increases the variance of the gradient estimate and impacts the model's convergence performance. Conversely, setting this parameter too low can reduce the variance caused by gradient accumulation, but it also reduces the effective batch size, potentially lowering the model's training effectiveness. Larger batch training typically provides more stable gradient estimates.

[0104] `learning_rate`: The learning rate. A learning rate that is too high may cause gradient explosion, leading to numerical overflow and affecting model stability. A learning rate that is too low may cause the model to get stuck in a local optimum instead of the global optimum. Therefore, we usually need to tune the parameters to find a suitable learning rate.

[0105] `logging_steps`: How many steps should the training log be logged every? Don't set it too high, as SwanLab might interrupt the process due to prolonged logging failures.

[0106] warmup_steps: The number of warmup steps in the learning rate scheduler. Too few warmup steps may cause the model to be unstable at the beginning of training; too many may waste training time.

[0107] lora_rank: The rank of the LoRA matrix. A higher lora_rank results in more parameters needing to be trained, potentially improving the model's representational power, but also increasing training overhead. A lower lora_rank may reduce training costs, but may also limit the model's adaptability, leading to a decline in model performance.

[0108] `lora_alpha`: The scaling factor for LoRA. The rank of the LoRA matrix, `lora_rank`, is typically scaled by multiplying by an alpha factor. This parameter controls the influence of the low-rank matrix. A larger `lora_alpha` results in a greater influence from the LoRA matrix, potentially causing the model to rely more heavily on LoRA for adaptation, thus impacting performance. A smaller `lora_alpha` results in a smaller contribution from the LoRA matrix, leading to greater reliance on the original model parameters for prediction. Choosing an appropriate `lora_alpha` helps balance LoRA adaptability and training efficiency.

[0109] `lora_dropout`: The dropout rate in the LoRA matrix. A higher `lora_dropout` value increases the regularization effect and prevents overfitting of the LoRA matrix. A lower `lora_dropout` value may lead to overfitting of the LoRA matrix, especially with limited training data. An appropriate dropout value helps improve the model's generalization ability.

[0110] Compared with existing technologies, the AI-based deep semantic matching-based professional knowledge base retrieval optimization method described in the above embodiments has defects or incompleteness in the information related to the question collected before deep semantic matching, which leads to errors or omissions during deep semantic matching and adversely affects the accuracy of the subsequent deep semantic matching results, thus affecting the accuracy of the question answers. This invention, through the collection of relevant standards and core management documents of a company, can form a personal knowledge base for knowledge retrieval. By extensively training a large model with basic neurons using professional engineering project knowledge, it helps improve the quality and reliability of question answers, avoiding errors or delays in deep semantic matching caused by knowledge base errors or anomalies, thus ensuring the accuracy and timeliness of deep semantic matching. Based on a vector database, rapid similarity retrieval finds the vector most similar to the target vector and extracts key information, avoiding inaccurate or incomplete output answers due to mismatched extracted information. Furthermore, by systematically collecting and classifying solutions to similar questions, summarizing and analyzing the retrieved solutions with previously recorded results, and continuously optimizing and adjusting the solutions, this invention helps maximize the accuracy of answers while ensuring efficiency in answer output, saving resources and costs, thereby achieving optimized and managed professional knowledge base retrieval based on AI deep semantic matching.

[0111] Obviously, the embodiments described above are merely preferred embodiments of the present invention, and not all embodiments. The accompanying drawings illustrate preferred embodiments of the present invention, but do not limit the scope of the patent. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this invention.

Claims

1. An AI deep semantic matching-based professional knowledge base retrieval optimization method, characterized in that, The method comprises the following steps: S10, the system collects engineering project specification standards or company core management files, and forms a standardized knowledge base system through classification and arrangement; S20, vectorization processing is performed based on the personal knowledge base, and a professional vector database is constructed, pre-training of a large language model is performed, and fine-tuning training is performed through specific field knowledge; S30, similarity retrieval is performed based on the vector database, the most similar vector to the target vector is found, and key information is extracted; S40, the same type of problems is collected and arranged through systematization, the retrieved solutions are summarized and analyzed with the previous recorded results, and the solutions are optimized and adjusted. 2.The AI deep semantic matching-based professional knowledge base retrieval optimization method of claim 1, wherein, The specific steps of the step S10 are as follows: S101, collecting engineering project specification standards or company core management files, including bidding documents, contract terms, project construction standards and quotas; S102, setting an engineering project specification standard application scenario, and matching the engineering project specification standards according to the engineering project specification standard application scenario; S103, screening the matched engineering project specification standards, determining the engineering project specification standards that meet the engineering project application scenario, and integrating the engineering project specification standards that meet the engineering project application scenario into a personal knowledge base. 3.The AI deep semantic matching-based professional knowledge base retrieval optimization method of claim 1, wherein, The vectorization processing and construction of the professional vector database in the step S20 specifically include: Word segmentation, Chinese word segmentation is performed by using Python, the default parameters for word segmentation are segmentation size chunk_size=512 and repeated content chunk_overlap=20 for each segment, for large file word segmentation, a parent-child segmentation mode is adopted, the child text is segmented according to a normal mode, and the parent text content is retained; Vector conversion, vector conversion is performed through the encode method of the vector model BGE-M3, and dense vectors and sparse vectors are generated at the same time during conversion; The dense vector representation, the sparse vector representation and the corresponding original text are associatedly stored, and a retrievable multi-modal vector database is constructed.

4. The professional knowledge base retrieval optimization method based on AI deep semantic matching according to claim 3, characterized in that, The input text is subjected to multi-level semantic segmentation through a recursive segmentation algorithm, and a hierarchical text structure containing parent-child relationships is generated, wherein the child text paragraphs are subjected to overlapping segmentation within a set length range, and the parent text retains complete context information; A multi-encoding fusion model is used for vectorization processing of the text paragraphs, and a dimensionally compressed dense vector and a sparse vector that explicitly expresses features are synchronously generated, wherein the dense vector is used for semantic similarity calculation, and the sparse vector is used for keyword matching retrieval; A storage structure containing a triple mapping relationship is constructed: joint index of the dense vector and the sparse vector; bidirectional pointer association of the vector data and the original text paragraphs; hierarchical relationship mapping of the child paragraphs and the parent context.

5. The professional knowledge base retrieval optimization method based on AI deep semantic matching according to claim 1, characterized in that, The specific steps of the step S30 are as follows: S301, generating a query vector, receiving user query text, and parallelly encoding the text into a dense vector and a sparse vector through a pre-trained BGE-M3 model; S302, vector retrieval, input the dense vector and the sparse vector into the vector database, call the hybrid_search interface for hybrid similarity retrieval, and obtain a search result set containing subtext content, similarity distance value distance, and hierarchical index identification; S303, according to the hierarchical index identification, extract the parent text content corresponding to the subtext from the pre-set parent-child associated text library; S304, input the user question, the search subtext and the parent text content into the AI generation model to generate a natural language answer combining the hierarchical context.

6. The professional knowledge base retrieval optimization method based on AI deep semantic matching according to claim 5, characterized in that, In step S302, the comprehensive similarity score is calculated by the following formula: where a, b are dynamic weight coefficients (a + b = 1), q dense ,d dense denotes the dense vector of the query and document, respectively, f t，d is the term frequency of term t in document d, k1, b are BM25 algorithm tuning parameters, is the inverse document frequency formula.

7. The professional knowledge base retrieval optimization method based on AI deep semantic matching according to claim 6, characterized in that, The hierarchical index identification includes the parent text UUID and the paragraph position offset, and the parent text is located by the following method: According to the UUID, the parent text file is matched, and the complete paragraph containing the associated child text is extracted based on the offset, and the expression is as follows: Wherein, λ is a preset paragraph length threshold, and δ is a context window compensation coefficient. 8.The AI deep semantic matching-based professional knowledge base retrieval optimization method of claim 1, wherein, In step S40, the model fine-tuning is realized by LoRa to optimize and adjust the solution.

9. The professional knowledge base retrieval optimization method based on AI deep semantic matching according to claim 8, characterized in that, The specific steps of the model fine-tuning are as follows: Dataset and data preprocessing: read the data from the specified JSON file using the read_json function of the pandas library, get a DataFrame object after reading, convert the DataFrame to a Dataset object of Hugging Face, apply the preprocessing function process_data to each sample in the Dataset object through the map function to process the data, remove all columns in the original dataset by the remove_columns parameter, only keep the processed data, and create a DataCollatorForSeq2Seq object to arrange the data batches during model training; Set parameters, including model path related parameters, dataset path, training hyperparameters, LoRA specific parameters, distributed training parameters, and hardware related parameters.

10. The professional knowledge base retrieval optimization method based on AI deep semantic matching according to claim 9, characterized in that, The training parameters include the training batch size on each device, the gradient accumulation step number, the learning rate, the number of steps to record the training log every time, the warm-up step number in the learning rate scheduler, the rank of the matrix, the scaling factor of LoRA, and the dropout rate in the LoRA matrix.

Images