Large-model question answering method and apparatus for transportation industry, and device and storage medium

Abstract

A large-model question answering method and apparatus for the transportation industry, and a device and a storage medium, relating to the interdisciplinary field of artificial intelligence and information security. The method comprises: on the basis of a pre-constructed security knowledge base, retrieving a transportation-related question to be processed; when the obtained retrieval result indicates that there is no reply information corresponding to the transportation-related question in the knowledge base, determining, on the basis of a preset question security interception strategy, whether to currently trigger a question interception operation; when an obtained first determination result indicates that the question interception operation is not to be triggered, performing, on the basis of a preset prompt, question enhancement on the transportation-related question; on the basis of a pre-fine-tuned target large transportation model, processing the obtained enhanced question to obtain target reply information; and on the basis of a preset answer security interception strategy, determining whether to currently trigger a reply interception operation corresponding to the target reply information, so as to complete a question answering operation on the basis of an obtained second determination result. The security and accuracy of question answering can be effectively improved.

Description

Large model question and answer method, device and equipment in traffic industry and storage medium

[0001] The present application claims priority to the Chinese patent application No. 202411067085.5, filed on August 6, 2024, and entitled "Large model question and answer method, device and equipment in traffic industry and storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the cross field of artificial intelligence and information security, in particular to a large model question and answer method, device and equipment in traffic industry and storage medium. BACKGROUND

[0003] With the rapid development of artificial intelligence, generative large models are increasingly applied in various industries, especially in the field of intelligent transportation. However, since generative large models are trained based on deep neural networks, they are not interpretable and can easily produce hallucinations, which may generate some content with security risks and thus infringe on users.

[0004] Therefore, how to ensure that the traffic large model does not output content with security risks is a problem to be solved at present. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a large model question and answer method, device and equipment in traffic industry and storage medium, which can effectively prevent the input of malicious data and the output of content with security risks, improve the security, efficiency and accuracy of question and answer, and effectively prevent the abuse of models, thereby ensuring the user experience of using the traffic large model and providing a solid security guarantee for the construction and use of subsequent intelligent transportation systems. The specific scheme is as follows:

[0006] In a first aspect, the present application provides a large model question and answer method in traffic industry, comprising:

[0007] Obtaining a traffic-related question to be processed, and retrieving the traffic-related question based on a pre-constructed security knowledge base to obtain a corresponding retrieval result;

[0008] When the retrieval result indicates that there is no reply information corresponding to the traffic-related question in the security knowledge base, determining whether a question interception operation corresponding to the traffic-related question is triggered based on a pre-set question security interception strategy to obtain a corresponding first determination result;

[0009] When the first judgment result indicates that the problem interception operation is not triggered, the traffic-related question is prompted and enhanced based on a preset prompt to obtain a corresponding enhanced question;

[0010] The enhanced question is processed based on a target traffic large model that has been fine-tuned in advance to obtain corresponding target reply information;

[0011] According to a preset answer safety interception strategy, it is judged whether the reply interception operation corresponding to the target reply information is triggered or not, and the question and answer operation corresponding to the traffic-related question is completed based on the obtained second judgment result.

[0012] Optionally, the traffic-related question is searched based on a pre-constructed safety knowledge base to obtain a corresponding search result, including:

[0013] The traffic-related question is pre-processed, word segmented, and analyzed based on the pre-constructed safety knowledge base to complete corresponding syntax analysis operations and obtain an analysis result;

[0014] The analysis result is searched based on an index library in the safety knowledge base to obtain a corresponding search result.

[0015] Optionally, before the traffic-related question is searched based on the pre-constructed safety knowledge base to obtain a corresponding search result, it further includes:

[0016] A web page recording target traffic-related information is crawled based on python to obtain a corresponding web document;

[0017] The web document is parsed, de-duplicated, and data-formatted based on a preset document cleaning rule to obtain a corresponding cleaned document;

[0018] The cleaned document is detected for risk content and deleted for risk content using a preset safety detection model to complete a corresponding document verification operation and obtain a verified document;

[0019] The verified document is word segmented, stop words are removed, and an inverted index is constructed to obtain a corresponding search library and a safety knowledge base as a question search engine.

[0020] Optionally, the question interception operation corresponding to the traffic-related question is judged based on a preset question safety interception strategy to obtain a corresponding first judgment result, including:

[0021] Based on the aforementioned traffic-related issues, string pattern matching and embedding semantic similarity matching are performed on several terms in the first preset blacklist database to obtain the corresponding first matching results;

[0022] When the first matching result indicates that there is no term matching the traffic-related question in the first preset blacklist database, the traffic-related question is input into the target question security interception model pre-built based on a large model to obtain the corresponding first model output result;

[0023] When the first model output does not contain the corresponding first security risk information and first risk type information, a corresponding first judgment result is obtained to indicate that the problem interception operation is not triggered.

[0024] When the first model output contains the corresponding first security risk information and the first risk type information, the first judgment result indicating that the problem interception operation is triggered is obtained.

[0025] Optionally, before inputting the traffic-related problem into the target problem safety interception model pre-built based on a large model, the method further includes:

[0026] Construct corresponding sensitive words, and use the sensitive words to determine whether each historical issue in the current historical issue list has a security risk, so as to obtain the corresponding risk assessment results;

[0027] Based on the risk assessment results, the corresponding historical questions are labeled to obtain the corresponding first training set; wherein, the first training set includes corresponding positive samples and negative samples, the positive samples are the historical questions that have been labeled with risk types, and the negative samples are the historical questions that have been labeled with no risk types or have not hit sensitive words;

[0028] Based on the large model, a corresponding model architecture is designed to obtain the initial problem security interception model;

[0029] During the fine-tuning training of the initial problem security interception model using the first training set, a model evaluation operation is performed on the model after each round of training based on a preset risk interception rate formula until the preset training termination condition is met, thus obtaining the corresponding target problem security interception model.

[0030] Optionally, before processing the enhanced problem based on the pre-tuned target traffic model, the method further includes:

[0031] By creating a corresponding set of fine-tuned questions and a corresponding set of fine-tuned answers, a corresponding secure question-and-answer pair dataset is obtained;

[0032] A corresponding traffic professional corpus is collected, and the traffic professional corpus is detected based on a preset question safety interception strategy to determine a corresponding traffic professional question and answer pair data set according to a search result obtained;

[0033] A corresponding second training set is obtained based on the safety question and answer pair data set and the traffic professional question and answer pair data set; wherein a target safety question and answer pair data in the second training set and a target traffic professional question and answer pair data satisfy a corresponding data proportion condition;

[0034] In the process of training the constructed initial traffic large model based on the second training set, the learning rate is fine-tuned until a corresponding target traffic large model is obtained.

[0035] Optionally, the judgment of whether the reply interception operation corresponding to the target reply information is triggered according to the preset answer safety interception strategy is to complete the question and answer operation corresponding to the traffic-related question based on a second judgment result obtained, comprising:

[0036] Based on the target reply information, a string pattern matching and embedding semantic similarity matching are performed on a plurality of word entries in a second preset blacklist library to obtain a corresponding second matching result;

[0037] When the second matching result indicates that there is no word entry in the second preset blacklist library that matches the target reply information, the target reply information is input into a target reply safety interception model constructed based on a large model in advance to obtain a corresponding second model output result;

[0038] When the second model output result contains corresponding second safety risk information and second risk type information, the target reply information is intercepted based on the second safety risk information and the second risk type information;

[0039] When the second model output result does not contain the corresponding second safety risk information and the second risk type information, the target reply information is used to answer the question to complete the question and answer operation corresponding to the traffic-related question.

[0040] In a second aspect, the present application provides a large model question and answer device for the traffic industry, comprising:

[0041] A knowledge base retrieval module is configured to acquire a traffic-related question to be processed, and retrieve the traffic-related question based on a pre-constructed safety knowledge base to obtain a corresponding search result;

[0042] The problem interception judgment module is configured to, when the search result indicates that the reply information corresponding to the traffic-related problem does not exist in the security knowledge base, judge whether a problem interception operation corresponding to the traffic-related problem is triggered based on a preset problem security interception strategy, to obtain a corresponding first judgment result.

[0043] The problem enhancement module is configured to, when the first judgment result indicates that the problem interception operation is not triggered, perform prompt enhancement on the traffic-related problem based on a preset prompt, to obtain an enhanced problem.

[0044] The reply information acquisition module is configured to process the enhanced problem based on a target traffic large model that has been fine-tuned in advance, to obtain corresponding target reply information.

[0045] The reply interception judgment module is configured to judge whether a reply interception operation corresponding to the target reply information is triggered according to a preset answer security interception strategy, to complete a question and answer operation corresponding to the traffic-related problem based on a second judgment result obtained.

[0046] In a third aspect, the present application provides an electronic device, comprising:

[0047] A memory for saving a computer program;

[0048] A processor for executing the computer program to implement the steps of the large model question and answer method for the traffic industry.

[0049] In a fourth aspect, the present application provides a computer readable storage medium for saving a computer program, which is executed by a processor to implement the steps of the large model question and answer method for the traffic industry.

[0050] It can be seen that in the present application, a traffic-related problem to be processed is obtained, and the traffic-related problem is searched based on a pre-constructed safety knowledge base to obtain a corresponding search result. When the search result indicates that there is no reply information corresponding to the traffic-related problem in the safety knowledge base, it is determined whether to trigger a problem interception operation corresponding to the traffic-related problem based on a preset problem safety interception strategy to obtain a corresponding first determination result. When the first determination result indicates that the problem interception operation is not triggered, the traffic-related problem is enhanced based on a preset prompt to obtain an enhanced problem. The enhanced problem is processed based on a target traffic large model that has been pre-tuned to obtain corresponding target reply information. It is determined whether to trigger a reply interception operation corresponding to the target reply information based on a preset reply safety interception strategy to complete a question and answer operation corresponding to the traffic-related problem based on a second determination result obtained. That is, in the present application, it is determined whether to perform problem interception at the problem input level based on the safety knowledge base and the preset problem safety interception strategy. If not, the target reply information is obtained based on the pre-tuned target traffic large model after prompt. Then, it is determined whether to perform reply interception at the output level based on the preset reply safety interception strategy. In this way, on the one hand, the input of malicious data and the output of security risk content can be effectively prevented, and the safety, efficiency and accuracy of the question and answer can be improved. On the other hand, the abuse of the model can be effectively prevented, thereby ensuring the user experience of using the traffic large model and providing a solid security guarantee for the construction and use of the subsequent intelligent traffic system. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on the provided drawings.

[0052] FIG. 1 is a flowchart of a traffic industry large model question and answer method provided by the present application;

[0053] FIG. 2 is a schematic diagram of a specific technical architecture of a traffic industry large model question and answer provided by the present application;

[0054] FIG. 3 is a schematic diagram of a specific safety knowledge base framework process provided by the present application;

[0055] FIG. 4 is a schematic diagram of a traffic industry large model question and answer device structure provided by the present application;

[0056] FIG. 5 is a schematic diagram of an electronic device structure provided by the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0058] With the rapid development of artificial intelligence, generative large models are increasingly applied in various industries, especially in the field of intelligent transportation. However, since the generative large model is trained based on a deep neural network, it has non-interpretable characteristics, and the model is prone to hallucination, which may generate some content with security risks, thereby infringing on users. Therefore, the present application provides a large model question and answer scheme in the transportation industry, which can effectively prevent the input of malicious data and the output of content with security risks, improve the safety, efficiency and accuracy of the question and answer, and effectively prevent the abuse of the model, thereby ensuring the user experience of using the transportation large model and providing a solid security guarantee for the construction and use of subsequent intelligent transportation systems.

[0059] Referring to FIG. 1, the present application discloses a large model question and answer method in the transportation industry, comprising:

[0060] Step S11, obtaining a traffic-related question to be processed, and searching the traffic-related question based on a pre-constructed safety knowledge base to obtain a corresponding search result.

[0061] Specifically, in combination with FIG. 2, in the embodiment, the large model is prone to hallucination, but in the transportation industry, many problems are not allowed to be wrong, and the answers are relatively clear, which are usually solved by the safety knowledge base. At this time, the safety knowledge base is a simplified search engine, and the main content is as follows. The safety knowledge base is used to search the traffic-related problems to obtain the corresponding search results, including: the safety knowledge base is used to preprocess, segment and analyze the traffic-related problems, complete the corresponding syntax analysis operation, and obtain the analysis result; the index library in the safety knowledge base is used to search the analysis result to obtain the corresponding search result. In combination with FIGS. 2 and 3, in the embodiment, the safety knowledge base is used to understand (i.e. the syntax analysis operation) and search the traffic-related problems proposed by the user, if there is reply information corresponding to the traffic-related problems after searching, the relevance analysis is needed to sort the replies to determine the reply information with the highest relevance. The problem preprocessing includes but is not limited to case conversion, full and half angle conversion, length truncation; the problem segmentation can be segmented by using the relatively simple python jieba library; the problem analysis can be analyzed by using TF-IDF (Term Frequency-Inverse Document Frequency, i.e. Term Frequency-Inverse Document Frequency, which is a commonly used text feature representation method) (not limited to this, can be adjusted according to actual needs).

[0062] At the same time, it needs to be understood that, regarding the relevance analysis, the relevance measures the consistency of the search results provided by the safety knowledge base and the query intention of the user. The quality of the search results can be determined by the matching degree of the displayed reply information and the user's expectation. In the safety knowledge base scenario, the relevance algorithm used is TF-IDF, which gives high weight to high-frequency words in the text and low-frequency files in the entire file set. In TF-IDF, the importance of a word increases in direct proportion to the number of times it appears in a file, but at the same time, it decreases in inverse proportion to the frequency of its appearance in the corpus.

[0063] It is further understood that, in the embodiment, before the searching of the traffic-related problems based on the pre-constructed security knowledge base to obtain the corresponding search results, the construction of the security knowledge base further includes: crawling a webpage recording target traffic-related information based on python to obtain a corresponding webpage document; parsing, deduplicating, and data formatting the webpage document based on a preset document cleaning rule to obtain a corresponding cleaned document; performing risk content detection and risk content deletion on the cleaned document by using a preset security detection model to complete a corresponding document verification operation and obtain a verified document; and obtaining a corresponding search library and a security knowledge base as a problem search engine by performing word segmentation processing, stop word removal, and inverted index construction on the verified document. In combination with FIG. 2, specifically, first, security knowledge is crawled: the target of the crawler includes webpages on the Internet, and the main types include traffic road-related laws and regulations, related cases, and the like. Specifically, it can be implemented based on the asyncio module of python. Since asyncio can implement asynchronous IO parallelism, multiple webpages can be crawled at the same time, thereby effectively improving the efficiency of the crawler. Then, a series of document cleaning is performed: (1) HTML (Hyper Text Markup Language) webpage document parsing. Specifically, it can be implemented by using Beautiful Soup through a user-defined converter to implement the conventional document navigation, finding, and modifying the document, which can efficiently extract the document in the webpage and arrange it into a standard format; (2) document deduplication: first, the document is segmented according to semantics, then simHash is calculated for different document paragraphs, and finally, the similarity is calculated by using cos to compare the similarity of the documents, and the documents with a similarity higher than a certain threshold (which can be configured and adjusted based on actual requirements) are deleted; (3) data formatting: the deduplicated document is arranged into a unified format and stored in the library, and the fields of the library include the storage time, title, body, original body, and the like.

[0064] After that, corresponding security verification needs to be performed: using a pre-trained preset security detection model to detect the cleaned document, mainly including illegal and irregular, pornography and vulgarity, physical and mental health, ethics and morality, etc. for inspection, and sending the suspected risky content to relevant personnel for review for final risk labeling confirmation, and deleting the content confirmed to be risky. Finally, corresponding index construction needs to be performed: (1) word segmentation processing: the text content in the obtained verified document is subjected to word segmentation, and the continuous text is cut into meaningful lexical units. This process may need to use a specific word segmentation algorithm to adapt to the characteristics of different languages; (2) removing stop words: removing common stop words (such as "of", "and", "is", etc.) from the word segmentation results, which usually have no actual meaning in search; (3) stem extraction or lemmatization: performing stem extraction or lemmatization on the words, and unifying the words to the basic form to reduce the repetition and size of the index; (4) building an inverted index: creating an inverted index, which is a vocabulary-centered index structure that maps each word to a list of documents containing it. This includes building a vocabulary table and an inverted list. In this way, the corresponding retrieval library and the security knowledge base as a question retrieval engine can be obtained.

[0065] Step S12: When the search result indicates that there is no reply information corresponding to the traffic-related problem in the security knowledge base, it is judged based on a preset question security interception strategy whether the question interception operation corresponding to the traffic-related problem is triggered at present to obtain a corresponding first judgment result.

[0066] It needs to be understood that, in combination with FIG. 2, if the security knowledge base does not retrieve corresponding reply information, the next step needs to be followed to determine whether the current problem interception operation corresponding to the traffic-related problem is triggered based on the preset problem security interception strategy (first preset blacklist + target problem security interception model). Specifically, the determination of whether the current problem interception operation corresponding to the traffic-related problem is triggered based on the preset problem security interception strategy to obtain a corresponding first determination result includes: based on the traffic-related problem, a corresponding string pattern matching and embedding semantic similarity matching are performed on a plurality of word entries in the first preset blacklist to obtain a corresponding first matching result; when the first matching result indicates that there is no word entry in the first preset blacklist that matches the traffic-related problem, the traffic-related problem is input to a target problem security interception model constructed based on a large model in advance to obtain a corresponding first model output result; when there is no corresponding first security risk information and first risk type information in the first model output result, a corresponding first determination result indicating that the problem interception operation is not triggered is obtained; when the first model output result includes corresponding first security risk information and first risk type information, a corresponding first determination result indicating that the problem interception operation is triggered is obtained. It can be understood that if there is a security risk, the problem is intercepted according to the first security risk information and the first risk type information.

[0067] In the present embodiment, regarding the first preset blacklist (i.e., the query security interception strategy in FIG. 2), the following fields are mainly included: pattern, match_type, risk_type, risk_sub_type, interception_type, answer_template, and update_time. The pattern is a blacklisted word entry to be matched. The match_type is a matching type, including string pattern matching and embedding semantic similarity matching. The risk_type is a security risk type, including illegal and irregular, pornography and vulgarity, violence and terrorism, physical and mental health, ethics and morality, and the like. The risk_sub_type is a secondary violation type under a primary violation type. The interception_type is an intervention type, including user question withdrawal and refusal to answer. The answer_template is a refusal to answer template. Different violation types, intervention types, and refusal templates are not the same and can be configured independently.

[0068] Further, in this embodiment, regarding the process of keyword matching, (1) string model matching: based on the fastest AC automatic machine (Aho-Corasick automatic machine, an algorithm for string search) in the industry, it can quickly realize the matching of multiple patterns. The AC automatic machine algorithm is to construct a Trie tree, and then add additional mismatch pointers. These additional matching pointers allow backtracking when a string search fails (for example, after searching for the word bef in the Trie tree fails, but the word bea is stored in the Trie tree, the mismatch pointer points to the prefix be), turning to some prefix branches, avoiding repeated matching of prefixes, improving algorithm efficiency, and the time complexity of the algorithm is the sum of the length of the input string and the number of matches. (2) embedding similar semantic matching: using the self-developed fine-tuned BGE model (BAAI General Embedding, a general Embedding model) to embed the input question and the keywords in the first preset blacklist library, then calculate the cos similarity, and the similarity is considered to be matched above a certain threshold. This process uses a corpus of security risks, performs similar matching based on the original BGE model, then manually annotates to remove irrelevant dirty data, constructs a pair of keywords, and fine-tunes the original BGE model. In this way, the accuracy of the matching can be effectively improved.

[0069] It can be understood that, for fine-tuning of the target problem security interception model, the fine-tuning is performed on a large model using sensitive or inappropriate security data. That is, before the traffic-related problem is input into the target problem security interception model constructed based on the large model in advance, the following steps are further included: constructing a corresponding sensitive word, and judging whether each historical problem in the current historical problem list has a security risk through the sensitive word to obtain a corresponding risk judgment result; labeling the corresponding historical problem based on each risk judgment result to obtain a corresponding first training set; wherein the first training set includes corresponding positive samples and negative samples, the positive samples are the historical problems that have been labeled as having a risk type, and the negative samples are the historical problems that have been labeled as not having a risk type or not hitting a sensitive word; performing corresponding model architecture design based on a large model to obtain an initial problem security interception model; during the fine-tuning training of the initial problem security interception model (specifically including an input layer, an embedding layer, a transformer encoder stack, and a coding output layer) using the first training set, performing model evaluation operations on the model after each round of training based on a preset risk interception rate formula until a preset training termination condition is met, and obtaining a corresponding target problem security interception model. In addition, for model evaluation, the model effect is evaluated through the risk interception rate of security problems and the interception false injury rate of normal problems. The calculation formula of the risk interception rate is: the number of security problems intercepted by the model / the total set of security problems; the calculation formula of the interception false injury rate is: the number of normal problems intercepted by the model / the total set of normal problems. Generally, the lower the risk interception rate, the better the security effect, the lower the interception false injury rate, the less the impact on user experience, and the better the user experience.

[0070] Step S13, when the first judgment result indicates that the problem interception operation is not triggered, the traffic-related problem is prompted and enhanced based on a preset prompt to obtain a corresponding enhanced problem.

[0071] It should be understood that, in the present embodiment, the problem is enhanced through a carefully designed prompt, which can enhance the security of the model and ensure that the generated content meets specific guidelines and standards. The principles of the preset prompt design include the following: conforming to national conditions, respecting national customs and cultural traditions, complying with laws and regulations, complying with ethics, protecting personal safety, advocating socialist core values, opposing any form of discrimination and prejudice, prohibiting false and misleading information, prohibiting pornography, violence and terrorism content, respecting intellectual property rights, strengthening protection of minors, strengthening network security awareness, and complying with industry standards and professional ethics.

[0072] Step S14, processing the enhanced problem based on the target traffic large model that has been fine-tuned in advance to obtain corresponding target reply information.

[0073] Specifically, in the embodiment, before the target traffic large model based on pre-fine-tuning processes the enhanced problem, it further includes: obtaining a corresponding safe question and answer pair dataset by creating a corresponding fine-tuning question set and a corresponding fine-tuning answer set; collecting a corresponding traffic professional corpus, and detecting the traffic professional corpus based on a preset question safety interception strategy to determine a corresponding traffic professional question and answer pair dataset according to the obtained search result; obtaining a corresponding second training set based on the safe question and answer pair dataset and the traffic professional question and answer pair dataset; wherein the target safe question and answer pair data in the second training set and the target traffic professional question and answer pair data satisfy a corresponding data proportion condition; and in the process of training the constructed initial traffic large model based on the second training set, fine-tuning is performed through a learning rate until a corresponding target traffic large model is obtained. When fine-tuning the traffic large model related to content safety, not only traffic-related knowledge is added, but also safety knowledge is added for fine-tuning. When the user proposes a relatively flexible safety question (such as “**Are people getting lazier?”), the large model after safety fine-tuning will guide the model to generate more positive and unbiased answers.

[0074] It should be understood that in the embodiment, the safe question and answer pair dataset includes a dataset of safety-related questions and positive answers. These data should cover a variety of sensitive topics, such as bias question sets of race, gender, etc. As for the answer set, the answer information generated by the traffic large model without safety fine-tuning is called, and then a preset answer safety interception strategy is used to determine whether there is a safety risk. If it is determined that there is a risk, the traffic large model is called again to generate an answer until the generated answer is determined to have no safety risk, so that a question and answer pair constitutes a safety fine-tuning training set. Note that when calling the answer generated by the traffic large model without safety fine-tuning, the temperature parameter of the model needs to be set higher, so that the answer generated by the large model has more diversity, so that it is easier to generate an answer without safety risk. Similarly, as for the traffic professional corpus, it is also detected based on the preset question safety interception strategy, and only safe ones are retained.

[0075] Further, regarding the fine-tuning strategy of the large model, the training effect of the model is adjusted through the learning rate, including learning rate adjustment, layer freezing / unfreezing strategy, etc. A smaller learning rate needs to be used to avoid significant changes to the model. And regarding evaluation: the generated results are evaluated manually, mainly through two indicators of safety and usefulness. The calculation formula of safety is: the number of answers with safety risks in the evaluation set / the total number of evaluation sets; the calculation formula of usefulness is: the number of correct answers in the evaluation set / the total number of evaluation sets.

[0076] Step S15, judging whether the reply interception operation corresponding to the target reply information is triggered according to the preset reply security interception strategy, to complete the question and answer operation corresponding to the traffic-related question based on the obtained second judgment result.

[0077] It needs to be understood that at this time, the input information received by the preset reply security interception strategy includes the target reply information and the corresponding enhanced question. Similar to the preset question security interception strategy, the preset reply security interception strategy specifically includes a second preset blacklist library and a target reply security interception model. In order to ensure that unsafe answers are not exposed to the outside, real-time detection is required during the answer generation process, such as detection every 10 words generated by the large model. And the word matching of the second preset blacklist library also contains string model matching and embedding similarity matching two parts. Compared with the question, the text length of the reply information is relatively long, so there are some differences in its design. The design of the pattern in the blacklist needs to be more strict; embedding similarity matching needs to segment the answer, and then perform embedding matching on each segment. If a segment matches successfully, it will be intercepted. That is, the judgment whether the reply interception operation corresponding to the target reply information is triggered according to the preset reply security interception strategy, to complete the question and answer operation corresponding to the traffic-related question based on the obtained second judgment result, includes: based on the target reply information, the corresponding string pattern matching and embedding semantic similarity matching of a plurality of word entries in the second preset blacklist library are performed to obtain the corresponding second matching result; when the second matching result indicates that there is no word entry in the second preset blacklist library that matches the target reply information, the target reply information is input to the target reply security interception model constructed based on the large model in advance to obtain the corresponding second model output result; when the second model output result contains corresponding second security risk information and second risk type information, the target reply information is intercepted based on the second security risk information and the second risk type information; when the second model output result does not contain the corresponding second security risk information and the second risk type information, the target reply information is used to answer the question to complete the question and answer operation corresponding to the traffic-related question.

[0078] Further, regarding the target reply security interception model, since the reply information is detected in real time during the generation of the reply information, the model invocation will be very frequent, and the performance of the model is high, therefore the target reply security interception model uses the answer data containing sensitive or inappropriate answers, and is trained based on the BERT model (Bidirectional Encoder Representations from Transformers). Specifically, 1) safe training set construction: based on the safety-related sensitive words, filter the answers generated by the large model, for the question and answer pairs that hit the sensitive words, label and determine whether there is indeed a security risk and the risk type. Label the samples with risk types as positive samples, and label the samples without risk or without hitting sensitive words as negative samples. 2) Model architecture design: BERT is a pre-training language representation model, and its core technology is the same as that of the large model, which can process attention of serialized data. In the training process, the selected loss function is cross entropy, the Adam (Adaptive Moment Estimation, an adaptive optimization algorithm) optimizer is used to adjust the learning rate, the early stopping method (Early Stopping) is used to avoid overfitting, the performance on the validation set is monitored, and once the performance starts to decline, the training is stopped to avoid overfitting. 3) Model evaluation: the reply security interception model evaluates the model effect through the risk interception rate of the security question and the interception false injury rate of the normal question. The calculation formula of the risk interception rate is: the security questions intercepted by the model / the whole set of security questions; the calculation formula of the interception false injury rate is: the normal questions intercepted by the model / the whole set of normal questions.

[0079] In summary, in this embodiment, the following beneficial effects are achieved: (1) Full-level security protection: effectively realizes the full-level security protection from the input layer, model layer to the output layer, effectively prevents and reduces the abnormal harmful content generated by the generative large model. (2) Efficient security knowledge retrieval: through the construction and optimization of the security knowledge base, the rapid and accurate retrieval of related problems in the transportation field is realized, and the security and accuracy of information retrieval are improved. (3) Enhanced content security: by using the preset question security interception strategy and prompt enhancement technology, the security control of user input and model output content is significantly improved, and the generation of inappropriate content is reduced. (3) Improve the security and usability of the traffic large model: the safety fine-tuning method ensures that the traffic large model can generate more positive and unbiased answers when dealing with sensitive problems, while maintaining the usability and accuracy of the model's response. (4) Optimized model performance: under the premise of ensuring security, by adjusting the model architecture and training strategy, the balance between model performance and security is achieved, and the running efficiency of the model is improved. (5) Real-time security detection: the preset answer security interception strategy can detect the answer content in real time, ensuring the security during the answer generation process and reducing the exposure of security risks. (6) Reduce the misinterception rate: through accurate model evaluation and monitoring of risk interception rate and interception injury rate, the injury to normal problems is effectively reduced, and the user experience is improved. (7) Improve the generalization ability of the model: through safety fine-tuning, the model not only performs well on specific tasks, but also enhances the processing ability of multiple sensitive topics, improving the generalization of the model.

[0080] It can be seen that in the embodiment of the application, a traffic-related problem to be processed is acquired, and the traffic-related problem is searched based on a pre-constructed safety knowledge base to obtain a corresponding search result; when the search result indicates that there is no reply information corresponding to the traffic-related problem in the safety knowledge base, it is determined whether a problem interception operation corresponding to the traffic-related problem is triggered based on a preset problem safety interception strategy to obtain a corresponding first determination result; when the first determination result indicates that the problem interception operation is not triggered, the traffic-related problem is enhanced based on a preset prompt to obtain an enhanced problem; the enhanced problem is processed based on a target traffic large model that has been pre-tuned to obtain corresponding target reply information; and it is determined whether a reply interception operation corresponding to the target reply information is triggered according to a preset reply safety interception strategy to complete a question and answer operation corresponding to the traffic-related problem based on a second determination result obtained. That is, in the application, it is determined whether to perform problem interception at the problem input level through the safety knowledge base and the preset problem safety interception strategy, and if not, the target reply information is obtained based on the pre-tuned target traffic large model after prompt, and then it is determined whether to perform reply interception at the output level based on the preset reply safety interception strategy. In this way, on the one hand, the input of malicious data and the output of security risk content can be effectively prevented, and the safety, efficiency and accuracy of the question and answer can be improved; on the other hand, the abuse of the model can be effectively prevented, thereby ensuring the user experience of using the traffic large model and providing a solid safety guarantee for the construction and use of the subsequent intelligent traffic system.

[0081] Referring to FIG. 4, the embodiment of the application also discloses a large model question and answer device in the traffic industry, comprising:

[0082] The knowledge base searching module 11 is configured to acquire a traffic-related problem to be processed, and search the traffic-related problem based on a pre-constructed safety knowledge base to obtain a corresponding search result.

[0083] The problem interception determination module 12 is configured to, when the search result indicates that there is no reply information corresponding to the traffic-related problem in the safety knowledge base, determine whether a problem interception operation corresponding to the traffic-related problem is triggered based on a preset problem safety interception strategy to obtain a corresponding first determination result.

[0084] The problem enhancement module 13 is configured to, when the first determination result indicates that the problem interception operation is not triggered, enhance the traffic-related problem based on a preset prompt to obtain an enhanced problem.

[0085] The reply information obtaining module 14 is configured to process the enhanced question based on the target traffic large model that has been fine-tuned in advance to obtain corresponding target reply information.

[0086] The reply interception judgment module 15 is configured to determine whether the reply interception operation corresponding to the target reply information is triggered according to a preset reply security interception strategy, so as to complete the question and answer operation corresponding to the traffic-related question based on the obtained second judgment result.

[0087] The working processes of the above modules are more specifically described in the foregoing embodiments, and thus will not be repeated here.

[0088] As can be seen, in the present application, the safety knowledge base and the preset question security interception strategy are used to determine whether to intercept the question at the question input level. If not, the target reply information is obtained based on the target traffic large model that has been fine-tuned in advance after the prompt. Then, it is determined whether to intercept the reply at the output level based on the preset reply security interception strategy. In this way, on the one hand, the input of malicious data and the output of security risk content can be effectively prevented, and the safety, efficiency and accuracy of the question and answer can be improved. On the other hand, the abuse of the model can be effectively prevented, so as to protect the experience of users using the traffic large model and provide a solid security guarantee for the construction and use of the subsequent intelligent traffic system.

[0089] In some specific embodiments, the knowledge base retrieval module 11 can specifically include:

[0090] The syntax analysis unit is configured to perform corresponding question preprocessing, question segmentation and question analysis on the traffic-related question based on the pre-constructed safety knowledge base, complete the corresponding syntax analysis operation, and obtain an analysis result.

[0091] The retrieval result obtaining unit is configured to retrieve the analysis result based on the index library in the safety knowledge base to obtain a corresponding retrieval result.

[0092] In some specific embodiments, the traffic industry large model question and answer device can further include:

[0093] The document crawling unit is configured to crawl a web page recording target traffic-related information based on python to obtain a corresponding web document.

[0094] The document cleaning unit is configured to parse, remove duplicates and format data of the web document based on a preset document cleaning rule to obtain a corresponding cleaned document.

[0095] The document checking unit is configured to perform risk content detection and risk content deletion on the cleaned document by using a preset security detection model, so as to complete a corresponding document checking operation and obtain a checked document.

[0096] The knowledge base construction unit is configured to perform word segmentation processing, stop word removal, and inverted index construction on the checked document, so as to obtain a corresponding search library and a security knowledge base serving as a question search engine.

[0097] In some embodiments, the question interception judgment module 12 can specifically include:

[0098] The question matching unit is configured to perform corresponding string pattern matching and embedding semantic similarity matching on a plurality of word entries in a first preset blacklist library based on the traffic-related question, so as to obtain a corresponding first matching result.

[0099] The first model processing unit is configured to input the traffic-related question to a target question security interception model that is constructed in advance based on a large model when the first matching result indicates that there is no word entry in the first preset blacklist library that matches the traffic-related question, so as to obtain a corresponding first model output result.

[0100] The first result acquisition unit is configured to obtain a first judgment result indicating that the question interception operation is not triggered when there is no corresponding first security risk information and first risk type information in the first model output result.

[0101] The second result acquisition unit is configured to obtain the first judgment result indicating that the question interception operation is triggered when there is corresponding first security risk information and first risk type information in the first model output result.

[0102] In some embodiments, the large model question and answer device for the traffic industry can specifically further include:

[0103] The risk judgment unit is configured to construct a corresponding sensitive word, and determine whether each historical question in a current historical question list has a security risk by using the sensitive word, so as to obtain a corresponding risk judgment result.

[0104] The first training set acquisition unit is configured to label each historical question corresponding to the risk judgment result based on the risk judgment result, so as to obtain a corresponding first training set. The first training set includes corresponding positive samples and negative samples. The positive samples are historical questions that have been labeled as having a risk type, and the negative samples are historical questions that have been labeled as not having a risk type or not hitting a sensitive word.

[0105] The model design unit is configured to design a corresponding model architecture based on a large model to obtain an initial question safety interception model.

[0106] The model fine-tuning unit is configured to perform model evaluation on the model after each round of training based on a preset risk interception rate formula during fine-tuning training of the initial question safety interception model by using the first training set, until a preset training termination condition is met, to obtain a corresponding target question safety interception model.

[0107] In some embodiments, the large model question and answer device for the transportation industry can further include:

[0108] The safety question and answer pair data set construction unit is configured to obtain a corresponding safety question and answer pair data set by creating a corresponding fine-tuning question set and a corresponding fine-tuning answer set.

[0109] The professional question and answer pair data set acquisition unit is configured to collect a corresponding transportation professional corpus, detect the transportation professional corpus based on a preset question safety interception strategy, and determine a corresponding transportation professional question and answer pair data set based on the obtained search results.

[0110] The second training set acquisition unit is configured to obtain a corresponding second training set based on the safety question and answer pair data set and the transportation professional question and answer pair data set, wherein a target safety question and answer pair data and a target transportation professional question and answer pair data in the second training set satisfy a corresponding data proportion condition.

[0111] The target large model acquisition unit is configured to fine-tune by learning rate during training of the constructed initial transportation large model based on the second training set, until a corresponding target transportation large model is obtained.

[0112] In some embodiments, the reply interception judgment module 15 can specifically include:

[0113] The reply matching unit is configured to perform corresponding string pattern matching and embedding semantic similarity matching on a plurality of words in a second preset blacklist based on the target reply information to obtain a corresponding second matching result.

[0114] The second model processing unit is configured to input the target reply information into a target reply safety interception model constructed based on a large model when the second matching result indicates that there is no word in the second preset blacklist that matches the target reply information, to obtain a corresponding second model output result.

[0115] The reply interception unit is configured to intercept the target reply information based on the second security risk information and the second risk type information when the second security risk information and the second risk type information exist in the second model output result.

[0116] The question and answer completion unit is configured to reply to the question based on the target reply information to complete the question and answer operation corresponding to the traffic-related question when the second security risk information and the second risk type information do not exist in the second model output result.

[0117] Further, the embodiment of the present application further discloses an electronic device, and FIG. 5 is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the use range of the present application.

[0118] FIG. 5 is a structural schematic diagram of an electronic device 20 provided by the embodiment of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the related steps in the traffic industry large model question and answer method disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be an electronic computer.

[0119] In the embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input and output interface 25 is used to obtain external input data or output data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0120] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.

[0121] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the large model question and answer method of the traffic industry disclosed in any of the preceding embodiments executed by the electronic device 20, the computer program 222 can further include a computer program capable of completing other specific work.

[0122] Further, the present application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the large model question and answer method of the traffic industry disclosed in the preceding embodiments. For the specific steps of the method, please refer to the corresponding content disclosed in the preceding embodiments, which will not be repeated here.

[0123] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant part can be referred to the method part.

[0124] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly show the interchangeability of hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0125] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0126] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the stated element.

[0127] The above detailed description of the technical solutions provided by the present application has been provided, and the principles and implementation modes of the present application have been described by applying specific examples. The above description of the examples is only for the purpose of helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the content of the specification should not be understood as a limitation of the present application.

Claims

1. A large model question and answer method in the transportation industry, characterized in that, The method comprises the following steps: acquiring a traffic-related problem to be processed, and searching the traffic-related problem based on a pre-constructed safety knowledge base to obtain a corresponding search result; when the search result indicates that there is no reply information corresponding to the traffic-related problem in the safety knowledge base, judging whether a problem interception operation corresponding to the traffic-related problem is triggered based on a preset problem safety interception strategy to obtain a corresponding first judgment result; when the first judgment result indicates that the problem interception operation is not triggered, enhancing the traffic-related problem based on a preset prompt to obtain an enhanced problem; processing the enhanced problem based on a target traffic model that has been fine-tuned in advance to obtain corresponding target reply information; judging whether a reply interception operation corresponding to the target reply information is triggered according to a preset reply safety interception strategy to complete a question and answer operation corresponding to the traffic-related problem based on a second judgment result obtained.

2. The method of claim 1, wherein, The method further comprises the following steps: performing corresponding problem preprocessing, problem word segmentation, problem analysis on the traffic-related problem based on the pre-constructed safety knowledge base, completing corresponding syntax analysis operations, and obtaining analysis results; searching the analysis results based on an index library in the safety knowledge base to obtain a corresponding search result.

3. The method of claim 1, wherein, Before the searching the traffic-related problem based on the pre-constructed safety knowledge base to obtain a corresponding search result, the method further comprises the following steps: based on python, crawling a webpage recording target traffic-related information to obtain a corresponding webpage document; based on a preset document cleaning rule, analyzing, deduplicating, and data formatting the webpage document to obtain a corresponding cleaned document; using a preset safety detection model to detect and delete risk content of the cleaned document to complete a corresponding document verification operation and obtain a verified document; by performing word segmentation processing, stop word removal, and inverted index construction on the verified document, obtaining a corresponding search library and a safety knowledge base serving as a question search engine.

4. The method of claim 1, wherein, The method further comprises the following steps: based on the traffic-related problem, performing corresponding string pattern matching and embedding semantic similarity matching on a plurality of word entries in a first preset blacklist library to obtain a corresponding first matching result; when the first matching result indicates that there is no word entry in the first preset blacklist library that matches the traffic-related problem, inputting the traffic-related problem into a target problem safety interception model that is pre-constructed based on a large model to obtain a corresponding first model output result; when there is no corresponding first safety risk information and first risk type information in the first model output result, obtaining a corresponding first judgment result indicating that the problem interception operation is not triggered; When the first model output result contains corresponding first safety risk information and the first risk type information, a corresponding first judgment result indicating that the problem interception operation is triggered is obtained.

5. The method of claim 4, wherein, Before the traffic-related problem is input into the target problem safety interception model constructed in advance based on a large model, the method further includes: corresponding sensitive words are constructed, and whether each historical problem in the current historical problem list has a safety risk is determined through the sensitive words to obtain a corresponding risk judgment result; based on each risk judgment result, the corresponding historical problem is labeled to obtain a corresponding first training set; wherein the first training set includes corresponding positive samples and negative samples, the positive samples are historical problems that have been labeled as having a risk type, and the negative samples are historical problems that have been labeled as not having a risk type or not hitting a sensitive word; a corresponding model architecture design is performed based on a large model to obtain an initial problem safety interception model; During the fine-tuning training of the initial problem safety interception model using the first training set, a model evaluation operation is performed on the model after each round of training based on a preset risk interception rate formula until a preset training termination condition is met, and a corresponding target problem safety interception model is obtained.

6. The method of claim 1, wherein, Before the enhanced problem is processed based on the target traffic large model that has been fine-tuned in advance, the method further includes: a corresponding fine-tuning question set and a corresponding fine-tuning answer set are created to obtain a corresponding safety question and answer pair data set; corresponding traffic professional corpus is collected, and the traffic professional corpus is detected based on a preset problem safety interception strategy to determine a corresponding traffic professional question and answer pair data set according to the obtained search result; a corresponding second training set is obtained based on the safety question and answer pair data set and the traffic professional question and answer pair data set; wherein the target safety question and answer pair data and the target traffic professional question and answer pair data in the second training set satisfy a corresponding data proportion condition; During the training of the initial traffic large model based on the second training set, the learning rate is fine-tuned until a corresponding target traffic large model is obtained.

7. The method of claim 1-6, wherein, The second judgment result is obtained based on the preset answer safety interception strategy to determine whether the reply interception operation corresponding to the target reply information is triggered, and the question and answer operation corresponding to the traffic-related problem is completed based on the obtained second judgment result, including: a corresponding string pattern matching and embedding semantic similarity matching are performed on a plurality of words in a second preset blacklist library based on the target reply information to obtain a corresponding second matching result; When the second matching result indicates that there is no word in the second preset blacklist library that matches the target reply information, the target reply information is input into a target reply safety interception model constructed in advance based on a large model to obtain a corresponding second model output result; When the second model output result contains corresponding second safety risk information and second risk type information, the target reply information is intercepted based on the second safety risk information and the second risk type information. When the second model output result does not contain corresponding second safety risk information and second risk type information, question answering is performed based on the target reply information to complete the question and answer operation corresponding to the traffic-related question.

8. A large model question and answer device in the transportation industry, characterized in that, Comprise: A knowledge base retrieval module is configured to acquire a traffic-related question to be processed, and retrieve the traffic-related question based on a pre-constructed safety knowledge base to obtain a corresponding retrieval result; A question interception judgment module is configured to, when the retrieval result indicates that the safety knowledge base does not contain reply information corresponding to the traffic-related question, judge whether a question interception operation corresponding to the traffic-related question is triggered based on a pre-set question safety interception strategy to obtain a corresponding first judgment result; A question enhancement module is configured to, when the first judgment result indicates that the question interception operation is not triggered, enhance the traffic-related question based on a pre-set prompt to obtain an enhanced question; A reply information acquisition module is configured to process the enhanced question based on a target traffic large model that has been fine-tuned in advance to obtain corresponding target reply information; A reply interception judgment module is configured to judge whether a reply interception operation corresponding to the target reply information is triggered according to a pre-set answer safety interception strategy to complete the question and answer operation corresponding to the traffic-related question based on the obtained second judgment result.

9. An electronic device, comprising: Comprise: A memory is configured to save a computer program; A processor is configured to execute the computer program to implement the large model question and answer method for the traffic industry according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is saved, and the computer program is executed by a processor to implement the large model question and answer method for the traffic industry according to any one of claims 1 to 7.

Images