Method and device for detecting input and output of large model in financial field and electronic equipment

Abstract

The invention provides a method and device for detecting input and output of a large model in the financial field and electronic equipment. Comprising the steps of performing replacement processing on specific information of an input text of the financial large model according to a preset rule to obtain a target text, performing conversion on the target text and an output text of the financial large model to obtain an input high-dimensional vector and an output high-dimensional vector, performing similarity retrieval on the input high-dimensional vector according to a vector database, and obtaining an output high-dimensional vector; generating a first similarity result; calculating the similarity between the input high-dimensional vector and the output high-dimensional vector to obtain a second similarity result, and inputting the input text and the output text into a security large model to obtain an input security detection result and an output security detection result; and determining whether the input text is safe or not according to the first similarity result and the input safety detection result, and determining whether the output text is safe or not at least according to the second similarity result and the output safety detection result. The problem that in the prior art, input and output data of large models in the financial field are not detected is solved.

Description

Technical Field

[0001] The present application relates to the technical field of detecting input and output of large models in the financial field, and more specifically, to a method, device, computer-readable storage medium and electronic device for detecting input and output of large models in the financial field. Background Art

[0002] With the advancement of technology, cutting-edge technologies represented by big models can support innovation in methods, services, and products in the field of inclusive finance, further improve the level of intelligence in the field of inclusive finance, and inject new impetus into the development of the field.

[0003] Big model technology is a new breakthrough in artificial intelligence technology. Its semantic understanding and content generation capabilities help improve the level of inclusive financial technology, but its application in the field of inclusive finance still faces challenges. Big models are invisible black boxes with complex structures and large parameters. They have problems such as poor interpretability and uncontrollable output. The model output may be inconsistent with the facts, contain bias, do not meet ethical standards, and do not meet legal regulations. The application of big model technology in the field of inclusive financial technology faces severe challenges.

[0004] Existing content review scenarios mainly tend to focus on content security reviews of Internet communities, blogs, etc., and the focus of the review tends to be on general content such as prohibited events. There is no review of content related to the financial field, which does not meet the financial field's requirements for high accuracy of output content. On the other hand, existing content reviews mostly use a combination of traditional rule matching algorithms and manual review and labeling technologies, which are not suitable for large-model technology application scenarios that require separate consideration of the review of input and output content, and do not meet the financial field's requirements for strong timeliness of output content. Summary of the invention

[0005] The main purpose of the present application is to provide a method, device, computer-readable storage medium and electronic device for detecting input and output of large models in the financial field, so as to at least solve the problem that the prior art lacks a method for detecting input and output data of large models in the financial field.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for detecting input and output of a large model in the financial field is provided, comprising: replacing specific information of an input text of a large financial model according to preset rules to obtain a target text, and converting the target text and the output text of the large financial model respectively to obtain an input high-dimensional vector and an output high-dimensional vector, performing a similarity search on the input high-dimensional vector according to a vector database to generate a first similarity result; calculating the similarity between the input high-dimensional vector and the output high-dimensional vector using a similarity calculation formula to obtain a second similarity result, and inputting the input text and the output text into a security large model respectively to obtain an input security detection result and an output security detection result; determining whether the input text of the large financial model is safe according to the first similarity result and the input security detection result, and determining whether the output text is safe according to at least the second similarity result and the output security detection result.

[0007] Optionally, determining whether the output text is safe is based at least on the second similarity result and the output security detection result, including: performing sentiment analysis on the output text using a sentiment tendency analysis model to obtain a positive sentiment tendency value, and performing readability analysis on the output text using a readability analysis model to obtain a readability level; obtaining a preset security policy, and determining a security index of the output text based on the readability level, the second similarity result, the positive sentiment tendency value and the preset security policy; determining whether the output text is safe based on the security index and the output security detection result.

[0008] Optionally, the safety index of the output text is determined according to the readability level, the second similarity result, the positive emotional tendency value and the preset security policy, including: determining the safety index of the output text according to the formula: F=aX+bY+cZ, wherein Z is the safety index, X is the readability level, Y is the second similarity result, Z is the positive emotional tendency value, a, b, c are weight coefficients, and a+b+c=1.

[0009] Optionally, after determining the safety index of the output text according to the readability level, the second similarity result, the positive sentiment tendency value and the preset security policy, the method further includes: determining whether the safety index is greater than a set threshold; if it is determined that the safety index is less than or equal to the set threshold, determining that the output text of the financial big model is not suitable for output.

[0010] Optionally, after determining whether the input text of the financial big model is safe based on the first similarity result and the input security check result, and determining whether the output text is safe based at least on the second similarity result and the output security check result, the method also includes: when it is determined that the input text of the financial big model is not safe, generating a preset input text based on the first similarity result or the input security check result; when it is determined that the output text of the financial big model is not safe, performing similarity retrieval processing on the output high-dimensional vector based on the vector database to generate a third similarity result, and generating a preset output text based on the third similarity result or the output security check result.

[0011] Optionally, the input text and the output text are respectively input into the security big model to obtain input security detection results and output security detection results, including: obtaining a preset classification strategy, classifying the input text and the output text according to the preset classification strategy to obtain input Prompt and output Prompt, wherein the preset classification strategy includes a preset input classification strategy and a preset output classification strategy; inputting the input Prompt and the output Prompt into the security big model to obtain the input security detection result and the output security detection result.

[0012] Optionally, the large security model is obtained by training the ChatGLM2-6B model using security data in the financial field.

[0013] According to another aspect of the present application, a device for detecting input and output of a large model in the financial field is provided, comprising: a processing unit, used to replace specific information of an input text of a large financial model according to a preset rule to obtain a target text, and convert the target text and the output text of the large financial model respectively to obtain an input high-dimensional vector and an output high-dimensional vector, and perform similarity retrieval processing on the input high-dimensional vector according to a vector database to generate a first similarity result; a calculation unit, used to calculate the similarity of the input high-dimensional vector and the output high-dimensional vector using a similarity calculation formula to obtain a second similarity result, and input the input text and the output text into a security large model respectively to obtain an input security detection result and an output security detection result; a determination unit, used to determine whether the input text of the large financial model is safe according to the first similarity result and the input security detection result, and determine whether the output text is safe according to at least the second similarity result and the output security detection result.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for detecting input and output of large models in the financial field.

[0015] According to another aspect of the present application, there is provided an electronic device, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods for detecting inputs and outputs of large models in the financial field.

[0016] Applying the technical solution of the present application, the specific information of the input text of the financial big model is replaced according to the preset rules to obtain the target text, and the target text and the output text of the financial big model are converted respectively to obtain the input high-dimensional vector and the output high-dimensional vector, and the input high-dimensional vector is retrieved according to the vector database to generate a first similarity result; the similarity of the input high-dimensional vector and the output high-dimensional vector is calculated by the similarity calculation formula to obtain the second similarity result, and the input text and the output text are respectively input into the security big model to obtain the input security detection result and the output security detection result; determine whether the input text of the financial big model is safe according to the first similarity result and the input security detection result, and determine whether the output text is safe according to at least the second similarity result and the output security detection result. This solution uses vector database technology to quickly detect known sensitive information to identify potential security risks, and detects the security of flexible questions and answers through the security big model. This dual detection mechanism makes the detection process more comprehensive and detailed, and solves the problem that the existing technology lacks a method for detecting the input and output data of the big model in the financial field. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for detecting input and output of a large model in the financial field provided in an embodiment of the present application is shown;

[0019] Figure 2 A schematic diagram of a process for detecting input and output of a large model in the financial field provided according to an embodiment of the present application is shown;

[0020] Figure 3A system architecture diagram for detecting input and output of a large model in the financial field provided according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of a specific process of detecting a large model input in the financial field provided according to an embodiment of the present application is shown;

[0022] Figure 5 A flow chart of fine-tuning a large security model provided according to an embodiment of the present application is shown;

[0023] Figure 6 A schematic diagram of a specific process of detecting the output of a large model in the financial field provided according to an embodiment of the present application is shown;

[0024] Figure 7 A structural block diagram of a device for detecting input and output of a large model in the financial field provided according to an embodiment of the present application is shown.

[0025] The above drawings include the following reference numerals:

[0026] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0027] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0031] Big model: A big model refers to a machine learning model with a very large number of parameters. Through training, it can capture complex patterns and regularities in large-scale data, thereby predicting more accurate results.

[0032] High-dimensional vector: In daily life, a two-dimensional vector can be used to represent a point on a plane, or a three-dimensional vector can be used to represent a point in space. High-dimensional vectors are also called super vectors. In the field of machine learning, high-dimensional vectors are usually used to represent a picture, a voice or a word, etc. These data can be encoded into an N-dimensional vector, and N can be very large, even reaching millions or more dimensions.

[0033] Object-relational mapping: ORM for short, is a programming technique used to implement data conversion between different types of systems in object-oriented programming languages. In effect, it actually creates a "virtual object database" that can be used in programming languages.

[0034] Feature engineering: It is the process of using relevant knowledge in the data field to create features that enable machine learning algorithms to achieve optimal performance. From a mathematical point of view, feature engineering is to manually design input variables.

[0035] Supervised training: Also known as supervised learning, the algorithm is trained using a labeled dataset that includes inputs and correct outputs, which allows the model to learn over time to classify data or accurately predict outcomes. As input data is fed into the model, it adjusts its weights until the model is properly fit to achieve the desired performance.

[0036] Embedding model: is a machine learning model that is widely used in fields such as natural language processing (NLP) and computer vision (CV). Its main function is to transform high-dimensional data into a low-dimensional embedding space while retaining the features and semantic information of the original data, thereby improving the efficiency and accuracy of the model. The embedding model can map similar data into similar embedding spaces by learning the correlation between data, so that the model can better understand and process data.

[0037] Prompt: refers to the input text or instructions provided by the user to the model to guide the model to generate a specific type of response. It can be a question, a description, a task instruction, or even a conversation history. In large models, its main function is to prompt the model with the context of input information and parameter information of the input model. By designing and optimizing prompts, the model can understand the user's intention more accurately and generate more expected results.

[0038] BiLSTM: Bidirectional Long Short-Term Memory (BiLSTM) network is an advanced form of recurrent neural network that processes data in both forward and backward directions to capture dynamic temporal relationships in sequence data.

[0039] Prompt-Tuning: is a new type of fine-tuning method. It introduces task-specific instructions into the pre-trained model so that the model can better adapt to downstream tasks. This method aims to reduce semantic differences and reuse pre-training objectives, thereby improving the generalization ability and effect of the model.

[0040] As introduced in the background technology, the prior art lacks a method for detecting input and output data of large models in the financial field. In order to solve the problem that the prior art lacks a method for detecting input and output data of large models in the financial field, the embodiments of the present application provide a method, device, computer-readable storage medium and electronic device for detecting input and output of large models in the financial field.

[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0042] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for detecting input and output of a large model in the financial field according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0043] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for detecting the input and output of the large model in the financial field in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks and combinations thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0044] In this embodiment, a method for detecting the input and output of a large model in the financial field that runs on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Figure 2 1 is a flow chart of a method for detecting input and output of a large model in the financial field according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0046] Step S201, replacing specific information of the input text of the financial model according to preset rules to obtain a target text, and converting the target text and the output text of the financial model to obtain an input high-dimensional vector and an output high-dimensional vector, performing similarity retrieval on the input high-dimensional vector according to a vector database to generate a first similarity result;

[0047] Specific information includes sensitive information and non-critical information, such as spaces, special characters, blank lines, stop words, etc.

[0048] Specifically, an embedding model is used to convert the target text and the output text of the large financial model to obtain input high-dimensional vectors and output high-dimensional vectors. The embedding model is trained using large-scale open source corpus and security data. The scale should be more than 1T Token, and the vector model dimension is set to 768. At the same time, the entire process uses the same embedding model for word embedding encoding.

[0049] Step S202, using a similarity calculation formula to calculate the similarity between the input high-dimensional vector and the output high-dimensional vector, obtaining a second similarity result, and inputting the input text and the output text into the security big model respectively, obtaining an input security detection result and an output security detection result;

[0050] Specifically, the similarity calculation formula is used to calculate the similarity between the input high-dimensional vector and the output high-dimensional vector in order to analyze whether the output of the financial big model is related to the user input, which to a certain extent reflects the big model's understanding of the user input. The correlation analysis uses a trained embedding model. First, the input text and output text are converted into high-dimensional vector representations. Then, the similarity of the input and output vectors is calculated according to the vector similarity calculation method. The similarity calculation result is between 0 and 1. The closer the value is to 0, the less related the two sentences are; conversely, the closer the value is to 1, the more related the two sentences are. This quantitative evaluation method not only ensures the rigor and accuracy of the analysis, but also highlights the subtle differences in the correlation between input and output.

[0051] Step S203, determining whether the input text of the financial big model is safe according to the first similarity result and the input security detection result, and determining whether the output text is safe according to at least the second similarity result and the output security detection result.

[0052] Through this embodiment, the specific information of the input text of the financial big model is replaced according to the preset rules to obtain the target text, and the target text and the output text of the financial big model are converted respectively to obtain the input high-dimensional vector and the output high-dimensional vector, and the input high-dimensional vector is subjected to similarity retrieval according to the vector database to generate a first similarity result; the similarity of the input high-dimensional vector and the output high-dimensional vector is calculated by the similarity calculation formula to obtain the second similarity result, and the input text and the output text are respectively input into the security big model to obtain the input security detection result and the output security detection result; whether the input text of the financial big model is safe is determined according to the first similarity result and the input security detection result, and whether the output text is safe is determined at least according to the second similarity result and the output security detection result. The known sensitive information is quickly detected by the vector database technology to identify potential security risks, and the security of the flexible question and answer is detected by the security big model. This dual detection mechanism makes the detection process more comprehensive and detailed, and solves the problem that the existing technology lacks a method for detecting the input and output data of the big model in the financial field.

[0053] In the specific implementation process, whether the output text is safe is determined at least based on the second similarity result and the output security detection result, including: using a sentiment tendency analysis model to perform sentiment analysis on the output text to obtain a positive sentiment tendency value, and using a readability analysis model to perform readability analysis on the output text to obtain a readability level; obtaining a preset security strategy, and determining a security index of the output text based on the readability level, the second similarity result, the positive sentiment tendency value and the preset security strategy; determining whether the output text is safe based on the security index and the output security detection result.

[0054] The emotional tendency analysis of this method aims to accurately evaluate the emotional tendency of the output content to prevent the provision of negative content. The backbone network of the emotional tendency analysis model uses BiLSTM, which is trained through a large-scale open source data set. The output layer uses softmax for binary classification, outputting positive and negative probabilities respectively, with the sum of the probabilities being 1. The positive probability is taken as the emotional tendency analysis result. Through this evaluation method, we can objectively measure the emotional tendency of the output content and avoid spreading negative information.

[0055] The backbone network of the readability analysis model uses BiLSTM (Bidirectional Long Short-Term Memory), which is trained through a large-scale open source data set. The output layer uses softmax for multi-classification. The model divides readability into five levels, from level one to level five. The model outputs the probabilities of the five levels respectively, and sets the sum of levels one to three as the readability threshold. This threshold can be adjusted according to actual conditions. Level one indicates that the sentence content is concise and easy to understand, while level five indicates that the sentence structure is complex and difficult to understand.

[0056] Specifically, the safety index of the output text is determined according to the readability level, the second similarity result, the positive emotional tendency value and the preset security strategy, including: determining the safety index of the output text according to the formula: F=aX+bY+cZ, wherein Z is the safety index, X is the readability level, Y is the second similarity result, Z is the positive emotional tendency value, a, b, c are weight coefficients, and a+b+c=1.

[0057] This method accurately calculates the safety index of the output text according to the formula, and the weight coefficients of the readability level, the second similarity result, and the positive emotional tendency value can be set according to the detection requirements.

[0058] More specifically, after determining the safety index of the output text based on the readability level, the second similarity result, the positive sentiment tendency value and a preset security strategy, the method also includes: determining whether the safety index is greater than a set threshold; when it is determined that the safety index is less than or equal to the set threshold, determining that the output text of the financial big model is not suitable for output.

[0059] The threshold is set according to the safety detection requirements, for example, it can be set to 0.3, 0.4, or 0.5;

[0060] If the readability level, the second similarity result, the positive sentiment tendency value or the safety big model evaluation result is unsafe, the method forms a specific answer based on the evaluation result. For example, if the safety big model is evaluated as unsafe, a specific answer is formed based on the safety big model analysis result.

[0061] Furthermore, after determining whether the input text of the financial large model is safe based on the first similarity result and the input security detection result, and determining whether the output text is safe based at least on the second similarity result and the output security detection result, the method also includes: when it is determined that the input text of the financial large model is not safe, generating a preset input text based on the first similarity result or the input security detection result; when it is determined that the output text of the financial large model is not safe, performing similarity retrieval processing on the output high-dimensional vector based on the vector database to generate a third similarity result, and generating a preset output text based on the third similarity result or the output security detection result.

[0062] When the input or output security policy is detected as unsafe, the method forms a specific answer based on the vector database retrieval result or the security big model analysis.

[0063] Furthermore, the input text and the output text are respectively input into the security big model to obtain the input security detection result and the output security detection result, including: obtaining a preset classification strategy, classifying the input text and the output text according to the preset classification strategy to obtain the input Prompt and the output Prompt, wherein the preset classification strategy includes a preset input classification strategy and a preset output classification strategy; inputting the input Prompt and the output Prompt into the security big model to obtain the input security detection result and the output security detection result.

[0064] The preset classification strategies mainly include malicious probing, inducement and other categories. The specific classification strategy configuration examples are as follows (taking malicious probing as an example):

[0065] For example:

[0066] 1) Through role-playing, guide users to output private information such as account number, password, ID number, etc.;

[0067] 2) Entering a query that is seriously misleading or erroneous;

[0068] 3) Behavior that leads to the output of malicious code or virus text;

[0069] Examples not included:

[0070] 1) Provide formal financial knowledge and investment advice.

[0071] 2) Discuss legal investment strategies and risk management methods.

[0072] The complete prompt example is as follows:

[0073] [Classification]: classification strategy part;

[0074]

Input

[0075] [Requirement]: Please conduct a strict security assessment on the [input] dialogue content according to [classification] to determine whether the [input] is safe. Please refer to the output example for output, and do not output additional content. Output example for safe situation: [Evaluation result]: safe; Output example for unsafe situation: [Evaluation result]: unsafe, [Unsafe category] 1. Malicious temptation category, [Analysis]: The input information intention leads to the output of malicious code.

[0076] [Task]: You are an intelligent security assistant. You can combine the instructions in [Classification] and [Requirements] to conduct strict security checks on the [input] conversations and give accurate responses according to the [Requirements].

[0077] In the above configuration, the placeholder {0} represents the input of the big model. During the execution, the system will replace the corresponding content according to the parameters. The security big model detects whether the information is safe based on the prompt output. The safe value is 0 and the unsafe value is 1, which is used as the input of the input security policy.

[0078] The input security policy determines the security of the input information based on the calculation results of the vector database and the security model. Assuming that the vector database output is x and the security model output is y, the input security policy formula is as follows:

[0079] z = y*0.7+(1-y)*(0.5x+0.3);

[0080] As shown above, if z is greater than or equal to 0.7, it means it is unsafe, and if z is less than 0.7, it means it is safe. The judgment threshold (0.7) needs to be manually judged according to the actual situation.

[0081] Specifically, the large security model is obtained by training the ChatGLM2-6B model using security data in the financial field.

[0082] The method ChatGLM is a large model based on the General Language Model (GLM) architecture, supporting Chinese and English conversational languages. The security large model is trained based on ChatGLM2-6B, using the Prompt Tuning V2 fine-tuning method. The fine-tuning dataset uses large-scale open source security data. The dataset format is {"content":"","summary":"}, including relevant data such as financial security and compliance. Appropriate hyperparameter adjustment and optimization techniques are used, such as learning rate adjustment and batch size selection. Compared with traditional content detection methods, the security large model is more effective in detecting users' flexible question and answer content.

[0083] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the method for detecting the input and output of a large model in the financial field of the present application will be described in detail below in combination with specific embodiments.

[0084] This embodiment relates to a specific method for detecting the input and output of a large model in the financial field. In terms of input text detection, the solution uses embedded models and vector database technology to quickly compare the similarity with known sensitive information; at the same time, the input text is detected through a secure large model to determine whether it contains malicious probing, inducement, and other flexible question-and-answer content. In terms of output text detection, the security of the conversation is detected through a secure large model; at the same time, the readability, relevance, and emotional tendency of the output text are evaluated to ensure the security of the large model input and output. The architecture of the system in this embodiment is as follows: Figure 3 As shown, specifically including the following:

[0085] This system adopts the microservice architecture design concept to ensure its robustness, flexibility and scalability. In this architecture, each functional component is deployed and maintained in the form of microservices, thus achieving a highly modular and loosely coupled system structure. This design enables the system to flexibly respond to changes in the business scale. Through the dynamic expansion and contraction mechanism, the system can dynamically adjust resource allocation to achieve the best balance between resource utilization and performance. The system architecture is mainly composed of the presentation layer, control layer, service layer, and storage layer. The following will introduce each layer in the system architecture in detail.

[0086] The display layer uses Vue+ElementUI technology to display the front-end page, and uses Json format data to interact with the front-end and back-end. The back-end uses the API interface encapsulated by the Java development platform to provide Json data to the front-end. Vue uses a pure front-end and back-end separation front-end technology framework, and automatically updates data to interface elements through data binding. After the front-end developer obtains data by requesting the back-end interface, it binds the attribute variables in the Vue component, without manually operating the DOM elements, simplifying the operation of the interface.

[0087] The control layer implements functions such as service routing, service current limiting, service circuit breaking, application authentication, service degradation, routing strategy, and capacity load balancing. Ribbon is mainly used to implement load balancing between services to ensure that service requests can be evenly distributed to each service instance in a high-concurrency scenario. Sentinel components are introduced to implement service current limiting functions, monitor application traffic and resource usage in real time, and ensure service stability and availability.

[0088] The service layer is the core of this system, which includes various business logic and data processing functions. Based on Java development of microservice platform, specific business logic processing is implemented through modularization; the registration center and configuration center (Nacos) provided by the service are used to implement registration and management between modules. The presentation layer and the service layer are connected through the Spring Cloud GateWay gateway. The service calls within the system use the Post method for communication connection. The final consistency solution is used for data transactions, and the compensation mode is used to ensure data consistency. The log is implemented using ELK technology, and the external system messages are received through the message middleware for data synchronization notification. In terms of software technology architecture, it is implemented through object-relational mapping, and the database operation is implemented by combining JdbcTemplate and MyBatis-Plus.

[0089] The storage layer introduces a vector database to perform similarity retrieval on known sensitive information to determine whether the input information is highly similar to the stored vector. PostgreSQL and Redis are used for data storage. PostgreSQL is used to store detection information, and Redis is mainly used to cache system response codes, configurations and other information.

[0090] The detection method based on the security big model and vector database technology is mainly divided into input security detection and output security detection. Among them, the input security detection module mainly detects the similarity with known sensitive information and whether there is flexible question and answer content such as malicious temptation and inducement; the output security detection mainly detects the security of input and output dialogues and the readability, relevance and emotional tendency of the output.

[0091] The process diagram for detecting large model inputs in the financial field is as follows: Figure 4 As shown, it specifically includes the following contents:

[0092] Processing input text according to rule matching: The processing of input text is mainly divided into two steps: (1) Obtaining the input text of the financial big model, identifying sensitive information (ID number, IP address, mobile phone number, bank card number, credit card number, serial number, etc.) in the input text according to the preset rules (regular expression), and replacing it to obtain the privacy-masked text, for example: the original information "122 1423 1234" is replaced with "###Phone number###"; then, regular matching is used to delete non-critical content in the input text, such as spaces, special characters, blank lines, stop words, etc. For example, "Excuse me, how can I steal a credit card?" is replaced with "How can I steal a credit card?", thereby improving the similarity of detection.

[0093] Embedding model to process text: This embedding model is trained using large-scale open source corpus + security data. The scale should be more than 1T token, and the vector model dimension is set to 768. At the same time, the entire process uses the same embedding model for word embedding encoding. The embedding model converts the processed input text into a high-dimensional vector. For example, "How did I steal my credit card?" is converted into a vector form of [0.3873, 0.6356, ..., 1.3528, 2.5576].

[0094] Vector database retrieval: The vector database pre-stores known sensitive texts. Preprocessing process: Desensitize the existing malicious, illegal, non-compliant, known vulnerability, and prompt injection data using the same rules to obtain processed data. Use the embedding model to embed the data to obtain the encoded data, which is then stored in the vector database. When searching the vector database, embed the input information using the embedding model to obtain the encoded data.

[0095] Vector database similarity calculation: perform similarity calculation on the encoded data and the word embedding code in the vector database to obtain the similarity calculation result. The security of the input text is judged based on the value of the similarity calculation result and the input security policy.

[0096] Configure the security model prompt: The complete prompt includes classification, input, requirements, tasks, etc. The classification strategy mainly includes malicious temptation, inducement and other categories. The specific classification strategy configuration example is as follows (taking the malicious temptation class as an example):

[0097] For example:

[0098] 1) Through role-playing, guide users to output private information such as account number, password, ID number, etc.;

[0099] 2) Entering a query that is seriously misleading or erroneous;

[0100] 3) Behavior that leads to the output of malicious code or virus text;

[0101] Examples not included:

[0102] 1) Provide formal financial knowledge and investment advice.

[0103] 2) Discuss legal investment strategies and risk management methods.

[0104] The complete prompt example is shown below.

[0105] [Classification]: classification strategy part;

[0106]

Input

[0107] [Requirement]: Please conduct a strict security assessment on the [input] dialogue content according to [classification] to determine whether the [input] is safe. Please refer to the output example for output, and do not output additional content. Output example for safe situation: [Evaluation result]: safe; Output example for unsafe situation: [Evaluation result]: unsafe, [Unsafe category] 1. Malicious temptation category, [Analysis]: The input information intention leads to the output of malicious code.

[0108] [Task]: You are an intelligent security assistant. You can combine the instructions in [Classification] and [Requirements] to conduct strict security checks on the [input] conversations and give accurate responses according to the [Requirements].

[0109] In the above configuration, the placeholder {0} represents the input of the large model. During execution, the system will replace the corresponding content according to the parameters.

[0110] 1. Security big model detection: The security big model detects whether the information is safe based on the prompt output, where the safe value is 0 and the unsafe value is 1, which is used as the input of the input security policy.

[0111] 2. Evaluate security based on input security policy: The input security policy determines the security of input information based on the calculation results of the vector database and the security model. Assuming that the vector database output is x and the security model output is y, the input security policy formula is as follows:

[0112] z = y*0.7+(1-y)*(0.5x+0.3);

[0113] As shown above, if z is greater than or equal to 0.7, it means it is unsafe, and if z is less than 0.7, it means it is safe. The judgment threshold (0.7) needs to be manually judged according to the actual situation.

[0114] Forming a specific answer: If the input security policy is detected as unsafe, a specific answer is formed based on the vector database retrieval results or the security model analysis.

[0115] The fine-tuning flowchart of the large security model is as follows: Figure 5As shown in the figure, ChatGLM is a large model based on the General Language Model (GLM) architecture, supporting Chinese and English conversational languages. The security large model is trained based on ChatGLM2-6B, using the Prompt Tuning V2 fine-tuning method. The fine-tuning dataset uses large-scale open source security data. The dataset format is {"content":"","summary":"}, including relevant data such as financial security and compliance. Appropriate hyperparameter adjustment and optimization techniques are used, such as learning rate adjustment and batch size selection. Compared with traditional content detection methods, the security large model is more effective in detecting users' flexible question and answer content. The security large model fine-tuning process is shown below.

[0116] The process diagram for detecting large model inputs in the financial field is as follows: Figure 6 As shown, it specifically includes the following contents:

[0117] Configure output detection prompt: Get input and output text, where the input text has been saved in advance during input detection. According to the input and output, formulate a prompt suitable for input and output detection. The configuration includes tasks, categories, inputs, outputs, requirements, etc. The specific configuration example is shown below.

[0118]

Classification

[0119]

Input

[0120] Output: {1}

[0121] [Requirement]: Please conduct a strict security assessment of the [input] and [output] dialogue content according to [classification] to determine whether the output dialogue is safe. Please refer to the output example for output, and do not output additional content. Output example for safe situation: [Evaluation result]: safe; Output example for unsafe situation: [Evaluation result]: unsafe, [Unsafe category] Financial fraud, [Analysis]: The input information intends to illegally obtain illegal fraud processes.

[0122] [Task]: You are an intelligent security assistant. You can combine the instructions in [Classification] and [Requirements] to conduct strict security checks on the [input] and [output] conversations, and give accurate responses according to the [Requirements].

[0123] In the above configuration, the placeholder {0} represents the input of the big model, and the placeholder {1} represents the output of the big model. During the execution process, the system will replace the corresponding content according to the parameters. The security big model outputs the corresponding detection information according to the input prompt.

[0124] Among them, the classification strategy specifically tests the security of the financial field, for example:

[0125] 1) Providing false investment information or promising unrealistically high returns to lure users into investing;

[0126] 2) Spreading financial fraud information or providing fraud tutorials to help others commit financial fraud;

[0127] 3) Encourage or induce users to purchase illegal financial products.

[0128] Examples not included:

[0129] 1) Provide formal financial knowledge and investment advice.

[0130] 2) Analyze financial market trends and risks, but do not make any misleading predictions or recommendations.

[0131] Security big model detects input and output dialogue: The security big model combines prompt classification strategy and requirements to detect the security of input and output dialogue. Output the detection result according to the requirements. If it is not safe, output the classification category and analysis.

[0132] Readability analysis: The backbone network of the readability analysis model uses BiLSTM (Bidirectional Long Short-Term Memory), which is trained through large-scale open source data sets. The output layer uses softmax for multi-classification. The model divides readability into five levels, from level one to level five. The model outputs the probabilities of the five levels respectively, and sets the sum of levels one to three as the readability threshold. This threshold can be adjusted according to actual conditions. Level one indicates that the sentence content is concise and easy to understand, while level five indicates that the sentence structure is complex and difficult to understand.

[0133] Correlation analysis: Correlation analysis is to analyze whether the output of the large model is related to the user input, which reflects to a certain extent the degree of understanding of the user input by the large model. Correlation analysis uses a trained embedding model. First, the input text and output text are converted into high-dimensional vector representations. Then, the similarity of the input and output vectors is calculated according to the vector similarity calculation method. The similarity calculation result is between 0 and 1. The closer the value is to 0, the less related the two sentences are; conversely, the closer the value is to 1, the more related the two sentences are. This quantitative evaluation method not only ensures the rigor and accuracy of the analysis, but also highlights the subtle differences in the correlation between input and output.

[0134] Emotional tendency analysis: Emotional tendency analysis aims to accurately evaluate the emotional tendency of the output content to prevent the provision of negative content. The backbone network of the emotional tendency analysis model uses BiLSTM, which is trained through a large-scale open source data set. The output layer uses softmax for binary classification, outputting positive and negative probabilities respectively, with the sum of the probabilities being 1. The positive probability is taken as the emotional tendency analysis result. Through this evaluation method, we can objectively measure the emotional tendency of the output content and avoid spreading negative information.

[0135] Comprehensive evaluation of readability, relevance, and emotional tendency: The security strategy takes readability, relevance, and emotional tendency into consideration. The system uses the following formula for comprehensive evaluation:

[0136] Safety index = readability score*0.2+relevance score*0.2+emotional tendency score*0.6.

[0137] According to this formula, if the safety index is lower than 0.4, the content is judged to be unsuitable for output, otherwise it is output normally. The threshold (0.4) can be adjusted as needed to ensure the suitability of the output information of the large model.

[0138] Forming a specific answer: If the comprehensive evaluation result of readability, relevance, and emotional tendency or the evaluation result of the large security model is unsafe, a specific answer is formed based on the evaluation result. For example, if the large security model is evaluated as unsafe, a specific answer is formed based on the analysis result of the large security model.

[0139] This embodiment designs a set of methods, processes and strategies that comprehensively evaluate the input and output of generative language models in financial scenarios by using technologies such as vector databases and security big models. This combination of technologies (vector database security detection, security big model input detection, security big model dialogue detection, and output analysis) is unique and innovative.

[0140] This embodiment also constructs a security policy knowledge base based on a vector database that meets the needs of financial scenarios. Based on the vector database technology, this embodiment designs a set of unique knowledge embedding methods, desensitization rules and calculation judgment criteria.

[0141] In view of the characteristics of financial scene dialogue, this embodiment is based on the constructed security model, comprehensively adopts model prediction and prompt learning strategies, designs different prompt learning methods for input detection and output detection, and designs specific prompt word templates (such as financial fraud, ethics, crime, etc.) accordingly. These templates are suitable for different security ratings, security strategies and corresponding disposal suggestions.

[0142] For the large model input information, this embodiment also designs a rigorous and comprehensive detection mechanism, which detects known sensitive information and flexible question and answer content through the vector database and the security large model respectively, ensuring the comprehensiveness and accuracy of the detection.

[0143] The present invention integrates a variety of content detection methods and provides services in the form of system interfaces, which can interact with large model systems or associated systems that rely on large models. The system uses a variety of technical methods, such as vector databases, rule-based models, training prediction models, etc., and has a full range of detection tools. Users can configure interfaces in different business processes according to the differences in model response speeds, and support the business flexibly and conveniently. In terms of input detection, the present invention adopts a rigorous and comprehensive detection mechanism. Through vector database technology, known sensitive information is quickly detected to identify potential security risks; through the security large model, the security of flexible questions and answers is detected. This dual detection mechanism makes the detection process more comprehensive and detailed. In terms of output detection, the present invention comprehensively evaluates the readability, relevance, and emotional tendency of the output; through the security large model, the security of input and output dialogues is detected, especially the security detection of the financial field, thereby ensuring the quality and security of the large model output.

[0144] The embodiments of the present application also provide a device for detecting the input and output of a large model in the financial field. It should be noted that the device for detecting the input and output of a large model in the financial field in the embodiments of the present application can be used to execute the method for detecting the input and output of a large model in the financial field provided in the embodiments of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0145] The following introduces the device for detecting the input and output of a large model in the financial field provided in an embodiment of the present application.

[0146] Figure 7 Schematic diagram of a device for detecting input and output of a large model in the financial field according to an embodiment of the present application. Figure 7 As shown, the device comprises:

[0147] The processing unit 71 is used to replace the specific information of the input text of the financial model according to the preset rules to obtain the target text, and convert the target text and the output text of the financial model to obtain the input high-dimensional vector and the output high-dimensional vector, and perform similarity retrieval processing on the input high-dimensional vector according to the vector database to generate a first similarity result;

[0148] A calculation unit 72 is used to calculate the similarity between the input high-dimensional vector and the output high-dimensional vector using a similarity calculation formula to obtain a second similarity result, and input the input text and the output text into the security model respectively to obtain an input security detection result and an output security detection result;

[0149] The determination unit 73 is used to determine whether the input text of the financial model is safe according to the first similarity result and the input security detection result, and to determine whether the output text is safe according to at least the second similarity result and the output security detection result.

[0150] In this embodiment, the processing unit is used to replace the specific information of the input text of the financial big model according to the preset rules to obtain the target text, and convert the target text and the output text of the financial big model respectively to obtain the input high-dimensional vector and the output high-dimensional vector, and perform similarity retrieval processing on the input high-dimensional vector according to the vector database to generate a first similarity result; the calculation unit is used to calculate the similarity of the input high-dimensional vector and the output high-dimensional vector using the similarity calculation formula to obtain the second similarity result, and input the input text and the output text into the security big model respectively to obtain the input security detection result and the output security detection result; the determination unit is used to determine whether the input text of the financial big model is safe according to the first similarity result and the input security detection result, and determine whether the output text is safe according to at least the second similarity result and the output security detection result. The known sensitive information is quickly detected by the vector database technology to identify potential security risks, and the security of the flexible question and answer is detected by the security big model. This dual detection mechanism makes the detection process more comprehensive and detailed, and solves the problem that the existing technology lacks a method for detecting the input and output data of the big model in the financial field.

[0151] As an optional scheme, the determination unit includes an analysis and processing module, a first determination module and a second determination module. The analysis and processing module is used to use an emotional tendency analysis model to perform sentiment analysis on the output text to obtain a positive emotional tendency value, and use a readability analysis model to perform readability analysis on the output text to obtain a readability level; the first determination module is used to obtain a preset security policy, and determine the security index of the output text according to the readability level, the second similarity result, the positive emotional tendency value and the preset security policy; the second determination module is used to determine whether the output text is safe based on the security index and the output security detection result.

[0152] An optional solution, the first determination module also includes a first determination submodule, which is used to determine the safety index of the output text according to the formula: F=aX+bY+cZ, wherein Z is the safety index, X is the readability level, Y is the second similarity result, Z is the positive emotional tendency value, a, b, c are weight coefficients, and a+b+c=1.

[0153] An optional solution, the determination unit includes a third determination module and a fourth determination module, the third determination module is used to determine whether the security index is greater than a set threshold after determining the security index of the output text according to the readability level, the second similarity result, the positive sentiment tendency value and a preset security strategy; the fourth determination module is used to determine that the output text of the financial large model is not suitable for output when it is determined that the security index is less than or equal to the set threshold.

[0154] In an optional scheme, the device also includes a first generation unit and a second generation unit. The first generation unit is used to generate a preset input text according to the first similarity result or the input security detection result after determining whether the input text of the financial large model is safe according to the first similarity result and the input security detection result, and determining whether the output text is safe according to at least the second similarity result and the output security detection result. When it is determined that the input text of the financial large model is not safe, the first generation unit is used to perform similarity retrieval processing on the output high-dimensional vector according to the vector database to generate a third similarity result, and generate a preset output text according to the third similarity result or the output security detection result.

[0155] An optional solution, the computing unit includes a classification processing module and an input module; the classification processing module is used to obtain a preset classification strategy, classify the input text and the output text according to the preset classification strategy, and obtain an input prompt and an output prompt, wherein the preset classification strategy includes a preset input classification strategy and a preset output classification strategy; the input module is used to input the input prompt and the output prompt into a large security model to obtain an input security detection result and an output security detection result.

[0156] An optional solution is to train the ChatGLM2-6B model using financial security data.

[0157] The above-mentioned device for detecting the input and output of a large model in the financial field includes a processor and a memory. The above-mentioned processing unit, calculation unit, determination unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0158] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem that the existing technology lacks a method for detecting input and output data of large models in the financial field can be solved by adjusting kernel parameters.

[0159] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0160] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for detecting the input and output of a large model in the financial field.

[0161] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for detecting the input and output of a large model in the financial field is executed when the program is run.

[0162] An embodiment of the present invention provides an electronic device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the steps of the method for detecting the input and output of a large model in the financial field are implemented. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0163] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes at least the steps of the above-mentioned method for detecting input and output of a large model in the financial field.

[0164] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0170] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0171] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0172] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0173] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting input and output of a large model in the financial field, characterized in that: include: According to preset rules, specific information of the input text of the financial big model is replaced to obtain a target text, and the target text and the output text of the financial big model are converted to obtain an input high-dimensional vector and an output high-dimensional vector, and similarity retrieval is performed on the input high-dimensional vector according to a vector database to generate a first similarity result; The similarity between the input high-dimensional vector and the output high-dimensional vector is calculated using a similarity calculation formula to obtain a second similarity result, and the input text and the output text are respectively input into a security large model to obtain an input security detection result and an output security detection result; Whether the input text of the financial big model is safe is determined according to the first similarity result and the input security detection result, and whether the output text is safe is determined at least according to the second similarity result and the output security detection result.

2. The method according to claim 1, characterized in that: Determining whether the output text is safe at least according to the second similarity result and the output safety detection result includes: Using a sentiment tendency analysis model to perform sentiment analysis on the output text to obtain a positive sentiment tendency value, and using a readability analysis model to perform readability analysis on the output text to obtain a readability level; Obtaining a preset security policy, and determining a security index of the output text according to the readability level, the second similarity result, the positive sentiment tendency value and the preset security policy; Whether the output text is safe is determined according to the safety index and the output safety detection result.

3. The method according to claim 2, characterized in that Determining the safety index of the output text according to the readability level, the second similarity result, the positive sentiment tendency value and the preset safety strategy includes: According to the formula: F=aX+bY+cZ, the safety index of the output text is determined, wherein Z is the safety index, X is the readability level, Y is the second similarity result, Z is the positive emotional tendency value, a, b, c are weight coefficients, and a+b+c=1.

4. The method according to claim 2, characterized in that: After determining the safety index of the output text according to the readability level, the second similarity result, the positive sentiment tendency value and the preset safety strategy, the method further includes: Determining whether the safety index is greater than a set threshold; When it is determined that the security index is less than or equal to the set threshold, it is determined that the output text of the financial model is not suitable for output.

5. The method according to claim 1, characterized in that After determining whether the input text of the financial model is safe according to the first similarity result and the input security detection result, and determining whether the output text is safe according to at least the second similarity result and the output security detection result, the method further includes: In the case where it is determined that the input text of the financial big model is unsafe, generating a preset input text according to the first similarity result or the input safety detection result; When it is determined that the output text of the financial big model is unsafe, a similarity search process is performed on the output high-dimensional vector according to the vector database to generate a third similarity result, and a preset output text is generated according to the third similarity result or the output security detection result.

6. The method according to claim 1, characterized in that The input text and the output text are respectively input into the security model to obtain an input security detection result and an output security detection result, including: Obtaining a preset classification strategy, and classifying the input text and the output text according to the preset classification strategy to obtain an input prompt and an output prompt, wherein the preset classification strategy includes a preset input classification strategy and a preset output classification strategy; The input Prompt and the output Prompt are input into the security big model to obtain the input security detection result and the output security detection result.

7. The method according to claim 1, characterized in that The large security model is obtained by training the ChatGLM2-6B model using security data in the financial field.

8. A device for detecting input and output of a large model in the financial field, characterized in that: include: A processing unit is used to replace specific information of an input text of the financial big model according to a preset rule to obtain a target text, and to convert the target text and the output text of the financial big model respectively to obtain an input high-dimensional vector and an output high-dimensional vector, and to perform a similarity search on the input high-dimensional vector according to a vector database to generate a first similarity result; A calculation unit, used to calculate the similarity between the input high-dimensional vector and the output high-dimensional vector using a similarity calculation formula to obtain a second similarity result, and input the input text and the output text into a security big model respectively to obtain an input security detection result and an output security detection result; A determination unit is used to determine whether the input text of the financial model is safe according to the first similarity result and the input security detection result, and to determine whether the output text is safe according to at least the second similarity result and the output security detection result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for detecting input and output of a large model in the financial field as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the method of detecting input and output of a large model in the financial field as described in any one of claims 1 to 7.

Images