Financial system content query method and system, product, equipment and storage medium

Abstract

The invention discloses a financial system content query method and system, a product, equipment and a storage medium, and relates to the field of data processing, and the method comprises the steps that a preset financial system text query model feeds back a financial system text with the highest semantic matching degree with a financial system query text to a front end, pre-training the initial financial system text query model by using a system file with missing characters as a first training data set of training data, and performing semantic comparative learning based on a second training data set comprising positive sample training data and negative sample training data with different semantics by utilizing the pre-trained initial financial system text query model to obtain a preset financial system text query model. The financial system query text is subjected to semantic recognition based on the preset financial system text query model, the financial system text with the highest semantic matching degree with the financial system query text is extracted, and the query precision of the financial system content is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, system, product, device and storage medium for querying the content of a financial system. Background Art

[0002] Due to the characteristics of the financial industry, a large amount of financial-related institutional content needs to be cited in the process of financial activities. In the existing practice, financial practitioners will use text matching models configured with text matching rules to filter out institutional content with the highest text matching degree based on the query content entered by enterprise personnel.

[0003] However, the existing text matching model based on text matching rules has the problem of low matching accuracy. Specifically, the existing text matching rules are manually extracted from the keyword information in the commonly used system content, and the corresponding text matching rules are configured based on the matching degree between the keyword information and the system content. However, the existing text matching model based on text matching rules only compares based on the text content. When faced with system content with complex semantic relationships or when the query content lacks key characters, the query accuracy of the existing text matching model will be reduced. Therefore, how to improve the query accuracy of financial system content has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above problems, this application provides a method, system, product, device and storage medium for querying the content of financial systems, so as to achieve the purpose of improving the query accuracy of the content of financial systems. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for querying the content of a financial system, comprising:

[0006] The financial system query text input by the front end is input into a preset financial system text query model, so that the preset financial system text query model extracts the financial system text with the highest semantic matching degree with the financial system query text based on the semantic recognition result of the financial system query text, and feeds the financial system text back to the front end, wherein the training process of the preset financial system text query model includes:

[0007] Pre-training an initial financial system text query model using a first training data set, wherein the training data in the first training data set is a system document with missing characters;

[0008] The pre-trained initial financial system text query model is used to perform semantic contrast learning based on a second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model, and the semantics of the positive sample training data and the negative sample training data are different.

[0009] In a possible implementation, the pre-training of the initial financial system text query model using the first training data set includes:

[0010] determining a plurality of types of the system documents as initial training data;

[0011] Performing a character replacement operation on each of the initial training data to obtain the first training data set including each of the training data having the missing characters, and storing a replacement character set for each of the training data respectively, wherein the replacement character set includes each replacement character replaced in the training data and the character position of each replacement character;

[0012] The initial financial system text query model is pre-trained based on the first training data set, and the initial financial system text query model is adjusted based on the replacement character set.

[0013] In a possible implementation, pre-training the initial financial system text query model based on the first training data set, and adjusting parameters of the initial financial system text query model based on the replacement character set, include:

[0014] For each of the training data in the first training data set:

[0015] Input the training data into the initial financial system text query model to obtain a predicted missing character set of the training data output by the initial financial system text query model, wherein the predicted missing character set includes each predicted missing character of the training data and the character position of each predicted missing character;

[0016] Based on the character positions of the replacement characters and the character positions of the predicted missing characters, a character consistency comparison is performed on the predicted missing character set and the replacement character set of the training data, and a parameter adjustment operation is performed on the initial financial system text query model based on the comparison result.

[0017] In a possible implementation, the using the pre-trained initial financial system text query model to perform semantic contrast learning based on a second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model includes:

[0018] Obtaining a plurality of financial system history query texts and their corresponding system file identifiers, and splicing the financial system history query texts and their corresponding system file identifiers to obtain a plurality of basic training data;

[0019] For each of the basic training data: another basic training data having the system file identifier in the basic training data is determined as the positive sample training data, and the positive sample training data has the same semantics as the financial system history query text in the basic training data; the financial system history query text in the basic training data is semantically changed at least once to obtain at least two of the negative sample training data, the negative sample training data has the same system file identifier as the basic training data, and the negative sample training data has different semantics from the financial system history query text in the basic training data; a data set including the basic training data, the positive sample training data and the at least two negative sample training data is determined as a batch training data set corresponding to the basic training data;

[0020] The semantic contrast learning is performed on the pre-trained initial financial system text query model using the second training data set including each of the batch training data sets to obtain the preset financial system text query model, wherein the input of the preset financial system text query model is the financial system query text, and the output is the financial system text with the highest semantic matching degree with the financial system query text.

[0021] In a possible implementation, the using the second training data set including each batch training data set to perform the semantic contrast learning on the pre-trained initial financial system text query model to obtain the preset financial system text query model includes:

[0022] For each batch training data set in the second training data set: the basic training data, the positive sample training data and at least two negative sample training data in the batch training data set are sequentially input into the pre-trained initial financial system text query model, so that the pre-trained initial financial system text query model performs semantic recognition on the basic training data, the positive sample training data and the negative sample training data respectively to obtain a semantic recognition result, wherein the semantic recognition result includes: a first semantic vector of the basic training data and a second semantic vector of the positive sample training data, and a third semantic vector and a fourth semantic vector of the negative sample training data; the semantic result is substituted into a preset loss function to obtain an operation result of the preset loss function;

[0023] When the calculation result meets the preset termination training condition, the preset financial system text query model is obtained.

[0024] In a possible implementation, substituting the semantic result into a preset loss function to obtain a calculation result of the preset loss function includes:

[0025] According to the first semantic vector r1, the second semantic vector r2, the third semantic vector r3 and the fourth semantic vector r4 in the semantic result, by formula:

[0026]

[0027] The operation result L(r1, r2) is obtained, wherein the operation result represents the semantic matching degree between the basic training data and the financial system history query text in the positive sample training data, the sim is a similarity function, and the exp is an exponential function.

[0028] The second aspect of the present application provides a query system for financial system content, including: a query module and a model training module.

[0029] The query module is used to input the financial system query text input by the front end into a preset financial system text query model, so that the preset financial system text query model extracts the financial system text with the highest semantic matching degree with the financial system query text based on the semantic recognition result of the financial system query text, and feeds the financial system text back to the front end.

[0030] Among them, the model training module is used in the training process of the preset financial system text query model: using a first training data set to pre-train the initial financial system text query model, the training data in the first training data set is a system document with missing characters; using the pre-trained initial financial system text query model to perform semantic comparative learning based on a second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model, the positive sample training data and the negative sample training data have different semantics.

[0031] In a possible implementation, the model training module is set to:

[0032] determining a plurality of types of the system documents as initial training data;

[0033] Performing a character replacement operation on each of the initial training data to obtain the first training data set including each of the training data having the missing characters, and storing a replacement character set for each of the training data respectively, wherein the replacement character set includes each replacement character replaced in the training data and the character position of each replacement character;

[0034] The initial financial system text query model is pre-trained based on the first training data set, and the initial financial system text query model is adjusted based on the replacement character set.

[0035] In a possible implementation, when the model training module pre-trains the initial financial system text query model based on the first training data set and performs parameter adjustment operation on the initial financial system text query model based on the replacement character set, the model training module is set to:

[0036] For each of the training data in the first training data set:

[0037] Input the training data into the initial financial system text query model to obtain a predicted missing character set of the training data output by the initial financial system text query model, wherein the predicted missing character set includes each predicted missing character of the training data and the character position of each predicted missing character;

[0038] Based on the character positions of the replacement characters and the character positions of the predicted missing characters, a character consistency comparison is performed on the predicted missing character set and the replacement character set of the training data, and a parameter adjustment operation is performed on the initial financial system text query model based on the comparison result.

[0039] In a possible implementation, the model training module is configured to:

[0040] Obtaining a plurality of financial system history query texts and their corresponding system file identifiers, and splicing the financial system history query texts and their corresponding system file identifiers to obtain a plurality of basic training data;

[0041] For each of the basic training data: another basic training data having the system file identifier in the basic training data is determined as the positive sample training data, and the positive sample training data has the same semantics as the financial system history query text in the basic training data; the financial system history query text in the basic training data is semantically changed at least once to obtain at least two of the negative sample training data, the negative sample training data has the same system file identifier as the basic training data, and the negative sample training data has different semantics from the financial system history query text in the basic training data; a data set including the basic training data, the positive sample training data and the at least two negative sample training data is determined as a batch training data set corresponding to the basic training data;

[0042] The semantic contrast learning is performed on the pre-trained initial financial system text query model using the second training data set including each of the batch training data sets to obtain the preset financial system text query model, wherein the input of the preset financial system text query model is the financial system query text, and the output is the financial system text with the highest semantic matching degree with the financial system query text.

[0043] In a possible implementation, when the model training module uses the second training data set including each of the batch training data sets to perform the semantic contrast learning on the pre-trained initial financial system text query model to obtain the preset financial system text query model, the model training module is set to:

[0044] For each batch training data set in the second training data set: the basic training data, the positive sample training data and at least two negative sample training data in the batch training data set are sequentially input into the pre-trained initial financial system text query model, so that the pre-trained initial financial system text query model performs semantic recognition on the basic training data, the positive sample training data and the negative sample training data respectively to obtain a semantic recognition result, wherein the semantic recognition result includes: a first semantic vector of the basic training data and a second semantic vector of the positive sample training data, and a third semantic vector and a fourth semantic vector of the negative sample training data; the semantic result is substituted into a preset loss function to obtain an operation result of the preset loss function;

[0045] When the calculation result meets the preset termination training condition, the preset financial system text query model is obtained.

[0046] In a possible implementation, when substituting the semantic result into a preset loss function to obtain the operation result of the preset loss function, the model training module is set to:

[0047] According to the first semantic vector r1, the second semantic vector r2, the third semantic vector r3 and the fourth semantic vector r4 in the semantic result, by formula:

[0048]

[0049] The operation result L(r1, r2) is obtained, wherein the operation result represents the semantic matching degree between the basic training data and the financial system history query text in the positive sample training data, the sim is a similarity function, and the exp is an exponential function.

[0050] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for querying the content of financial systems according to the first aspect or any implementation of the first aspect.

[0051] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0052] The memory is used to store computer programs;

[0053] The processor is used to execute the computer program so that the electronic device can implement the method for querying the content of financial systems in the above-mentioned first aspect or any implementation manner of the first aspect.

[0054] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can query the content of the financial system according to the first aspect or any implementation method of the first aspect.

[0055] By means of the above technical scheme, the query method, system, product, device and storage medium of the financial system content provided by the present application pre-trains the initial financial system text query model by configuring the first training data set using the system file with missing characters as the training data, so that when facing the query content with missing key characters, the preset financial system text can supplement the missing characters based on the context of the financial system query text, thereby avoiding the risk of reduced query accuracy due to the missing key characters in the query content in the prior art. Subsequently, by configuring the pre-trained initial financial system text query model to perform semantic comparison learning based on the second training data set including positive sample training data and negative sample training data with different semantics, a preset financial system text query model is obtained, and the preset financial system model is configured to query the financial system text with the highest matching degree with the financial system query text based on the semantic recognition result of the financial system query text, so that the present application realizes the recognition of complex semantic relationships compared with the existing text matching model based on text content, and avoids the risk of reduced query accuracy in the prior art when facing system content with complex semantic relationships. It can be seen that the present application improves the query accuracy of financial system content. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0057] Figure 1 A schematic diagram of a system architecture provided for this application;

[0058] Figure 2 A schematic diagram of the hardware structure of a terminal provided in this application;

[0059] Figure 3 A schematic diagram of the structure of a server provided for this application;

[0060] Figure 4 A flowchart of a method for querying the content of a financial system provided for this application;

[0061] Figure 5 A schematic diagram of a process for pre-training an initial financial system text query model provided for this application;

[0062] Figure 6 A flowchart of semantic contrastive learning of a pre-trained initial financial system text query model provided for this application;

[0063] Figure 7 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0064] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0065] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0066] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0067] See also Figure 1 , Figure 1A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0068] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters by itself without the cooperation of the server, and the embodiments of the present application are not limited to this.

[0069] Among them, the terminal 100 can be installed with an application for executing a method for querying the content of a financial system provided in the first aspect of the present application. The above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the interface and send the above parameters to the server 200. The server 200 can obtain processing results based on the received parameters and return the processing results to the terminal 100.

[0070] Next describe Figure 1 The product form of the mid-terminal 100;

[0071] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0072] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0073] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0074] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0075] Among them, the input device 132 can receive input data and the like.

[0076] The display unit 140 may be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file displays, and / or playback of any multimedia files. In the embodiment of the present application, the display unit 140 may be used to display the interface of an application program for executing the method for querying the content of financial regulations, processing results, etc.

[0077] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0078] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0079] Among them, the memory 120 can be used to store software codes related to the query method of financial system content, the processor 170 can execute the steps of the query method of financial system content, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to realize corresponding functions.

[0080] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0081] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0082] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0083] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0084] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0085] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0086] Next describe Figure 1 The product form of the server 200;

[0087] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0088] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0089] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0090] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0091] The memory 204 may be used to store software codes related to the method for querying the content of financial regulations, and the processor 202 may execute the steps of the method for querying the content of financial regulations of the chip, and may also schedule other units to realize corresponding functions.

[0092] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0093] The first aspect of the present application provides a method for querying the content of a financial system, such as Figure 4 As shown, the query methods for the content of the financial system include:

[0094] S401. Input the financial system query text input by the front end into a preset financial system text query model, so that the preset financial system text query model extracts the financial system text with the highest semantic matching degree with the financial system query text based on the semantic recognition result of the financial system query text, and feeds the financial system text back to the front end, wherein the training process of the preset financial system text query model includes: pre-training the initial financial system text query model using a first training data set, the training data in the first training data set is the system file with missing characters; using the pre-trained initial financial system text query model to perform semantic comparison learning based on a second training data set including positive sample training data and negative sample training data, to obtain a preset financial system text query model, the semantics of the positive sample training data and the negative sample training data are different.

[0095] It should be noted that in actual application scenarios, the above-mentioned financial system query text can be a text string generated by financial practitioners that is related to the content of the financial system and has query semantics, such as: "What are the employee work behavior rules", "What rules should employees follow at work", "What are the confidentiality codes", etc.

[0096] It should be noted that in actual application scenarios, the above-mentioned preset financial system text query model can be constructed based on an unsupervised training language model (Bidirectional Encoder Representations from Transformers, BERT) for natural language processing tasks. Since the financial system query text input by the front-end is manually generated by financial practitioners according to their own habits and actual business needs, there may be gaps or interruptions in the text content related to the financial system content, which makes the existing formal comparison method based on text content unable to accurately identify the implicit semantics in the financial system query text, thereby reducing the query accuracy of the existing method. The BERT model introduces the attention mechanism to capture long-distance dependencies in the text, thereby realizing the semantic capture of the financial system query text, thereby improving the query accuracy.

[0097] It should be noted that in actual application scenarios, since the above-mentioned financial system query text is manually generated by financial practitioners according to their own habits, in order to facilitate daily work management and communication, the financial practitioners are accustomed to using general descriptions instead of the proprietary names of standard financial systems. For example: the financial system with the proprietary name "Bank Attendance System Guidelines" is called "Attendance System" by financial practitioners. If the existing text content-based comparison method is adopted, the "Attendance System" and the "Bank Attendance System Guidelines" are very likely to fail to hit due to the large text differences, thereby reducing the query accuracy. However, the present application configures the initial financial system text query model to be pre-trained by using the first training data set with the system file with missing characters as training data, so that the preset financial system text query model can supplement the missing text in the financial system query text, thereby improving the query accuracy when performing financial system content query based on the financial system query text with text missing.

[0098] It should be noted that in actual application scenarios, contrastive learning is a self-supervised learning method based on similarities and differences between samples. Because for the same financial system text, different financial practitioners will generate financial system query texts with the same semantics but different text forms. For example, the content of the financial system query text generated by person A is "What are the employee's work code of conduct", and the content of the financial system query text generated by person B is "What behavioral norms should employees abide by during work." It can be seen that the semantics of both are "query employee code of conduct", but the text representation is quite different. Therefore, this application configures and utilizes a pre-trained initial financial system text query model to perform semantic contrastive learning based on a second training data set including positive sample training data and negative sample training data with different semantics, so that the preset financial system text query model is obtained to improve the semantic recognition accuracy of financial system query texts with complex semantic relationships, thereby improving the query accuracy.

[0099] The present application performs pre-training on the initial financial system text query model by configuring a first training data set that uses system files with missing characters as training data, so that when faced with query content that lacks key characters, the preset financial system text can supplement the missing characters based on the context of the financial system query text, thereby avoiding the risk of reduced query accuracy due to missing key characters in the query content in the prior art. Subsequently, by configuring the pre-trained initial financial system text query model to perform semantic comparison learning based on a second training data set including semantically different positive sample training data and negative sample training data, a preset financial system text query model is obtained, and the preset financial system model is configured to query the financial system text with the highest matching degree with the financial system query text through the semantic recognition result of the financial system query text, so that the present application realizes the recognition of complex semantic relationships compared with the existing text matching model based on text content, avoiding the risk of reduced query accuracy in the prior art when faced with system content with complex semantic relationships. It can be seen that the present application improves the query accuracy of financial system content.

[0100] In a possible implementation, pre-training the initial financial system text query model using the first training data set includes:

[0101] Identify various types of institutional documents as initial training data;

[0102] Performing a character replacement operation on each initial training data to obtain a first training data set including each training data having missing characters, and storing a replacement character set for each training data respectively, wherein the replacement character set includes each replacement character replaced in the training data and the character position of each replacement character;

[0103] The initial financial system text query model is pre-trained based on the first training data set, and the initial financial system text query model is adjusted based on the replacement character set.

[0104] It should be noted that, in actual application scenarios, the above-mentioned system file is a complete system file including the system file name and the system file text. Since the text in the system file includes various interpretation clauses and details of the system file, and these interpretation clauses and details store semantic information that is highly associated with the system file name. Therefore, the present application configures the initial training data to perform a character replacement operation, uses the first training data set including each training data with missing characters to pre-train the initial financial system text query model, and adjusts the parameters of the initial financial system text query model based on the replacement character set, so that the pre-trained initial financial system text query model can supplement the missing characters based on the contextual information in the training data with missing characters, thereby improving the query accuracy of the preset financial system text query model when performing financial system content query based on the financial system query text with text missing.

[0105] It should be noted that, in actual application scenarios, there are many implementation methods for the above-mentioned character replacement operation, and an exemplary implementation method is provided here, including the following steps A1 to A2.

[0106] Step A1, import the system file to be replaced and the character set to be replaced into a preset character replacement script, and trigger step A2.

[0107] Step A2, the preset character replacement script calls the string replacement function, replaces the characters to be replaced in the character set to be replaced in the system file to be replaced with blank characters, obtains training data with missing characters, and stores each character to be replaced and its corresponding position information into the initial replacement character set to obtain the replacement character set.

[0108] In a possible implementation, the string replacement function in the above step A2 can be determined based on the programming language used by the above preset character replacement script. For example, if the preset character replacement script is compiled in Python or Java, the string replacement function can be a string replacement function (Replace) provided by Python or Java. If the preset character replacement script is compiled in C language, the above string replacement function can be implemented by regular expressions.

[0109] In a possible implementation, the position information corresponding to the character to be replaced in the above step A2 may include the row number and column number of the character to be replaced in the system file to be replaced.

[0110] In a possible implementation, pre-training the initial financial system text query model based on the first training data set, and adjusting the parameters of the initial financial system text query model based on the replacement character set include:

[0111] For each training data in the first training data set:

[0112] The training data is input into an initial financial system text query model to obtain a predicted missing character set of the training data output by the initial financial system text query model, wherein the predicted missing character set includes each predicted missing character of the training data and a character position of each predicted missing character;

[0113] Based on the character positions of the replacement characters and the character positions of the predicted missing characters, a character consistency comparison is performed on the predicted missing character set and the replacement character set of the training data, and the parameters of the initial financial system text query model are adjusted based on the comparison results.

[0114] In order to facilitate the understanding of the above-mentioned pre-training process of the initial financial system text query model based on the first training data set, a possible implementation of the present application is specifically described here:

[0115] like Figure 5 Shown is a schematic diagram of the process of pre-training the initial financial system text query model.

[0116] Assume that the training data content is "Bank Attendance System M1 Standard M2". The replacement character set of the training data includes: "M1, degree, 1 row and 6 columns" and "M2, then, 1 row and 8 columns".

[0117] The training data is imported into a preset encoder to convert each text character in the training data into a corresponding text vector. Each text vector is then input into an initial financial system text query model (BERT) so that the initial financial system text query model outputs a character vector of a predicted missing character and a character position of the predicted missing character.

[0118] It should be noted that the above M1 and M2 are only for explanation. In actual application scenarios, the characters used to replace the missing characters can be spaces or numeric characters.

[0119] Subsequently, by traversing the preset mapping relationship table, the characters in the preset mapping relationship table that have a mapping relationship with the character vector of the predicted missing character are determined as predicted missing characters ("M1=degree" and "M2=then"), and a correspondence between the preset missing characters and the character positions is established.

[0120] After the initial financial system text query model completes one training, the predicted missing character set and the replacement character set of the training data are compared for character consistency, and the initial financial system text query model is adjusted based on the comparison results.

[0121] If the consistency comparison of multiple consecutive training data is passed, it is determined that the output accuracy of the initial financial system text query model meets the preset requirements, and the pre-training of the initial financial system text query model is stopped.

[0122] In a possible implementation, a pre-trained initial financial system text query model is used to perform semantic contrast learning based on a second training data set including positive sample training data and negative sample training data to obtain a preset financial system text query model, including:

[0123] Obtain multiple financial system historical query texts and their corresponding system file identifiers, and splice the financial system historical query texts and their corresponding system file identifiers to obtain multiple basic training data;

[0124] For each basic training data: another basic training data having the system file identifier in the basic training data is determined as positive sample training data, the positive sample training data has the same semantics as the financial system history query text in the basic training data; the financial system history query text in the basic training data is semantically changed at least once to obtain at least two negative sample training data, the negative sample training data has the same system file identifier as the basic training data, and the negative sample training data has different semantics from the financial system history query text in the basic training data; a data set including the basic training data, the positive sample training data and the at least two negative sample training data is determined as a batch training data set corresponding to the basic training data;

[0125] A second training data set including each batch of training data sets is used to perform semantic comparative learning on the pre-trained initial financial system text query model to obtain a preset financial system text query model. The input of the preset financial system text query model is the financial system query text, and the output is the financial system text with the highest semantic matching degree with the financial system query text.

[0126] In a possible implementation, the batch training data set may be as shown in Table 1 below.

[0127] Table 1

[0128] Basic training data Positive sample training data What are the system document identification + employee work behavior rules? What are the system document identification + employee work behavior rules? Negative sample training data Negative sample training data System document identification + Can employees be late for work? What are the system document identification + confidentiality rules?

[0129] In a possible implementation, a second training data set including each batch of training data sets is used to perform semantic contrast learning on a pre-trained initial financial system text query model to obtain a preset financial system text query model, including:

[0130] For each batch of training data sets in the second training data set: the basic training data, positive sample training data and at least two negative sample training data in the batch of training data sets are sequentially input into the pre-trained initial financial system text query model, so that the pre-trained initial financial system text query model performs semantic recognition on the basic training data, the positive sample training data and the negative sample training data respectively to obtain a semantic recognition result, the semantic recognition result includes: a first semantic vector of the basic training data and a second semantic vector of the positive sample training data, and a third semantic vector and a fourth semantic vector of the negative sample training data; the semantic result is substituted into a preset loss function to obtain an operation result of the preset loss function;

[0131] When the calculation result meets the preset termination training condition, a preset financial system text query model is obtained.

[0132] In order to facilitate the understanding of the semantic contrast learning process of the pre-trained initial financial system text query model using the batch training data set, a possible implementation of the present application is described here:

[0133] like Figure 6 Shown is a flowchart of semantic contrastive learning of the pre-trained initial financial system text query model.

[0134] The basic training data (Text B ), positive sample training data (Text P ), negative sample training data 1 (Text N1 ) and negative sample training data 2 (Text N2 ) are sequentially input into the pre-trained initial financial system text query model (ZBERT) so that ZBERT can B 、Text P 、Text N1 and Text N2 Perform semantic contrast learning and output the primary representation vector of each training data in the batch training data set to the pooling layer (Pooler).

[0135] Pooler pools each primary representation vector to obtain the first semantic vector (R1) of the basic training data, the second semantic vector (R2) of the positive sample training data, the third semantic vector (R3) of the negative sample training data 1, and the fourth semantic vector (R4) of the negative sample training data 2.

[0136] Then, R1 to R4 are substituted into the preset loss function to obtain the calculation result of the preset loss function.

[0137] It should be noted that, in actual application scenarios, the input of the above-mentioned preset financial system text query model is the financial system query text, and the output is the financial system text.

[0138] In a possible implementation, substituting the semantic result into a preset loss function to obtain the operation result of the preset loss function includes:

[0139] According to the first semantic vector r1, the second semantic vector r2, the third semantic vector r3 and the fourth semantic vector r4 in the semantic result, the formula:

[0140]

[0141] The operation result L(r1, r2) is obtained, where the operation result represents the semantic matching degree between the basic training data and the financial system history query text in the positive sample training data, sim is the similarity function, and exp is the exponential function.

[0142] The second aspect of the present application provides a financial system content query system, the financial system content query system comprising: a query module and a model training module,

[0143] The query module is used to input the financial system query text input by the front end into the preset financial system text query model, so that the preset financial system text query model can extract the financial system text with the highest semantic matching degree with the financial system query text based on the semantic recognition result of the financial system query text, and feed back the financial system text to the front end.

[0144] Among them, the model training module is used in the training process of the preset financial system text query model: using the first training data set to pre-train the initial financial system text query model, the training data in the first training data set is the system file with missing characters; using the pre-trained initial financial system text query model to perform semantic comparative learning based on the second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model, and the semantics of the positive sample training data and the negative sample training data are different.

[0145] In a possible implementation, the model training module is set to:

[0146] Identify various types of institutional documents as initial training data;

[0147] Performing a character replacement operation on each initial training data to obtain a first training data set including each training data having missing characters, and storing a replacement character set for each training data respectively, wherein the replacement character set includes each replacement character replaced in the training data and the character position of each replacement character;

[0148] The initial financial system text query model is pre-trained based on the first training data set, and the initial financial system text query model is adjusted based on the replacement character set.

[0149] In a possible implementation, the model training module is set to:

[0150] For each training data in the first training data set:

[0151] The training data is input into an initial financial system text query model to obtain a predicted missing character set of the training data output by the initial financial system text query model, wherein the predicted missing character set includes each predicted missing character of the training data and a character position of each predicted missing character;

[0152] Based on the character positions of the replacement characters and the character positions of the predicted missing characters, a character consistency comparison is performed on the predicted missing character set and the replacement character set of the training data, and the parameters of the initial financial system text query model are adjusted based on the comparison results.

[0153] In a possible implementation, the model training module is set to:

[0154] Obtain multiple financial system historical query texts and their corresponding system file identifiers, and splice the financial system historical query texts and their corresponding system file identifiers to obtain multiple basic training data;

[0155] For each basic training data: another basic training data having the system file identifier in the basic training data is determined as positive sample training data, the positive sample training data has the same semantics as the financial system history query text in the basic training data; the financial system history query text in the basic training data is semantically changed at least once to obtain at least two negative sample training data, the negative sample training data has the same system file identifier as the basic training data, and the negative sample training data has different semantics from the financial system history query text in the basic training data; a data set including the basic training data, the positive sample training data and the at least two negative sample training data is determined as a batch training data set corresponding to the basic training data;

[0156] A second training data set including each batch of training data sets is used to perform semantic comparative learning on the pre-trained initial financial system text query model to obtain a preset financial system text query model. The input of the preset financial system text query model is the financial system query text, and the output is the financial system text with the highest semantic matching degree with the financial system query text.

[0157] In a possible implementation, the model training module is set to:

[0158] For each batch of training data sets in the second training data set: the basic training data, positive sample training data and at least two negative sample training data in the batch of training data sets are sequentially input into the pre-trained initial financial system text query model, so that the pre-trained initial financial system text query model performs semantic recognition on the basic training data, the positive sample training data and the negative sample training data respectively to obtain a semantic recognition result, the semantic recognition result includes: a first semantic vector of the basic training data and a second semantic vector of the positive sample training data, and a third semantic vector and a fourth semantic vector of the negative sample training data; the semantic result is substituted into a preset loss function to obtain an operation result of the preset loss function;

[0159] When the calculation result meets the preset termination training condition, a preset financial system text query model is obtained.

[0160] In a possible implementation, the above-mentioned model training module is set to:

[0161] According to the first semantic vector r1, the second semantic vector r2, the third semantic vector r3 and the fourth semantic vector r4 in the semantic result, the formula:

[0162]

[0163] The operation result L(r1, r2) is obtained, where the operation result represents the semantic matching degree between the basic training data and the financial system history query text in the positive sample training data, sim is the similarity function, and exp is the exponential function.

[0164] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for querying the content of financial systems according to the first aspect or any implementation of the first aspect.

[0165] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0166] The memory is used to store computer programs;

[0167] The processor is used to execute a computer program so that the electronic device can implement the method for querying the content of a financial system according to the first aspect or any implementation manner of the first aspect.

[0168] The structural diagram of the electronic device provided in the fourth aspect of the present application is as follows Figure 7 The electronic devices in the embodiments of the present application may include but are not limited to fixed terminals such as mobile phones, notebook computers, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, and the like. Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0169] like Figure 7 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0170] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a memory card, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0171] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can query the financial system content of the above-mentioned first aspect or any implementation method of the first aspect.

[0172] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0173] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0174] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0175] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for querying the content of a financial system, characterized in that: include: The financial system query text input by the front end is input into a preset financial system text query model, so that the preset financial system text query model extracts the financial system text with the highest semantic matching degree with the financial system query text based on the semantic recognition result of the financial system query text, and feeds the financial system text back to the front end, wherein the training process of the preset financial system text query model includes: Pre-training an initial financial system text query model using a first training data set, wherein the training data in the first training data set is a system document with missing characters; The pre-trained initial financial system text query model is used to perform semantic contrast learning based on a second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model, and the semantics of the positive sample training data and the negative sample training data are different.

2. The method for querying the content of financial system according to claim 1, characterized in that: The pre-training of the initial financial system text query model using the first training data set includes: determining a plurality of types of the system documents as initial training data; Performing a character replacement operation on each of the initial training data to obtain the first training data set including each of the training data having the missing characters, and storing a replacement character set for each of the training data respectively, wherein the replacement character set includes each replacement character replaced in the training data and the character position of each replacement character; The initial financial system text query model is pre-trained based on the first training data set, and the initial financial system text query model is adjusted based on the replacement character set.

3. The method for querying the content of financial system according to claim 2, characterized in that: The pre-training of the initial financial system text query model based on the first training data set and the parameter adjustment operation of the initial financial system text query model based on the replacement character set include: For each of the training data in the first training data set: Input the training data into the initial financial system text query model to obtain a predicted missing character set of the training data output by the initial financial system text query model, wherein the predicted missing character set includes each predicted missing character of the training data and the character position of each predicted missing character; Based on the character positions of the replacement characters and the character positions of the predicted missing characters, a character consistency comparison is performed on the predicted missing character set and the replacement character set of the training data, and a parameter adjustment operation is performed on the initial financial system text query model based on the comparison result.

4. The method for querying the content of financial system according to claim 1, characterized in that: The method of using the pre-trained initial financial system text query model to perform semantic contrast learning based on a second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model includes: Obtaining a plurality of financial system history query texts and their corresponding system file identifiers, and splicing the financial system history query texts and their corresponding system file identifiers to obtain a plurality of basic training data; For each of the basic training data: another basic training data having the system file identifier in the basic training data is determined as the positive sample training data, and the positive sample training data has the same semantics as the financial system history query text in the basic training data; the financial system history query text in the basic training data is semantically changed at least once to obtain at least two of the negative sample training data, the negative sample training data has the same system file identifier as the basic training data, and the negative sample training data has different semantics from the financial system history query text in the basic training data; a data set including the basic training data, the positive sample training data and the at least two negative sample training data is determined as a batch training data set corresponding to the basic training data; The semantic contrast learning is performed on the pre-trained initial financial system text query model using the second training data set including each of the batch training data sets to obtain the preset financial system text query model, wherein the input of the preset financial system text query model is the financial system query text, and the output is the financial system text with the highest semantic matching degree with the financial system query text.

5. The method for querying the content of financial system according to claim 4, characterized in that: The step of performing the semantic contrast learning on the pre-trained initial financial system text query model using the second training data set including each batch training data set to obtain the preset financial system text query model includes: For each batch training data set in the second training data set: the basic training data, the positive sample training data and at least two negative sample training data in the batch training data set are sequentially input into the pre-trained initial financial system text query model, so that the pre-trained initial financial system text query model performs semantic recognition on the basic training data, the positive sample training data and the negative sample training data respectively to obtain a semantic recognition result, wherein the semantic recognition result includes: a first semantic vector of the basic training data and a second semantic vector of the positive sample training data, and a third semantic vector and a fourth semantic vector of the negative sample training data; the semantic result is substituted into a preset loss function to obtain an operation result of the preset loss function; When the calculation result meets the preset termination training condition, the preset financial system text query model is obtained.

6. The method for querying the content of financial system according to claim 5, characterized in that: Substituting the semantic result into a preset loss function to obtain the operation result of the preset loss function includes: According to the first semantic vector r1, the second semantic vector r2, the third semantic vector r3 and the fourth semantic vector r4 in the semantic result, by formula: The operation result L(r1, r2) is obtained, wherein the operation result represents the semantic matching degree between the basic training data and the financial system history query text in the positive sample training data, the sim is a similarity function, and the exp is an exponential function.

7. A financial system content query system, characterized in that: include: Query module and model training module, The query module is used to input the financial system query text input by the front end into a preset financial system text query model, so that the preset financial system text query model extracts the financial system text with the highest semantic matching degree with the financial system query text based on the semantic recognition result of the financial system query text, and feeds the financial system text back to the front end. Among them, the model training module is used in the training process of the preset financial system text query model: using a first training data set to pre-train the initial financial system text query model, the training data in the first training data set is a system document with missing characters; using the pre-trained initial financial system text query model to perform semantic comparative learning based on a second training data set including positive sample training data and negative sample training data to obtain the preset financial system text query model, the positive sample training data and the negative sample training data have different semantics.

8. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the method for querying the content of a financial system as claimed in any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the method for querying the content of financial regulations as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the method for querying the content of a financial system as described in any one of claims 1 to 6.

Images