Legal document generation method and device, equipment and storage medium
By constructing a model input matrix and dynamically updating the cache, the computational and memory requirements of large language models in legal document generation are optimized, solving the efficiency and real-time issues of long context processing, and achieving efficient and accurate legal document generation.
Patent Information
- Application Number
- CN202511341382.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-19
- Publication Date
- 2025-12-19
AI Technical Summary
Existing large-scale language models have dramatically increased computational and memory requirements when dealing with long contexts, resulting in high computational costs and slow response times, failing to meet the real-time and high-efficiency requirements for legal document generation.
By constructing a model input matrix, the attention score of historical tokens is determined using a full attention mechanism. Target tokens are selected to build a recycling cache, and a cosine similarity algorithm is called to determine target tokens from legal case data. The cache is dynamically updated to optimize the generation process. The generation process is optimized by implementing a pre-set large model and the current recycling cache update strategy.
With limited time and hardware resources, the efficiency and accuracy of legal document generation have been improved, the amount of computation has been reduced, the system response speed has been increased, and the real-time requirements have been met.
Smart Images
Figure CN121168419A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, in particular to a legal document generation method, device, equipment and storage medium. BACKGROUND
[0002] With the rapid development of artificial intelligence technology, large language models have shown excellent ability in processing complex and long text data. They can understand and generate long sequences of text, providing strong support for various application scenarios. However, when dealing with long context, such models face a sharp increase in computational and memory requirements, which becomes a key factor restricting their widespread application. When dealing with long context, large language models need to perform attention computation on large input sequences at each generation step. This amount of computation increases significantly with the length of the context, not only leading to a surge in computational cost, but also significantly reducing the response speed of the model in practical applications, and even making it difficult to meet real-time requirements. More seriously, as the length of the context increases, the memory requirements of the Key-Value (KV) cache that the model needs to store also increase linearly, while the time delay shows a quadratic growth trend. This growth pattern not only exacerbates the computational burden, but also may cause the model to perform poorly in practical applications, failing to meet the efficient and real-time processing requirements. In the face of legal application scenarios, using existing long context processing methods cannot quickly generate legal documents, which may affect the speed of handling corresponding cases.
[0003] Therefore, how to improve the efficiency and accuracy of legal document text generation is a problem to be solved. SUMMARY
[0004] Therefore, the purpose of the present application is to provide a legal document generation method, device, equipment and storage medium, which can improve the efficiency and accuracy of legal document text generation. The specific solutions are as follows:
[0005] In a first aspect, the present application discloses a legal document generation method, comprising:
[0006] Based on the large model legal text data, a corresponding model input matrix is constructed, and a first attention score of each historical token in the model input matrix is determined by using a preset full attention mechanism; the large model legal text data includes legal case data and legal contract data;
[0007] Based on the first attention score, a first target token is selected from each of the historical tokens to construct a current recycling cache, and then a cosine similarity algorithm is called and a second target token is determined from the legal case data by using the preset large model and the current recycling cache;
[0008] determining a second attention score of the second target token, and adding the second target token into the model input matrix;
[0009] updating the current recycling cache by using the second attention score to obtain a new current recycling cache, and jumping to the step of calling the cosine similarity algorithm and determining a second target token from the legal case data by using a preset large model and the current recycling cache until the generation of the legal document is completed.
[0010] Optionally, the constructing of the corresponding model input matrix based on the large model legal text data comprises:
[0011] obtaining legal text data of a large model, and performing data preprocessing and data formatting on the legal text data to obtain processed legal text data;
[0012] constructing a KV cache corresponding to the processed legal text data to obtain a model input matrix.
[0013] Optionally, the determining of the first attention score of each historical token in the model input matrix by using a preset full attention mechanism comprises:
[0014] analyzing attention models between different layers and different attention heads in the large model to determine a target attention head from the attention heads by using a preset clustering algorithm;
[0015] allocating computing resources of the large model to the attention heads according to the target attention head and a preset computing resource allocation method;
[0016] determining the first attention score of each historical token in the model input matrix by using the attention heads and a preset similarity measurement algorithm.
[0017] Optionally, the selecting of a first target token from each historical token based on the first attention score to construct a current recycling cache comprises:
[0018] selecting a plurality of tokens from the historical tokens by using a fast selection algorithm to obtain the first target token; the first attention score of the first target token is greater than the first attention score of other tokens in the historical tokens;
[0019] constructing a current recycling cache based on the first target token.
[0020] Optionally, before the updating of the current recycling cache by using the second attention score to obtain a new current recycling cache, the method further comprises:
[0021] obtaining real-time performance feedback and computing resource usage of the large model, and determining whether to start a cache update operation based on the real-time performance feedback and the computing resource usage; the real-time performance feedback and the computing resource usage are determined based on the first attention score and the second attention score;
[0022] If the cache update operation is started, the step of updating the current recycled cache using the second attention score to obtain a new current recycled cache is performed.
[0023] Optionally, before the step of updating the current recycled cache using the second attention score to obtain a new current recycled cache, the method further comprises:
[0024] obtaining the current recycled cache and a historical recycled cache, and determining an attention similarity value between the current recycled cache and the historical recycled cache using a preset similarity analysis algorithm;
[0025] If the attention similarity value is less than a preset similarity threshold, the step of updating the current recycled cache using the second attention score to obtain a new current recycled cache is performed.
[0026] Optionally, after the step of updating the current recycled cache using the second attention score to obtain a new current recycled cache, the method further comprises:
[0027] compressing cache data in the current recycled cache using a preset cache compression technology to obtain compressed cache, and storing and managing the compressed cache using a binary heap.
[0028] In a second aspect, the application discloses a legal document generation device, comprising:
[0029] a first attention determination module configured to construct a corresponding model input matrix based on large model legal text data, and determine a first attention score of each historical token in the model input matrix using a preset full attention mechanism; the large model legal text data comprises legal case data and legal contract data;
[0030] a target token determination module configured to select a first target token from each of the historical tokens based on the first attention score to construct a current recycled cache, and then call a cosine similarity algorithm and determine a second target token from the legal case data using the large model and the current recycled cache;
[0031] a second attention determination module configured to determine a second attention score of the second target token and add the second target token into the model input matrix;
[0032] a cache update module configured to update the current recycled cache by using the second attention score to obtain a new current recycled cache, and jump to the step of determining a second target token from the legal case data by using a preset large model and the current recycled cache until the generation of the legal document is completed.
[0033] In a third aspect, the present application discloses an electronic device, comprising:
[0034] a memory configured to save a computer program;
[0035] a processor configured to execute the computer program to implement the legal document generation method described above.
[0036] In a fourth aspect, the present application discloses a computer readable storage medium configured to save a computer program, wherein the computer program is executed by a processor to implement the legal document generation method described above.
[0037] It can be seen that, in the present application, a corresponding model input matrix is constructed based on large model legal text data, and a first attention score of each historical token in the model input matrix is determined using a preset full attention mechanism; the large model legal text data includes legal case data and legal contract data; a first target token is selected from each historical token based on the first attention score to construct a current recycling cache, and then a cosine similarity algorithm is called and the current recycling cache is used to determine a second target token from the legal case data using a preset large model; a second attention score of the second target token is determined, and the second target token is added to the model input matrix; the current recycling cache is updated using the second attention score to obtain a new current recycling cache, and the step of calling the cosine similarity algorithm and using the preset large model and the current recycling cache to determine the second target token from the legal case data is jumped to until the generation of legal documents is completed. That is, the entire input sequence is processed by the full attention mechanism to construct a complete recycling cache. Then, according to the attention score, an initial recycling cache is selected and the attention weight of the token is recorded for the effectiveness and pertinence of the initial cache. In the subsequent token generation process, the recycling cache is constantly optimized and updated according to the newly generated token. In this way, in the generation process of long text, it is not necessary to calculate the attention of the token every time, and the recycling cache is updated in real time to complete the generation of legal text, which realizes the accurate processing of long text in limited time and hardware resources, and improves the generation efficiency of legal text. BRIEF DESCRIPTION OF DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.
[0039] Figure 1 A legal document generation method flowchart disclosed by the present application;
[0040] Figure 2 A legal document generation device structure schematic diagram disclosed by the present application;
[0041] Figure 3 A structure diagram of an electronic device disclosed by the present application. DETAILED DESCRIPTION
[0042] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the scope of protection of the present application.
[0043] At present, when processing legal documents that need to generate longer texts using the prior art, the problems of calculation memory overflow, reasoning lag and insufficient precision are easily encountered. Therefore, the present application will specifically introduce a legal document generation method, which can solve the above problems.
[0044] Referring to Figure 1 As shown in the drawings, the embodiments of the present application disclose a legal document generation method, comprising:
[0045] Step S11: constructing a corresponding model input matrix based on large model legal text data, and determining the first attention score of each history token in the model input matrix by using a preset full attention mechanism; the large model legal text data includes legal case data and legal contract data.
[0046] In the present embodiment, the construction of the corresponding model input matrix based on the large model legal text data comprises: obtaining the legal text data of the large model, and performing data preprocessing and data formatting on the legal text data to obtain processed legal text data; constructing the KV cache corresponding to the processed legal text data to obtain the model input matrix. That is, it is responsible for receiving input data and performing preprocessing and formatting operations, which can ensure the accuracy and consistency of the data. In other words, the original input data is received. Then apply the preprocessing function, the specific operations include but are not limited to noise removal, standardization, encoding and other operations, to obtain the preprocessed data.
[0047] In this embodiment, the first attention score of each historical token in the model input matrix is determined by using the preset full attention mechanism, which includes analyzing the attention model between different layers and different attention heads in the large model, using a preset clustering algorithm to determine the target attention head from the attention heads, allocating the computing resources of the large model to the attention heads according to the target attention head and a preset computing resource allocation method, and determining the first attention score of each historical token in the model input matrix by using the attention heads and a preset similarity measurement algorithm. That is, the full context attention and recycling attention are calculated, and the appropriate attention mode is selected for reasoning according to the dynamic scheduling strategy, to ensure the accuracy and efficiency of reasoning. Specifically, by analyzing the attention model between different layers and different attention heads in the large model in detail, the target attention head is identified from all attention heads by using a preset clustering algorithm (such as K-means or hierarchical clustering), which usually represents those heads that have the greatest impact on the performance of the model or are the most representative. Then, according to these target attention heads, the computing resources of the large model are intelligently allocated to each attention head by using a preset computing resource allocation method (such as a strategy based on priority or load balancing), to ensure efficient use of resources. Then, the first attention score of each historical token in the model input matrix is calculated by using the attention heads and a preset similarity measurement algorithm (such as cosine similarity or Euclidean distance), to capture key context information. On this basis, the system calculates the full context attention and recycling attention, and selects the most appropriate attention mode for reasoning by using a dynamic scheduling strategy (such as based on real-time performance monitoring or heuristic rules), to maximize the efficiency while ensuring the accuracy of reasoning.
[0048] In actual application scenarios, an example is given. A lawyer of a law firm represents a case of "employee of a technology company claiming illegal termination of labor contract", which needs to complete three core tasks: searching for 12 labor contract dispute cases (in PDF format) in the past 5 years, each with 3000-5000 words, and the total word count exceeds 50,000 words; extracting key information such as "legality of termination procedure" and "calculation of economic compensation" from the 8000-word labor contract text (in Word format) provided by the client; generating a "legal analysis report" (which needs to include "matching degree of the case and the cases", "basis for identifying illegal termination", and "compensation estimation") within 2 hours, which is used for the case discussion meeting in the afternoon. Under such demand, the generated journal analysis report must be real-time, accurate and efficient.
[0049] Using the method of the embodiment, 12 cases and client labor contracts are received, and preprocessing is automatically completed; specifically, it includes format cleaning to remove watermarks, page numbers, and redundant formats in tables in the cases; content denoising to filter irrelevant text such as “party privacy information” and “court procedural description” in the cases; and unified coding to convert all text to UTF-8 (Universal Character Set / Unicode Transformation Format, a variable-length character encoding) encoding, output standardized text data after word segmentation, and ensure consistency for subsequent calculations. Then, the model input matrix of all text (full context attention matrix) is calculated first. Further, all attention heads are clustered using K-means, 30% of the key heads related to “legal clause reference” and “case judgment reason” are identified (these heads directly affect the analysis accuracy), and 60% of the computing resources are allocated to them; the remaining secondary heads related to “case fact detail description” are compressed by sparseness to reduce the amount of calculation, and the overall calculation efficiency is improved by 50%. The attention scores in each model input matrix are determined using the allocated computing resources.
[0050] Step S12: selecting a first target token from each of the historical tokens based on the first attention score to construct a current recycling cache, then calling a cosine similarity algorithm and determining a second target token from the legal case data using a preset large model and the current recycling cache.
[0051] In this embodiment, the first target token is selected from each of the historical tokens based on the first attention score to construct the current recycling cache, including: selecting a number of tokens from the historical tokens using a quick selection algorithm to obtain the first target token; the first attention score of the first target token is greater than the first attention score of other tokens in the historical tokens; and constructing the current recycling cache based on the first target token. That is, the historical tokens are sorted based on the first attention score of each token, but the entire sequence does not need to be completely sorted, but a quick selection algorithm is used to efficiently locate and select the top K tokens, where K can be dynamically adjusted according to system requirements, thereby obtaining the first target token. The first target token is characterized by its first attention score being significantly higher than other tokens in the historical tokens, indicating that it has higher relevance or importance in the current context. Once the first target token is determined, the current recycling cache is constructed based on it, that is, by integrating the token and its associated context information (such as position encoding, embedding vector, etc.) into a dedicated cache structure, so as to quickly access and reuse in subsequent reasoning or generation steps, thereby reducing computational overhead and improving system response speed. The entire process ensures efficient resource management while maintaining high accuracy, suitable for real-time applications such as dialog systems or text generation. Then in the operation process, the cosine similarity algorithm is called, combined with the preset large model and the data in the current recycling cache, to perform vectorization processing and similarity calculation on the legal case database, thereby filtering and determining the second target token from the data.
[0052] In actual application, the Quickselect algorithm (quick selection algorithm) can be used to select 200 key tokens (such as “illegal termination circumstances”, “economic compensation calculation standard”, “trade union notice obligation”) from the model input matrix to construct an initial recycling cache. Then, the three types of query heads “compensation calculation”, “termination procedure legality” and “evidence requirement” are grouped, and the cosine similarity is aggregated with the group attention score to quickly locate the same type of key information in 12 cases (such as “legal consequences of not notifying the trade union” in all cases), avoiding repeated labor of searching each case. Then, the second target text is determined from the legal case data according to the cosine similarity algorithm and using the preset large model and the current recycling cache.
[0053] Step S13: determining the second attention score of the second target token, and adding the second target token to the model input matrix.
[0054] In this embodiment, based on the preset full attention mechanism, the second attention score of the second target token is determined, and then the second target token is added to the model input matrix. Because the second target token generated at this time can also be used to generate subsequent text with large model legal text data.
[0055] Step S14: updating the current recycling cache using the second attention score to obtain a new current recycling cache, and jumping to the step of calling the cosine similarity algorithm and determining the second target token from the legal case data using the preset large model and the current recycling cache until the generation of the legal document is completed.
[0056] In this embodiment, the current recycling cache is updated using the second attention score to obtain a new current recycling cache. Specifically, the current recycling cache can be maintained using a binary heap. When analyzing the customer labor contract, the newly calculated "contract 'termination requires 30-day written notice' clause" related attention score is higher, and the system automatically adds it to the current recycling cache, while removing "work injury termination" related tokens that are not related to the case, ensuring that the cache is always "the most useful information for the current analysis".
[0057] Before the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache, the method further includes: obtaining real-time performance feedback and computing resource usage of the large model, and determining whether to start a cache update operation based on the real-time performance feedback and the computing resource usage; the real-time performance feedback and the computing resource usage are determined based on the first attention score and the second attention score; if the cache update operation is started, the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache is performed. That is, according to the real-time performance feedback and the computing resource usage, the attention calculation mode and the cache management strategy are dynamically adjusted, so as to ensure the flexibility of reasoning and the efficient use of resources. It should be noted that the real-time performance feedback and the computing resource usage are determined by using a PID controller based on the first attention score and the second attention score. In actual operation, the reasoning accuracy is monitored by using a PID controller (Proportion Integration Differentiation, PID controller), and "analysis conclusion and target clause matching degree ≥ 90%" is set as A target; when the first three cases are analyzed, the matching degree of the "compensation standard" output is only 82% (lower than A target), and the system automatically shortens the full attention calculation step from S=3 to S=2 to ensure that no key information is missed; then the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache is performed. After the subsequent analysis accuracy meets the standard, the step is restored to S=3, and the speed is considered.
[0058] Before the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache, the method further includes: obtaining the current recycling cache and a historical recycling cache, and determining an attention similarity value between the current recycling cache and the historical recycling cache by using a preset similarity analysis algorithm; if the attention similarity value is less than a preset similarity threshold, the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache is performed. In actual operation, when a new query "whether the customer case meets the illegal termination situation of case 3" is input, the cosine similarity S=0.85 (the threshold is set to 0.7) between the current query current recycling cache and the historical recycling cache ("illegal termination situation of case 2") is calculated, and it is determined that the two are highly similar, and the full attention update is not triggered, and the historical recycling cache is directly used for reasoning, so that the response time is shortened from 12 seconds to 1.2 seconds.
[0059] After the current recycling cache is updated by using the second attention score to obtain a new current recycling cache, the method further includes: compressing cache data in the current recycling cache by using a preset cache compression technology to obtain compressed cache, and storing and managing the compressed cache by using a binary heap. That is, the cache data in the current recycling cache is compressed by using an LZ77 algorithm, so that the cache memory occupation is greatly reduced, and the local processor can easily load all the texts without processing in batches.
[0060] In the embodiment, steps S12-S14 can be repeatedly executed until the generation of all legal texts is completed to obtain a legal analysis report. It should be noted that in the pre-training stage, the cache management and attention allocation strategy in the reasoning process are simulated to enable the model to gradually adapt to this efficient reasoning manner. By comparing the output quality of the model under different reasoning strategies, the system will further guide the model to learn more effective attention allocation and cache management strategies to ensure the stable performance of the model in complex long context scenarios.
[0061] In addition, it should be noted that in actual application scenarios, whether it is multi-case analysis in legal practice, long document integration in academic research, or financial report interpretation in enterprises, the essence is to accurately process long texts within limited time and hardware resources. The above legal cases prove that the system can effectively solve the "efficiency, memory, and accuracy" pain points of traditional LLMs and provide practical solutions for long text intelligent processing in various industries.
[0062] It can be seen that, in the embodiment, a corresponding model input matrix is constructed based on large model legal text data, and a first attention score of each historical token in the model input matrix is determined by using a preset full attention mechanism; the large model legal text data includes legal case data and legal contract data; a first target token is selected from each of the historical tokens based on the first attention score to construct a current recycling cache, and then a cosine similarity algorithm is called and the current recycling cache is used to determine a second target token from the legal case data by using the preset large model; a second attention score of the second target token is determined, and the second target token is added to the model input matrix; the current recycling cache is updated by using the second attention score to obtain a new current recycling cache, and the step of calling the cosine similarity algorithm and using the preset large model and the current recycling cache to determine the second target token from the legal case data is jumped to until the generation of the legal document is completed. That is, the entire input sequence is processed by the full attention mechanism to construct a complete recycling cache. Then, the initial recycling cache is selected according to the attention score, and the attention weight of the token is recorded for the effectiveness and pertinence of the initial cache. In the subsequent token generation process, the recycling cache is continuously optimized and updated according to the newly generated token. In this way, in the generation process of long text, it is not necessary to calculate the attention of the token every time, and the recycling cache is updated in real time to complete the generation of the legal text, realize the accurate processing of long text in limited time and hardware resources, and improve the generation efficiency of the legal text.
[0063] Reference Figure 2 As described above, the embodiments of the present application also correspondingly disclose a legal document generation device, comprising:
[0064] The first attention determination module 11 is configured to construct a corresponding model input matrix based on large model legal text data, and determine a first attention score of each historical token in the model input matrix by using a preset full attention mechanism; the large model legal text data includes legal case data and legal contract data;
[0065] The target token determination module 12 is configured to select a first target token from each of the historical tokens based on the first attention score to construct a current recycling cache, and then call a cosine similarity algorithm and use the large model and the current recycling cache to determine a second target token from the legal case data;
[0066] The second attention determination module 13 is configured to determine a second attention score of the second target token, and add the second target token to the model input matrix.
[0067] The cache updating module 14 is configured to update the current recycling cache by using the second attention score to obtain a new current recycling cache, and jump to the step of calling the cosine similarity algorithm and determining a second target token from the legal case data by using a preset large model and the current recycling cache until the generation of the legal document is completed.
[0068] It can be seen that, in the embodiment, the entire input sequence is processed by the full attention mechanism to construct a complete recycling cache. Then, the initial recycling cache is selected according to the attention score, and the attention weights of the tokens are recorded to reflect the effectiveness and pertinence of the initial cache. In the subsequent token generation process, the recycling cache is constantly optimized and updated according to the newly generated token. In this way, in the generation process of a long text, it is not necessary to calculate the attention of the token every time, and the recycling cache that is updated in real time can be used to complete the generation of the legal text, so that the long text is accurately processed in a limited time and hardware resources, and the generation efficiency of the legal text is improved.
[0069] In some specific embodiments, the first attention determination module 11 can specifically include:
[0070] The data processing unit is configured to obtain legal text data of a large model, and perform data preprocessing and data formatting on the legal text data to obtain processed legal text data.
[0071] The cache construction unit is configured to construct a KV cache corresponding to the processed legal text data to obtain a model input matrix.
[0072] In some specific embodiments, the first attention determination module 11 can specifically include:
[0073] The attention head determination unit is configured to analyze the attention models between different layers and different attention heads in the large model, and determine a target attention head from the attention heads by using a preset clustering algorithm.
[0074] The resource allocation unit is configured to allocate computing resources of the large model to the attention heads according to the target attention head and a preset resource allocation method.
[0075] The attention score determination unit is configured to determine first attention scores of each historical token in the model input matrix by using the attention heads and a preset similarity measurement algorithm.
[0076] In some specific embodiments, the target token determination module 12 can specifically include:
[0077] a token selection unit configured to select a plurality of tokens from the historical tokens by using a fast selection algorithm to obtain first target tokens, wherein a first attention score of the first target tokens is greater than a first attention score of other tokens in the historical tokens;
[0078] a cache construction unit configured to construct a current recycling cache based on the first target tokens.
[0079] In some specific embodiments, the legal document generation apparatus can further include:
[0080] an update condition judgment module configured to obtain real-time performance feedback and computing resource usage of the large model, and determine whether to start a cache update operation based on the real-time performance feedback and the computing resource usage, wherein the real-time performance feedback and the computing resource usage are determined based on the first attention score and the second attention score;
[0081] a cache update module configured to, if the cache update operation is started, perform the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache.
[0082] In some specific embodiments, the legal document generation apparatus can further include:
[0083] a similarity calculation module configured to obtain the current recycling cache and a historical recycling cache, and determine an attention similarity value between the current recycling cache and the historical recycling cache by using a preset similarity analysis algorithm;
[0084] a step jump module configured to, if the attention similarity value is less than a preset similarity threshold, perform the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache.
[0085] In some specific embodiments, the legal document generation apparatus can further include:
[0086] a cache compression module configured to compress cache data in the current recycling cache by using a preset cache compression technology to obtain compressed cache, and store and manage the compressed cache by using a binary heap.
[0087] Further, the embodiments of the present application also disclose an electronic device, Figure 3 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the use range of the present application.
[0088] Figure 3 A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the processor 21 is configured to load and execute the computer program to implement the related steps in the legal document generation method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the embodiments of the present application can be specifically an electronic computer.
[0089] In the embodiments of the present application, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 is capable of creating a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input / output interface 25 is configured to obtain external legal text data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.
[0090] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.
[0091] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the legal document generation method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.
[0092] Further, the present application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the legal document generation method disclosed above. For the specific steps of the method, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.
[0093] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same or similar parts between the embodiments, please refer to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and please refer to the method part for the relevant part.
[0094] Those skilled in the art will further appreciate that the units and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their functionality, which has been described generally and symbolically in flow charts. Having thus described the functionality of the examples, a person of ordinary skill in the art will be able to implement such functions in hardware and / or software, using the means and methods available to those skilled in the art. The examples described herein are not meant to limit the scope of the application, but merely to provide examples of the methods and systems being described.
[0095] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0096] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of distinction from other elements in the specification. Also, the terms "comprise", "include" or "contain" or any other variant thereof are intended to encompass non-exclusive inclusions, such that processes, methods, articles, or apparatuses that comprise, include, or contain a list of elements are not limited to those elements, but can include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses. Without further limitation, an element defined by the phrase "comprising a... " does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0097] The above has described the technical solutions provided by the present application in detail, and the principles and implementation manners of the present application have been described by using specific examples; the above example descriptions are only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed; in conclusion, the content of the present description should not be understood as limiting the present application.
Claims
1. A legal document generation method characterized by comprising: The method comprises the following steps: Based on the large model legal text data, a corresponding model input matrix is constructed, and a first attention score of each historical token in the model input matrix is determined by using a preset full attention mechanism; The large model legal text data includes legal case data and legal contract data; Based on the first attention score, a first target token is selected from each historical token to construct a current recycling cache, and then a cosine similarity algorithm is called and a second target token is determined from the legal case data by using a preset large model and the current recycling cache; The second attention score of the second target token is determined, and the second target token is added to the model input matrix; The second attention score is used to update the current recycling cache to obtain a new current recycling cache, and the step of calling the cosine similarity algorithm and determining the second target token from the legal case data by using the preset large model and the current recycling cache is jumped to until the generation of legal documents is completed.
2. The legal document generating method of claim 1, wherein, The method comprises the following steps: Legal text data of a large model is obtained, and the legal text data is preprocessed and formatted to obtain processed legal text data; A KV cache corresponding to the processed legal text data is constructed to obtain a model input matrix.
3. The legal document generating method of claim 1, wherein, The method comprises the following steps: The attention models between different layers and different attention heads in the large model are analyzed to determine a target attention head from the attention heads by using a preset clustering algorithm; According to the target attention head, a calculation resource allocation method is designed to allocate the calculation resources of the large model to the attention heads; The first attention score of each historical token in the model input matrix is determined by using the attention heads and a preset similarity measurement algorithm.
4. The legal document generating method of claim 1, wherein, The method comprises the following steps: A fast selection algorithm is used to select a number of tokens from the historical tokens to obtain a first target token; the first attention score of the first target token is greater than the first attention score of other tokens in the historical tokens; A current recycling cache is constructed based on the first target token.
5. The legal document generating method of claim 1, wherein, Before the step of updating the current recycling cache by using the second attention score to obtain a new current recycling cache, the method further comprises the following steps: Real-time performance feedback and calculation resource usage of the large model are obtained, and it is determined whether to start a cache update operation based on the real-time performance feedback and the calculation resource usage; the real-time performance feedback and the calculation resource usage are determined based on the first attention score and the second attention score. If the cache update operation is started, the step of updating the current recycling cache using the second attention score to obtain a new current recycling cache is performed.
6. The legal document generating method of claim 1, wherein, Before the step of updating the current recycling cache using the second attention score to obtain a new current recycling cache, the method further comprises: obtaining the current recycling cache and a historical recycling cache, and determining an attention similarity value between the current recycling cache and the historical recycling cache using a preset similarity analysis algorithm; If the attention similarity value is less than a preset similarity threshold, the step of updating the current recycling cache using the second attention score to obtain a new current recycling cache is performed.
7. The legal document generating method according to any one of claims 1 to 6, characterized by, After the step of updating the current recycling cache using the second attention score to obtain a new current recycling cache, the method further comprises: compressing cache data in the current recycling cache using a preset cache compression technology to obtain compressed cache, and storing and managing the compressed cache using a binary heap.
8. A legal document generating apparatus characterized by comprising: Comprise: a first attention determination module configured to construct a corresponding model input matrix based on large model legal text data, and determine a first attention score of each historical token in the model input matrix using a preset full attention mechanism; The large model legal text data comprises legal case data and legal contract data; a target token determination module configured to select a first target token from each of the historical tokens based on the first attention score to construct a current recycling cache, and then call a cosine similarity algorithm and determine a second target token from the legal case data using the large model and the current recycling cache; a second attention determination module configured to determine a second attention score of the second target token, and add the second target token to the model input matrix; a cache update module configured to update the current recycling cache using the second attention score to obtain a new current recycling cache, and jump to the step of calling the cosine similarity algorithm and determining a second target token from the legal case data using a preset large model and the current recycling cache until the generation of the legal document is completed.
9. An electronic device, comprising: Comprise: a memory configured to save a computer program; a processor configured to execute the computer program to implement the legal document generation method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, A computer program is saved, and the computer program is executed by a processor to implement the legal document generation method according to any one of claims 1 to 7.