Text encoder training method, case search method, device and electronic equipment

Abstract

The present disclosure relates to a text encoder training method, a case retrieval method, a device and an electronic device. The text encoder training method comprises: obtaining a plurality of case groups, each case group comprising case information of at least two cases and corresponding similarity information; encoding the case information of each case using a text encoder to obtain case features of each case; determining intra-class feature similarity in each case group and inter-class feature similarity between each case and a different case according to the case features of each case; determining a contrast learning loss according to the intra-class feature similarity, the inter-class feature similarity and the similarity information corresponding to the cases, and training the text encoder based on the contrast learning loss. According to the embodiment of the present disclosure, fine-grained legal knowledge can be introduced to train the text encoder, so that the trained text encoder outputs high-quality case features, thereby improving the accuracy of case retrieval, and the retrieved same cases have interpretability.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a text encoder training method, a case retrieval method, an apparatus, and an electronic device. Background Technology

[0002] The definition of case retrieval is: given a query case, retrieving similar cases related to the query case from a candidate case database. Similar cases (classic cases) usually refer to cases with the same or similar essential facts or circumstances. For the modern judicial system, case retrieval is of great significance in ensuring consistent judgments in similar cases and promoting judicial fairness, as similar cases can serve as a reference for adjudicating cases.

[0003] In recent years, pre-trained language models (PLMs) have achieved good results in both natural language processing and retrieval tasks. Therefore, the application of PLM technology to case retrieval in legal cases has emerged. How to improve the performance of case retrieval tasks based on PLM has become a research hotspot. To address this issue, one existing approach is to propose BERT-XS based on BERT (a type of PLM), which is a BERT model pre-trained using legal documents. BERT-XS uses the same model structure as BERT, except that the training corpus is replaced with legal documents. Another approach is to use Lawformer, which is a PLM for the legal domain based on the Longformer model. This model takes into account the characteristics of long legal texts, thus expanding the length limit of the text input, and combining global and local attention mechanisms to help the model capture the contextual information of long texts.

[0004] However, neither of the two existing technologies mentioned above deeply integrates legal knowledge to optimize the model. They merely replace the training corpus with legal texts or optimize superficial features such as text length. In essence, they still use general PLM. However, the correlation between legal texts requires stronger legal knowledge, which is significantly different from the traditional text correlation. This means that the PLM mentioned above does not truly understand the concept of similar cases from a legal perspective, resulting in low accuracy of PLM when searching for similar cases. At the same time, the retrieved similar cases also lack good interpretability. Summary of the Invention

[0005] In view of this, this disclosure proposes a text encoder training method, a case retrieval method, an apparatus, and an electronic device that can introduce fine-grained legal knowledge to train the text encoder, enabling the trained text encoder to output high-quality case features. Thus, when using the case features output by the trained text encoder for case retrieval, the accuracy of case retrieval can be effectively improved, while ensuring that the retrieved similar cases are interpretable cases determined by legal knowledge.

[0006] According to one aspect of this disclosure, a text encoder training method is provided, comprising: acquiring a case set, the case set including multiple case groups, each case group including: case information of at least two cases that are of the same type and similarity information corresponding to each of the at least two cases, the similarity information being determined based on the similarity between the judgment information of the case and a set of unambiguous legal provisions involved in the case, cases in different case groups being of different types, the set of unambiguous legal provisions including at least one unambiguous legal provision; encoding the case information of each case in the multiple case groups using a text encoder to obtain case features of each case in the multiple case groups; determining the intra-class feature similarity between at least two cases in each case group and the inter-class feature similarity between each of the at least two cases in each case group and the case of the opposite type, based on the case features of each case in the multiple case groups; determining a contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to each of the at least two cases in each case group of the case set, and training the text encoder based on the contrastive learning loss.

[0007] In one possible implementation, obtaining the case set includes: obtaining judgment information for each case in a plurality of cases in a preset case database, and obtaining a set of unambiguous legal provisions involved in each case in the preset case database, wherein the judgment information includes the reasons for the judgment of the case; determining similarity information corresponding to each case in the preset case database based on the similarity between the judgment information of each case in the preset case database and each unambiguous legal provision in the set of unambiguous legal provisions involved in each case; determining similar cases of the same type as each case in the preset case database based on the similar cases of each case in the preset case database and the similarity information corresponding to each case; and determining at least one case set for at least one training batch based on the similar cases of each case in the preset case database and the similarity information corresponding to each case.

[0008] In one possible implementation, obtaining the set of unambiguous legal provisions involved in each case in the preset case database includes: obtaining multiple original legal provisions involved in multiple cases in the preset case database, with each case involving at least one original legal provision; splitting each original legal provision into at least one branch clause, and extracting keywords from each branch clause to obtain at least one unambiguous legal provision under each original legal provision; and determining the set of unambiguous legal provisions involved in each case in the preset case database based on the at least one original legal provision involved in each case and the at least one unambiguous legal provision under each original legal provision.

[0009] In one possible implementation, the preset case database includes I cases, where I is a positive integer. The step of determining similar cases for each case in the preset case database based on the similarity information of each case includes: for the i-th case in the preset case database, determining J initial similar cases (i < I, J < I) that involve the same original legal provisions as the i-th case; determining the relevance weight between the i-th case and each initial similar case based on the similarity information corresponding to the i-th case and the J initial similar cases; and determining similar cases related to the i-th case from the J initial similar cases based on the relevance weight between the i-th case and each initial similar case.

[0010] In one possible implementation, determining the relevance weight between the i-th case and each of the J initial similar cases based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases includes: determining a first unambiguous legal provision with the highest similarity to the judgment information of the i-th case based on the similarity information corresponding to the i-th case; and determining a second unambiguous legal provision with the highest similarity to the judgment information of the j-th initial similar case based on the similarity information corresponding to the j-th initial similar case, where j < J; and when the first unambiguous legal provision and the second unambiguous legal provision are the same, based on the... The degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case determines the relevance weight between the i-th case and the j-th initial similar case, wherein the degree of overlap is positively correlated with the relevance weight; or, if the first unambiguous legal provision and the second unambiguous legal provision are not the same, the relevance weight between the i-th case and the j-th initial similar case is determined based on the similarity score between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case, and the degree of overlap, wherein the similarity score is positively correlated with the relevance weight.

[0011] In one possible implementation, determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set includes: determining the relevance weight between each pair of cases in each case group based on the similarity information corresponding to at least two cases in each case group; determining a first convergence control parameter based on the relevance weight between each pair of cases in each case group and the intra-class feature similarity between each pair of cases in each case group; determining a second convergence control parameter based on the inter-class feature similarity between each of at least two cases in each case group and cases of the opposite class; and determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, the first convergence control parameter, and the second convergence control parameter.

[0012] In one possible implementation, the case information of each case in the case set is partially masked out according to a preset masking ratio. The method further includes: predicting the masked real words in the case information of each case based on the case features corresponding to each case in the case set, and obtaining the predicted words corresponding to each case; determining the masking language loss based on the predicted words corresponding to each case and the masked real words in the case information of each case, and training the text encoder based on the masking language loss.

[0013] According to another aspect of this disclosure, a method for retrieving similar cases is provided. The method includes: obtaining case information of a target case to be retrieved; encoding the case information of the target case using a text encoder to obtain case features corresponding to the target case, wherein the text encoder is trained using the aforementioned text encoder training method; and determining similar cases of the target case from the target case database based on the case features corresponding to the target case and the case features of multiple adjudicated cases in the target case database.

[0014] In one possible implementation, the method further includes: predicting the original legal provisions and / or unambiguous legal provisions involved in the target case based on the case characteristics corresponding to the target case.

[0015] According to another aspect of this disclosure, a text encoder training apparatus is provided, comprising: an acquisition module for acquiring a case set, the case set including multiple case groups, each case group including: case information of at least two cases of the same type and similarity information corresponding to each of the at least two cases, the similarity information being determined based on the similarity between the judgment information of the case and a set of unambiguous legal provisions involved in the case, the cases in different case groups being dissimilar cases, the set of unambiguous legal provisions including at least one unambiguous legal provision; an encoding module for encoding the case information of each case in the multiple case groups using a text encoder to obtain case features of each case in the multiple case groups; a determination module for determining, based on the case features of each case in the multiple case groups, the intra-class feature similarity between at least two cases in each case group, and the inter-class feature similarity between each of the at least two cases in each case group and dissimilar cases; and a training module for determining a contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to each of the at least two cases in each case group of the case set, and training the text encoder based on the contrastive learning loss.

[0016] In one possible implementation, obtaining the case set includes: obtaining judgment information for each case in a plurality of cases in a preset case database, and obtaining a set of unambiguous legal provisions involved in each case in the preset case database, wherein the judgment information includes the reasons for the judgment of the case; determining similarity information corresponding to each case in the preset case database based on the similarity between the judgment information of each case in the preset case database and each unambiguous legal provision in the set of unambiguous legal provisions involved in each case; determining similar cases of the same type as each case in the preset case database based on the similar cases of each case in the preset case database and the similarity information corresponding to each case; and determining at least one case set for at least one training batch based on the similar cases of each case in the preset case database and the similarity information corresponding to each case.

[0017] In one possible implementation, obtaining the set of unambiguous legal provisions involved in each case in the preset case database includes: obtaining multiple original legal provisions involved in multiple cases in the preset case database, with each case involving at least one original legal provision; splitting each original legal provision into at least one branch clause, and extracting keywords from each branch clause to obtain at least one unambiguous legal provision under each original legal provision; and determining the set of unambiguous legal provisions involved in each case in the preset case database based on the at least one original legal provision involved in each case and the at least one unambiguous legal provision under each original legal provision.

[0018] In one possible implementation, the preset case database includes I cases, where I is a positive integer. The step of determining similar cases for each case in the preset case database based on the similarity information of each case includes: for the i-th case in the preset case database, determining J initial similar cases (i < I, J < I) that involve the same original legal provisions as the i-th case; determining the relevance weight between the i-th case and each initial similar case based on the similarity information corresponding to the i-th case and the J initial similar cases; and determining similar cases related to the i-th case from the J initial similar cases based on the relevance weight between the i-th case and each initial similar case.

[0019] In one possible implementation, determining the relevance weight between the i-th case and each of the J initial similar cases based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases includes: determining a first unambiguous legal provision with the highest similarity to the judgment information of the i-th case based on the similarity information corresponding to the i-th case; and determining a second unambiguous legal provision with the highest similarity to the judgment information of the j-th initial similar case based on the similarity information corresponding to the j-th initial similar case, where j < J; and when the first unambiguous legal provision and the second unambiguous legal provision are the same, based on the... The degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case determines the relevance weight between the i-th case and the j-th initial similar case, wherein the degree of overlap is positively correlated with the relevance weight; or, if the first unambiguous legal provision and the second unambiguous legal provision are not the same, the relevance weight between the i-th case and the j-th initial similar case is determined based on the similarity score between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case, and the degree of overlap, wherein the similarity score is positively correlated with the relevance weight.

[0020] In one possible implementation, determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set includes: determining the relevance weight between each pair of cases in each case group based on the similarity information corresponding to at least two cases in each case group; determining a first convergence control parameter based on the relevance weight between each pair of cases in each case group and the intra-class feature similarity between each pair of cases in each case group; determining a second convergence control parameter based on the inter-class feature similarity between each of at least two cases in each case group and cases of the opposite class; and determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, the first convergence control parameter, and the second convergence control parameter.

[0021] In one possible implementation, the case information of each case in the case set is partially masked out according to a preset masking ratio. The device further includes: a vocabulary prediction module, used to predict the masked real words in the case information of each case based on the case features corresponding to each case in the case set, and obtain the predicted vocabulary corresponding to each case; and a model training module, used to determine the masking language loss based on the predicted vocabulary corresponding to each case and the masked real words in the case information of each case, and train the text encoder based on the masking language loss.

[0022] According to another aspect of this disclosure, a case retrieval device is provided, comprising: an information acquisition module for acquiring case information of a target case to be retrieved; an information encoding module for encoding the case information of the target case using a text encoder to obtain case features corresponding to the target case, wherein the text encoder is trained using the aforementioned text encoder training method; and a case determination module for determining similar cases to the target case from the target case database based on the case features corresponding to the target case and the case features of multiple adjudicated cases in the target case database.

[0023] In one possible implementation, the case retrieval device further includes a legal provision prediction module, used to predict the original legal provisions and / or unambiguous legal provisions involved in the target case based on the case characteristics corresponding to the target case.

[0024] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0025] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0026] According to another aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0027] According to embodiments of this disclosure, during the training process of the text encoder, the contrastive learning loss is determined by introducing similarity information between the set of unambiguous legal provisions involved in the case and the judgment information of the case. Since the judgment information of the case is highly related to the legal provisions involved, this is equivalent to introducing fine-grained legal knowledge to train the text encoder. This enables the encoder to learn the textual relevance between similar cases in the legal field. At the same time, by using intra-class feature similarity and inter-class feature similarity to determine the contrastive learning loss, the text encoder can be trained to bring case features of the same category closer together and push case features of different categories further apart. This allows the trained text encoder to output high-quality case features. Thus, when using the case features output by the trained text encoder for case retrieval, the accuracy of case retrieval can be effectively improved, and the retrieved similar cases are cases that are interpretable based on legal knowledge.

[0028] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0029] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0030] Figure 1 A flowchart illustrating a text encoder training method according to an embodiment of the present disclosure is shown.

[0031] Figure 2 A schematic diagram illustrating the process of determining an unambiguous legal provision according to an embodiment of the present disclosure is shown.

[0032] Figure 3 A schematic diagram illustrating a text encoder training process according to an embodiment of the present disclosure is shown.

[0033] Figure 4 A block diagram of a text encoder training apparatus according to an embodiment of the present disclosure is shown.

[0034] Figure 5A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. Detailed Implementation

[0035] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0036] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0037] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0038] Figure 1 A flowchart illustrating a text encoder training method according to an embodiment of this disclosure is provided. This text encoder training method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable instructions stored in memory, or it can be executed by a server. Figure 1 As shown, the text encoder training method includes:

[0039] Step S101: Obtain a case set. The case set includes multiple case groups. Each case group includes: case information of at least two cases that are of the same type and similarity information corresponding to each of the at least two cases. The similarity information is determined based on the similarity between the judgment information of the case and the set of unambiguous legal provisions involved in the case. Cases in different case groups are dissimilar to each other. The set of unambiguous legal provisions includes at least one unambiguous legal provision.

[0040] Step S102: Use a text encoder to encode the case information of each case in multiple case groups to obtain the case features of each case in multiple case groups.

[0041] Step S103: Based on the case characteristics of each case in multiple case groups, determine the intra-class feature similarity between at least two cases in each case group, and the inter-class feature similarity between each at least two cases in each case group and the case of the opposite class.

[0042] Step S104: Determine the contrastive learning loss based on the intra-class feature similarity, inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set, and train the text encoder based on the contrastive learning loss.

[0043] It should be understood that model training can be divided into multiple training batches, and each training batch can correspond to a case set. The number of case groups in the case set of each training batch can be customized, and this embodiment of the present disclosure does not limit this. The above-mentioned acquisition of the case set may refer to acquisition of the case set of the current training batch.

[0044] Case information may include facts, circumstances, parties involved, purpose, and outcome of the case. An unambiguous legal provision is a legal clause without ambiguity; it should be understood that each case may involve at least one unambiguous legal provision. Similar cases may refer to those with the same or similar essential facts or circumstances, while dissimilar cases may refer to those with different or dissimilar essential facts or circumstances.

[0045] It should be understood that the cases in the case set can be adjudicated cases. Therefore, the case information of each case and the set of unambiguous legal provisions involved in each case can be extracted from the judgment documents of each case. This disclosure does not limit the method of obtaining the case information and the set of unambiguous legal provisions.

[0046] As mentioned above, most PLMs simply replace general domain training corpora with legal texts without considering the legal connections between these texts. In other words, current research on law-oriented PLMs does not focus on bottom-up approaches to understanding legal texts. This is primarily because manual annotation of legal texts is time-consuming and requires extensive expertise, making the collection of large-scale labeled data a challenge. On the other hand, contrastive learning has proven effective in the pre-training phase. The training data used in contrastive learning needs to be pre-sampled as positive and negative cases. It should be understood that in this embodiment, at least two cases within the same case group can be similar cases, i.e., positive cases, while cases in different case groups can be dissimilar cases, i.e., negative cases. Unlike general domains, in the legal field, it is inappropriate to extract positive and negative cases solely based on judgment information (such as charges, legal provisions, etc.) in court documents. In real legal scenarios, judges adjudicate cases based on the degree of matching between the facts of the case, the essential elements, and the constituent elements of the legal provisions. Therefore, this disclosure proposes a fine-grained case sampling method, referencing the process of judges adjudicating cases, to determine the case set used in each training batch. The case set determined in this way (i.e., the sampled positive and negative cases) can match the relevance of manually labeled cases as much as possible in subsequent comparative learning tasks, which is beneficial to improving the training effect of the text encoder.

[0047] In one possible implementation, obtaining the case set in step S101 may include:

[0048] Step S1011: Obtain the judgment information of each case in a preset case database, and obtain the set of unambiguous legal provisions involved in each case in the preset case database. The judgment information includes the reasons for the judgment of the case.

[0049] Step S1012: Based on the similarity between the judgment information of each case in the preset case database and the unambiguous legal provisions in the set of unambiguous legal provisions involved in each case, determine the similarity information corresponding to each case in the preset case database;

[0050] Step S1013: Based on the similarity information corresponding to each case in the preset case database, determine the similar cases of each case in the preset case database;

[0051] Step S1014: Based on the similar cases of each case in the preset case database and the similarity information corresponding to each case, determine at least one case set of at least one training batch.

[0052] Among them, the multiple cases in the preset case database can be cases that have been adjudicated. Therefore, in step S1011, the judgment information of each case can be extracted from the judgment text of each case. The judgment information can at least include the reason for the judgment of the case, and the reason for the judgment is highly related to the legal provisions involved in the case.

[0053] Considering that a primary legal provision may cover multiple sub-provisions, each describing a situation applicable to that sub-provision, it should be understood that any case matching a sub-provision belongs to the primary legal provision to which that sub-provision pertains. Obviously, cases under the same primary legal provision may belong to different sub-provisions. Therefore, the relevance between two cases cannot be directly determined by the reason "they both belong to Article 133-1." Thus, a more refined set of sub-provisions can be determined for each case, i.e., the set of unambiguous legal provisions involved in each case can be determined. Based on this, in one possible implementation, in step S1011, the set of unambiguous legal provisions involved in each case in a preset case database is obtained, including:

[0054] The process involves acquiring multiple original legal provisions related to various cases from a pre-defined case database, with each case involving at least one original legal provision. Each original legal provision is then broken down into at least one branch clause, and keywords are extracted from each branch clause to obtain at least one unambiguous legal provision under each original legal provision. Based on the at least one original legal provision involved in each case in the pre-defined case database and the at least one unambiguous legal provision under each original legal provision, the set of unambiguous legal provisions related to each case in the pre-defined case database is determined. This method allows for the identification of more granular unambiguous legal provisions for each case, thereby facilitating the identification of more precise similar cases for each case in the pre-defined case database.

[0055] Since multiple branch clauses within an original legal provision have both sequential and parallel relationships, we can first extract phrases from the original legal provision and recombine them into unambiguous branch clauses. This means splitting the original legal provision into at least one branch clause. Then, we can extract keywords from each branch clause to form a sequence, where each sequence represents a corresponding unambiguous legal provision. From each original legal provision, at least one unambiguous legal provision can be extracted, thus constructing a set of unambiguous legal provisions {seq1, seq2, ..., seq...} under each original legal provision. T}, where T is a positive integer representing the number of sequences extracted from a primary legal provision; it should be understood that each case may involve at least one primary legal provision, and given at least one primary legal provision involved in each case and at least one unambiguous legal provision under each primary legal provision, the set of unambiguous legal provisions involved in each case can be determined.

[0056] For example, with Figure 2This section briefly explains the process of determining an unambiguous legal provision, specifically regarding the original provision: Article 133-1 of the Criminal Law, which states that the crime of dangerous driving refers to driving a motor vehicle on a road under any of the following circumstances, punishable by detention and a fine:

[0057] (i) Racing and chasing, with egregious circumstances;

[0058] (ii) Driving a motor vehicle while intoxicated;

[0059] (iii) Engaging in school bus operations or passenger transport, seriously exceeding the rated passenger capacity, or seriously exceeding the prescribed speed limit;

[0060] (iv) Violating regulations on the safety management of hazardous chemicals by transporting hazardous chemicals that endanger public safety.

[0061] We can first extract phrases from the original legal provisions to obtain: "driving a motor vehicle on a road," "racing, with egregious circumstances," "driving a motor vehicle while intoxicated," "operating a school bus business," "operating passenger transport," "seriously exceeding the rated passenger capacity," "seriously exceeding the speed limit," and "violating regulations on the safety management of hazardous chemicals and transporting hazardous chemicals that endanger public safety." Then, we can recombine these phrases into unambiguous sub-clauses, resulting in: "driving a motor vehicle on a road, engaging in racing, with egregious circumstances," "driving a motor vehicle on a road while intoxicated," "driving a motor vehicle on a road, operating a school bus business, seriously exceeding the rated passenger capacity," "driving a motor vehicle on a road, operating a school bus business, seriously exceeding the speed limit," and "driving a motor vehicle on a road, operating a passenger transport business, seriously exceeding the rated passenger capacity." The legal provisions are categorized as follows: "Driving a motor vehicle on a road, engaging in passenger transport, and seriously exceeding the speed limit," and "Driving a motor vehicle on a road, violating the regulations on the safety management of hazardous chemicals, and transporting hazardous chemicals that endanger public safety." Keywords are then extracted from each of these provisions to form unambiguous legal articles. For example, for "Driving a motor vehicle on a road, engaging in reckless racing, with egregious circumstances," the keywords can be extracted to obtain the unambiguous legal article {road, driving, motor vehicle, reckless, racing, circumstances, egregious}. For "Driving a motor vehicle on a road while intoxicated," the keywords can be extracted to obtain the unambiguous legal article {road, driving, motor vehicle, intoxicated}. For "Driving a motor vehicle on a road, engaging in passenger transport, and seriously exceeding the rated passenger capacity," the keywords can be extracted to obtain the unambiguous legal article {road, driving, motor vehicle, passenger transport, exceeding, rated passengers}, and so on.

[0062] It should be understood that Figure 2The process of determining unambiguous legal provisions shown is one possible implementation provided by the embodiments of this disclosure. Optionally, branch clauses can also be directly used as unambiguous legal provisions. In fact, those skilled in the art can also use any known splitting method in the art to split the original legal provisions into at least one unambiguous legal provision, and the embodiments of this disclosure do not limit this.

[0063] After obtaining the judgment information and the set of unambiguous legal provisions involved in each case through step S1011, steps S1012 to S1013 can be used to identify similar cases based on the text similarity between the judgment information and the set of unambiguous legal provisions involved in the case. For example, assuming the judgment information is represented as h, and the set of unambiguous legal provisions is {seq1, seq2, ..., seq...} T In step S1012, the similarity information v between the judgment information of each case and the set of unambiguous legal provisions involved in each case can be calculated using formula (1):

[0064] v=[BM25(seq1,h),BM25(seq2,h),…,BM25(seq T ,h)] (1)

[0065] Among them, BM25 is a model for calculating text similarity, BM25(seq t Let v(t, h) represent the similarity between judgment information h and the t-th unambiguous legal provision, where t ≤ T. v can be intuitively interpreted as a feature of a case at the legal provision level. It should be understood that using the BM25 model to calculate similarity is one possible implementation provided by this disclosure. In fact, those skilled in the art can use known text similarity calculation methods to calculate the similarity between judgment information and unambiguous legal provisions, and this disclosure does not limit this. The similarity information v can be represented as a vector, also known as a similarity vector, or other forms, and this disclosure does not limit this as well.

[0066] It should be understood that, based on the above step S1012, the similarity information corresponding to each case in the preset case library can be obtained, and then, based on the similarity information corresponding to each case, similar cases of each case can be determined. In one possible implementation, assuming that the preset case library includes I cases, where I is a positive integer, in step S1013, based on the similarity information corresponding to each case in the preset case library, similar cases of each case in the preset case library are determined, including:

[0067] For the i-th case in the pre-set case database, J initial similar cases involving the same original legal provisions as the i-th case are determined from the pre-set case database, i≤I, J<I;

[0068] Based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases, determine the relevance weight between the i-th case and each initial similar case;

[0069] Based on the relevance weights between the i-th case and each initial similar case, identify similar cases related to the i-th case from the J initial similar cases.

[0070] It should be understood that there are both similar and dissimilar cases among the multiple cases in the preset case database. In order to quickly determine the similar cases of each case in the preset case database, we can first select multiple initial similar cases from the preset case database that have the same original legal provisions as the i-th case based on the original legal provisions involved in each case. In other words, we can first select some initial similar cases that have intersection with the i-th case using the original legal provisions, and then determine the similar cases of the i-th case from the initial similar cases.

[0071] Since each case may involve at least one original legal provision, it can be set that if any case has the same original legal provision as the original legal provision involved in the i-th case, then the case can be considered as an initial case of the same type as the i-th case. For example, assuming the i-th case involves original legal provisions 1 and 2, and there are cases in the pre-set case library that involve original legal provisions 1 and 2, case C involves original legal provision 1, and case D involves original legal provision 2, then case B, case C, and case D can all be initial cases of the same type as the i-th case.

[0072] In one possible implementation, the determination of the relevance weight between the i-th case and each of the J initial similar cases based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases includes:

[0073] Based on the similarity information corresponding to the i-th case, determine the first unambiguous legal provision with the highest similarity to the judgment information of the i-th case, and based on the similarity information corresponding to the j-th initial similar case among J initial similar cases, determine the second unambiguous legal provision with the highest similarity to the judgment information of the j-th initial similar case, where j≤J;

[0074] If the first unambiguous legal provision and the second unambiguous legal provision are the same, the relevance weight between the i-th case and the j-th initial similar case is determined based on the degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case. The degree of overlap is positively correlated with the relevance weight; or,

[0075] When the first unambiguous legal provision and the second unambiguous legal provision are different, the relevance weight between the i-th case and the j-th initial similar case is determined based on the similarity score and the degree of overlap between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case. The similarity score and the relevance weight are positively correlated.

[0076] The process of determining the relevance weight between the i-th case and the j-th initial similar case can be expressed as formulas (2-1) and (2-2):

[0077]

[0078]

[0079] Among them, w ij A represents the relevance weight between the i-th case and the j-th initial similar case. i Let A represent the set of unambiguous legal provisions involved in the i-th case. j v represents the set of unambiguous legal provisions involved in the j-th initial case of the same type. i v represents the similarity information corresponding to the i-th case. j angmax(v) represents the similarity information corresponding to the j-th initial case of the same type. i ) represents the unambiguous legal provision with the highest similarity among the similarity information corresponding to the i-th case, that is, the first unambiguous legal provision with the highest similarity to the judgment information of the i-th case, angmax(v j ) represents the unambiguous legal provision that has the highest similarity among the similarity information corresponding to the j-th initial similar case, that is, the second unambiguous legal provision that has the highest similarity to the judgment information of the j-th initial similar case, v ih Represents v j The h-th similarity in v jh Represents v j The h-th similarity in the expression, cos(v) ih ,v jh ) represents v j The h-th similarity in v j The cosine similarity between the h-th similarity values ​​is the cosine similarity between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case, where max(cos(v)) = 1. ih ,v jh )) represents v j The h-th similarity in v j The maximum value of the cosine similarity among the h-th similarities in the dataset represents the similarity score between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case. Here, because rel(vi ,v j ) is based on all A i ∩A j The most relevant case is used, so the max function is used to determine the similarity score.

[0080] in, denoted by angmax(v), it represents the degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case. When the first unambiguous legal provision is the same as the second unambiguous legal provision, i.e., angmax(v) i ) = angmax(v j When ), rel(v i ,v j If ) = 1, then the relevance weight That is, at this point, the relevance weight between the i-th case and the j-th initial similar case is determined based on the degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case; when the first unambiguous legal provision and the second unambiguous legal provision are different, i.e., angmax(v i )≠angmax(v j When ), rel(v i ,v j ) = max(cos(v ih ,v jh Then the relevance weight In other words, the relevance weight between case i and case j needs to be determined based on the similarity score and degree of overlap between the similarity information corresponding to case i and the similarity information corresponding to the initial similar case j. In other words, the relevance weight between two cases is mainly determined by the degree of overlap between the sets of unambiguous legal provisions. If both cases are most similar to the same unambiguous legal provision, then these two cases can be considered the most relevant similar cases; otherwise, the relevance weight between the two cases will decrease and depend on the similarity score between the similarity information vi and vj of the two cases.

[0081] It should be understood that the relevance weight between the i-th case and each initial similar case can be obtained through the above formulas (2-1) and (2-2). Then, based on the relevance weight between the i-th case and each initial similar case, similar cases related to the i-th case can be determined from J initial similar cases. That is, at least one legally explanatory similar case can be extracted from multiple initial similar cases based on the relevance weight. Specifically, for example, the initial similar case with the largest relevance weight can be extracted, or the initial similar case with a relevance weight exceeding a preset threshold can also be extracted. This disclosure does not limit this aspect.

[0082] For example, for the i-th case ci, if the case with the highest relevance weight to the i-th case ci is determined to be ci+, then the i-th case ci and the case with the same class ci+ can form a case group in the form of a quadruple (ci,ci+,vi,vi+), where vi represents the similarity information corresponding to the i-th case ci, and vi+ represents the similarity information corresponding to the case with the same class ci+.

[0083] It should be understood that through the above steps S1012 to S1013, similar cases of each case in the preset case database and the similarity information corresponding to each case can be determined. Based on this, in step S1014, at least one case set of at least one training batch can be determined according to the similar cases of each case in the preset case database and the similarity information corresponding to each case. Specifically, multiple case groups can be determined by pre-setting the similar cases of each case in the preset case database and the similarity information corresponding to each case. The multiple case groups are divided into at least one case set according to the preset number of training batches. This embodiment of the present disclosure does not limit the number of case sets or the number of case groups within a case set.

[0084] In step S102, the case information of each case in multiple case groups is encoded using a text encoder to obtain the case features of each case in multiple case groups. This can be understood as inputting the case information of multiple cases in the case set into the text encoder to obtain the case features output by the text encoder. The case features can be represented in the form of vectors, matrices, tensors, etc., and this embodiment of the present disclosure does not limit this.

[0085] The text encoder can be any natural language model in the field, such as BERT, Longformer, Lawformer, Transformer, etc. The embodiments of this disclosure do not limit the model type or model structure used by the text encoder. For example, when the text encoder uses the Transformer model, the vector output by the last hidden layer of the Transformer model can be used as the case feature. For example, the [CLS] vector output by the last hidden layer can be used as the case feature.

[0086] After obtaining the case features of each case in multiple case groups using step S102, in step S103, any known similarity calculation method in the art can be used, such as cosine similarity, to determine the intra-class feature similarity between at least two cases in each case group and the inter-class feature similarity between each at least two cases in each case group and the case of the opposite class, based on the case features of each case in multiple case groups. This disclosure embodiment does not limit this.

[0087] The intra-class feature similarity between at least two cases in each case group can be understood as the intra-class feature similarity between every two cases in each case group; the inter-class feature similarity between at least two cases in each case group and the out-of-class cases can include the inter-class feature similarity between at least two cases in each case group and all or some out-of-class cases in other case groups.

[0088] For example, suppose there are 3 case groups "(c1,c1+,v1,v1+), (c2,c2+,v2,v2+), (c3,c3+,v3,v3+)". The intra-class feature similarity can be the feature similarity between cases c1 and c1+, c2 and c2+, and c3 and c3+. The inter-class feature similarity can be the feature similarity between c1 and at least one of the cases c2, c2+, c3, and c3+. For example, it can be the inter-class feature similarity between c1 and c2, c2+, c3, and c3+, or the inter-class feature similarity between c1 and c2 and c2+, or the feature similarity between c1 and c3 and c3+, or the feature similarity between c1 and c2 and c3, etc.

[0089] Based on the aforementioned intra-class feature similarity, inter-class feature similarity, and the unambiguous legal provisions involved in at least two cases in each case group of the case set, in one possible implementation, in step S104, the contrastive learning loss is determined according to the intra-class feature similarity, inter-class feature similarity, and the relevance information corresponding to at least two cases in each case group of the case set, including:

[0090] Based on the similarity information of at least two cases in each case group, determine the relevance weight between each pair of cases in each case group;

[0091] The first convergence control parameter is determined based on the correlation weight between every two cases in each case group and the intra-class feature similarity between every two cases in each case group.

[0092] The second convergence control parameter is determined based on the inter-class feature similarity between at least two cases in each case group and the out-of-class cases.

[0093] The contrastive learning loss is determined based on intra-class feature similarity, inter-class feature similarity, the first convergence control parameter, and the second convergence control parameter.

[0094] In this process, the relevance weight between each pair of cases in each case group can be determined by referring to the above formulas (2-1) and (2-2) based on the unambiguous legal provisions involved in each of the at least two cases in each case group. It should be understood that the relevance weight between each pair of cases in each case group has actually been calculated in the process of determining similar cases in the above steps S1012 to S1013. Therefore, the relevance weight calculated in the above steps S1012 to S1013 can also be directly used in step S104.

[0095] In this embodiment, a biased circular loss function (BCL) is provided to determine the contrastive learning loss. This BCL can incorporate fine-grained legal knowledge (i.e., the relevance weights determined by the similarity information between judgment information and the set of unambiguous legal provisions) to train the text encoder. For example, formulas (3-1) and (3-2) can be used to determine the contrastive learning loss L based on intra-class feature similarity, inter-class feature similarity, and the relevance weights determined by the similarity information. BCL .

[0096]

[0097]

[0098] in, Represents the feature similarity between the l-th classes. Let L represent the intra-class feature similarity, L represent the total number of inter-class feature similarities, and K represent the total number of intra-class feature similarities. The k-th first convergence control parameter is determined by the similarity of the k-th intra-class feature. The l-th second convergence control parameter, determined by the l-th inter-class feature similarity, is used to control the learning step size of the intra-class feature similarity and inter-class feature similarity, respectively, i.e., to control the convergence speed of the loss function. p γ and Δ represent the relevance weights between two cases corresponding to the inter-class feature similarity of the l-th class. n Δ p O P O n Both are loss functions L BCL Hyperparameters, γ is used to scale the loss, Δ n Δ p Used to control the distance between inter-class feature similarity and intra-class feature similarity respectively, O P O n, respectively, represent the upper boundary of intra-class feature similarity and the lower boundary of inter-class feature similarity, and exp represents an exponential function with the natural constant e as its base.

[0099] It should be understood that those skilled in the art can also use other known contrastive learning loss functions in the art, such as the Circle Loss (CL), to determine the above-mentioned contrastive learning loss, and this disclosure does not limit such actions.

[0100] The training objective of the contrastive learning loss determined using formulas (3-1) and (3-2) is to enable the text editor to have higher similarity between case features extracted from similar cases and lower similarity between cases from dissimilar cases. In other words, the contrastive learning loss allows the text encoder to be trained to bring case features of the same category closer together and push case features of different categories further apart. The distance between case features in the feature space depends on the values ​​of inter-class feature similarity and intra-class feature similarity. Furthermore, by incorporating the relevance weight determined by the similarity information between the judgment information corresponding to the case and the set of unambiguous legal provisions into the contrastive learning loss, the text encoder can learn text relevance at the legal level, which is beneficial for the trained text encoder to output high-quality case features.

[0101] In step S104, training the text encoder based on contrastive learning loss may include: optimizing the model parameters of the text encoder based on contrastive learning loss through backpropagation and gradient descent; wherein, multiple training batches of case sets may be used to iteratively train the text encoder until a well-trained text encoder is obtained.

[0102] According to embodiments of this disclosure, during the training process of the text encoder, the contrastive learning loss is determined by introducing similarity information between the set of unambiguous legal provisions involved in the case and the judgment information of the case. Since the judgment information of the case is highly related to the legal provisions involved, this is equivalent to introducing fine-grained legal knowledge to train the text encoder. This enables the encoder to learn the textual relevance between similar cases in the legal field. At the same time, by using intra-class feature similarity and inter-class feature similarity to determine the contrastive learning loss, the text encoder can be trained to bring case features of the same category closer together and push case features of different categories further apart. This allows the trained text encoder to output high-quality case features. Thus, when using the case features output by the trained text encoder for case retrieval, the accuracy of case retrieval can be effectively improved, and the retrieved similar cases are cases that are interpretable based on legal knowledge.

[0103] It is known that if two cases satisfy both a high degree of similarity between their essential facts and their factual details, the two cases can be considered to be of the same category. That is, when both the essential facts and factual details of one case are highly similar to those of two other cases, then this case can be considered a case of the same category related to the other case. The use of contrastive learning loss described above is equivalent to training a text encoder at the level of essential facts, which enables the case features output by the text encoder to represent essential facts. In order to enable the case features output by the text encoder to simultaneously represent factual details, a Masked Language Modeling (MLM) training task can be used to train the text encoder at the level of factual details. This means that the text encoder can capture the factual details of the case information at the natural language semantic level, which is beneficial for extracting key content from the case information.

[0104] Based on this, in one possible implementation, the case information of each case in the case set is partially masked by removing real words according to a preset masking ratio. The method further includes:

[0105] Step S201: Based on the case characteristics of each case in the case set, predict the real words that are blocked in the case information of each case to obtain the predicted words corresponding to each case.

[0106] Step S202: Determine the masking language loss based on the predicted words corresponding to each case and the masked real words in the case information of each case, and train the text encoder based on the masking language loss.

[0107] In this process, a portion of the real words in the case information can be randomly masked based on a preset masking ratio (e.g., 15%). Then, a text encoder is used to encode the masked case information to obtain case features. Furthermore, in step S201, a text decoder can be used to predict the masked real words in the case information of each case based on the case features corresponding to each case in the case set, thus obtaining the predicted words for each case. In other words, the text decoder can predict the masked words based on the remaining words in the case information. It should be understood that those skilled in the art can use text decoders known in the art to predict the masked real words in the case information of each case. This disclosure does not limit the model type or structure of the text decoder.

[0108] Optionally, in step S202, the masking language loss can be determined based on the difference or similarity between the predicted words corresponding to each case and the masked real words in the case information of each case. For example, the similarity between the predicted words and the real words can be determined based on the above BM25. This embodiment of the present disclosure does not limit this.

[0109] Optionally, in step S202, the masking language loss can be determined by formula (4) based on the predicted words corresponding to each case and the masked real words in the case information of each case; of course, any other known loss function in the art can also be used to determine the masking language loss based on the predicted words and real words, and this embodiment of the present disclosure does not limit this.

[0110]

[0111] Where x represents the input case information, and m(x) represents the predicted blocked words, i.e., the predicted words. Let x represent a specific predicted term, and let x\m(x) represent the remaining terms in the case information. Representative predictive words The probability relative to the remaining words in the case information.

[0112] According to the embodiments of this disclosure, training a text encoder using the above-mentioned masked language loss helps the text encoder understand the contextual relationships in the case information, enabling the text encoder to better extract case features that can represent the facts of the case information. In other words, it enables the text encoder to capture the facts of the case information at the natural language semantic level, which helps the text encoder extract the key content in the case information.

[0113] It should be understood that the text encoder can be trained once using the contrastive learning loss determined in steps S103 to S104 above, and then the text encoder can be trained again using the masked language loss above. Alternatively, the text encoder can be trained once using the masked language loss above, and then the text encoder can be trained again using the contrastive learning loss above. Of course, the text encoder can also be trained using both the contrastive learning loss and the masked language loss simultaneously. For example, the linear combination L of the masked language loss and the contrastive learning loss can be used. total =L MLM +λL BCL λ is a weight parameter used to optimize the text encoder, and this embodiment of the present disclosure does not limit this.

[0114] Figure 3 A schematic diagram of a text encoder training process is shown, such as... Figure 3 As shown, the case set is used as pre-training data and input into the text encoder to obtain case features. The case features are then input into the MLM training module and the contrastive learning training module, respectively, and the total loss L is output. total =L MLM +λL BCL The text encoder is optimized using backpropagation based on the total loss.

[0115] It should be understood that the MLM training module can be implemented in accordance with the above steps S201 to S202, and the contrastive learning training module can be implemented in accordance with the above steps S103 to S104, which will not be elaborated here.

[0116] In practical applications, the text encoder trained using the text encoder training method of the present disclosure embodiments described above can be deployed in various electronic devices (such as terminal devices or servers) to achieve the task of case retrieval. Based on this, the present disclosure embodiments also provide a case retrieval method, which includes:

[0117] Obtain case information of the target case to be searched;

[0118] The case information of the target case is encoded using a text encoder to obtain the case features corresponding to the target case. The text encoder is trained using the text encoder training method described above.

[0119] Based on the case characteristics corresponding to the target case and the case characteristics of multiple adjudicated cases in the target case database, similar cases to the target case are identified from the target case database.

[0120] Among them, adjudicated cases can be understood as cases with existing judgments. The characteristics of adjudicated cases can be obtained by pre-encoding the case information of adjudicated cases using the aforementioned text encoder. This enables a dual-tower architecture for case retrieval, resulting in fast retrieval speed. The case retrieval method of this disclosure can be applied to the recall stage of a retrieval task, equivalent to preliminary rapid screening.

[0121] In one possible implementation, based on the case features corresponding to the target case and the case features of multiple adjudicated cases in the target case database, similar cases to the target case are determined from the target case database. For example, this may include: calculating the feature similarity between the case features corresponding to the target case and the case features of each adjudicated case in the target case database, and taking the adjudicated case with the highest feature similarity as a similar case to the target case. Alternatively, adjudicated cases with feature similarity exceeding a preset threshold may also be taken as similar cases to the target case.

[0122] In practical applications, since the case features generated by the text encoder are generally applicable in the judicial field, they can also be used for other judicial tasks, such as legal provision prediction. In one possible implementation, based on the case features corresponding to the target case, the original legal provisions and / or unambiguous legal provisions involved in the target case are predicted. This can be achieved using prediction models known in the art, which allow for the prediction of the original legal provisions and / or unambiguous legal provisions involved in the target case based on the case features. This disclosure does not limit the model structure, model type, or training method of the prediction model.

[0123] The case retrieval method according to the embodiments of this disclosure can utilize the high-quality case features output by the above-mentioned text encoder, that is, use case features that can characterize the essential facts and / or case facts to retrieve more accurate similar cases, improve the accuracy of similar case retrieval, and also make the retrieved similar cases legally interpretable. At the same time, the case features can also be applied to various judicial tasks, with high generalization and a wider range of application scenarios.

[0124] Figure 4 A block diagram of a text encoder training apparatus according to an embodiment of the present disclosure is shown. Figure 4 As shown, the text encoder training device includes:

[0125] The acquisition module 401 is used to acquire a case set, which includes multiple case groups. Each case group includes: case information of at least two cases that are of the same type and similarity information corresponding to each of the at least two cases. The similarity information is determined based on the similarity between the judgment information of the case and the set of unambiguous legal provisions involved in the case. Cases in different case groups are heterogeneous cases. The set of unambiguous legal provisions includes at least one unambiguous legal provision.

[0126] The encoding module 402 is used to encode the case information of each case in the plurality of case groups using a text encoder to obtain the case features of each case in the plurality of case groups.

[0127] The determination module 403 is used to determine, based on the case characteristics of each case in the plurality of case groups, the intra-class feature similarity between at least two cases in each case group, and the inter-class feature similarity between each at least two cases in each case group and the case of the opposite class.

[0128] The training module 404 is used to determine the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set, and to train the text encoder based on the contrastive learning loss.

[0129] In one possible implementation, obtaining the case set includes: obtaining judgment information for each case in a plurality of cases in a preset case database, and obtaining a set of unambiguous legal provisions involved in each case in the preset case database, wherein the judgment information includes the reasons for the judgment of the case; determining similarity information corresponding to each case in the preset case database based on the similarity between the judgment information of each case in the preset case database and each unambiguous legal provision in the set of unambiguous legal provisions involved in each case; determining similar cases of the same type as each case in the preset case database based on the similar cases of each case in the preset case database and the similarity information corresponding to each case; and determining at least one case set for at least one training batch based on the similar cases of each case in the preset case database and the similarity information corresponding to each case.

[0130] In one possible implementation, obtaining the set of unambiguous legal provisions involved in each case in the preset case database includes: obtaining multiple original legal provisions involved in multiple cases in the preset case database, with each case involving at least one original legal provision; splitting each original legal provision into at least one branch clause, and extracting keywords from each branch clause to obtain at least one unambiguous legal provision under each original legal provision; and determining the set of unambiguous legal provisions involved in each case in the preset case database based on the at least one original legal provision involved in each case and the at least one unambiguous legal provision under each original legal provision.

[0131] In one possible implementation, the preset case database includes I cases, where I is a positive integer. The step of determining similar cases for each case in the preset case database based on the similarity information of each case includes: for the i-th case in the preset case database, determining J initial similar cases (i < I, J < I) that involve the same original legal provisions as the i-th case; determining the relevance weight between the i-th case and each initial similar case based on the similarity information corresponding to the i-th case and the J initial similar cases; and determining similar cases related to the i-th case from the J initial similar cases based on the relevance weight between the i-th case and each initial similar case.

[0132] In one possible implementation, determining the relevance weight between the i-th case and each of the J initial similar cases based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases includes: determining a first unambiguous legal provision with the highest similarity to the judgment information of the i-th case based on the similarity information corresponding to the i-th case; and determining a second unambiguous legal provision with the highest similarity to the judgment information of the j-th initial similar case based on the similarity information corresponding to the j-th initial similar case, where j < J; and when the first unambiguous legal provision and the second unambiguous legal provision are the same, based on the... The degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case determines the relevance weight between the i-th case and the j-th initial similar case, wherein the degree of overlap is positively correlated with the relevance weight; or, if the first unambiguous legal provision and the second unambiguous legal provision are not the same, the relevance weight between the i-th case and the j-th initial similar case is determined based on the similarity score between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case, and the degree of overlap, wherein the similarity score is positively correlated with the relevance weight.

[0133] In one possible implementation, determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set includes: determining the relevance weight between each pair of cases in each case group based on the similarity information corresponding to at least two cases in each case group; determining a first convergence control parameter based on the relevance weight between each pair of cases in each case group and the intra-class feature similarity between each pair of cases in each case group; determining a second convergence control parameter based on the inter-class feature similarity between each of at least two cases in each case group and cases of the opposite class; and determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, the first convergence control parameter, and the second convergence control parameter.

[0134] In one possible implementation, the case information of each case in the case set is partially masked out according to a preset masking ratio. The device further includes: a vocabulary prediction module, used to predict the masked real words in the case information of each case based on the case features corresponding to each case in the case set, and obtain the predicted vocabulary corresponding to each case; and a model training module, used to determine the masking language loss based on the predicted vocabulary corresponding to each case and the masked real words in the case information of each case, and train the text encoder based on the masking language loss.

[0135] According to the text encoder training apparatus of this disclosure, during the training process of the text encoder, the contrastive learning loss is determined by introducing similarity information between the set of unambiguous legal provisions involved in the case and the judgment information of the case. Since the judgment information of the case is highly related to the legal provisions involved, this is equivalent to introducing fine-grained legal knowledge to train the text encoder. This enables the encoder to learn the textual relevance between similar cases in the legal field. At the same time, the contrastive learning loss is determined by using intra-class feature similarity and inter-class feature similarity. This allows the text encoder to be trained to bring case features of the same category closer together and push case features of different categories further apart. Thus, the trained text encoder can output high-quality case features. In this way, when using the case features output by the trained text encoder for case retrieval, the accuracy of case retrieval can be effectively improved, and the retrieved similar cases are cases that are interpretable based on legal knowledge.

[0136] This disclosure also provides a case retrieval device, comprising: an information acquisition module for acquiring case information of a target case to be retrieved; an information encoding module for encoding the case information of the target case using a text encoder to obtain case features corresponding to the target case, wherein the text encoder is trained using the aforementioned text encoder training method; and a case determination module for determining similar cases to the target case from the target case database based on the case features corresponding to the target case and the case features of multiple adjudicated cases in the target case database.

[0137] In one possible implementation, the case retrieval device further includes a legal provision prediction module, used to predict the original legal provisions and / or unambiguous legal provisions involved in the target case based on the case characteristics corresponding to the target case.

[0138] According to the embodiments of this disclosure, the case retrieval device can utilize the high-quality case features output by the text encoder, that is, the case features that characterize the essential facts and / or the facts of the case, to retrieve more accurate similar cases, improve the accuracy of case retrieval, and make the retrieved similar cases legally interpretable. At the same time, the case features can be applied to various judicial tasks, and have a high generalization ability and a wider range of application scenarios.

[0139] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0140] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0141] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0142] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0143] Figure 5 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. The electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 5 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the aforementioned text encoder training method and / or case retrieval method.

[0144] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0145] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to complete the text encoder training method and / or case retrieval method described above.

[0146] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0147] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0148] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0149] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0150] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0151] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0152] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0154] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A text encoder training method, characterized in that, include: Obtain a case set, which includes multiple case groups. Each case group includes: case information of at least two cases that are of the same type and similarity information corresponding to each of the at least two cases. The similarity information is determined based on the similarity between the judgment information of the case and the set of unambiguous legal provisions involved in the case. Cases in different case groups are dissimilar to each other. The set of unambiguous legal provisions includes at least one unambiguous legal provision. The case information of each case in the multiple case groups is encoded using a text encoder to obtain the case characteristics of each case in the multiple case groups; Based on the case characteristics of each case in the plurality of case groups, determine the intra-class feature similarity between at least two cases in each case group, and the inter-class feature similarity between each at least two cases in each case group and out-of-class cases; wherein, the intra-class feature similarity between at least two cases in each case group includes the feature similarity of case characteristics between every two cases in each case group; the inter-class feature similarity between each at least two cases in each case group and out-of-class cases includes the feature similarity of case characteristics between each at least two cases in each case group and all or some out-of-class cases in other case groups; Based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set, the contrastive learning loss is determined, and the text encoder is trained based on the contrastive learning loss. The step of determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set includes: Based on the similarity information of at least two cases in each case group, determine the relevance weight between each pair of cases in each case group; The first convergence control parameter is determined based on the correlation weight between every two cases in each case group and the intra-class feature similarity between every two cases in each case group. The second convergence control parameter is determined based on the inter-class feature similarity between at least two cases in each case group and the out-of-class cases. The contrastive learning loss is determined based on the intra-class feature similarity, the inter-class feature similarity, the first convergence control parameter, and the second convergence control parameter.

2. The method according to claim 1, characterized in that, The acquisition of the case set includes: Obtain the judgment information of each case in a preset case database, and obtain the set of unambiguous legal provisions involved in each case in the preset case database, wherein the judgment information includes the reasons for the judgment of the case; Based on the similarity between the judgment information of each case in the preset case database and the unambiguous legal provisions in the set of unambiguous legal provisions involved in each case, the similarity information corresponding to each case in the preset case database is determined. Based on the similarity information corresponding to each case in the preset case database, determine the similar cases of each case in the preset case database; Based on the similar cases of each case in the preset case database and the similarity information corresponding to each case, at least one case set of at least one training batch is determined.

3. The method according to claim 2, characterized in that, The step of obtaining the set of unambiguous legal provisions related to each case in the preset case database includes: Obtain multiple original legal provisions related to multiple cases in the preset case database, with each case involving at least one original legal provision; Each original legal provision is broken down into at least one branch clause, and keywords are extracted from each branch clause to obtain at least one unambiguous legal provision under each original legal provision. Based on at least one original legal provision involved in each case in the preset case database and at least one unambiguous legal provision under each original legal provision, determine the set of unambiguous legal provisions involved in each case in the preset case database.

4. The method according to claim 2, characterized in that, The preset case database includes I cases, where I is a positive integer. The step of determining similar cases in the preset case database based on the similarity information corresponding to each case includes: For the i-th case in the preset case database, J initial similar cases involving the same original legal provisions as the i-th case are determined from the preset case database, i≤I, J<I; Based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases, the relevance weight between the i-th case and each initial similar case is determined; Based on the relevance weight between the i-th case and each initial similar case, similar cases related to the i-th case are determined from the J initial similar cases.

5. The method according to claim 4, characterized in that, The step of determining the relevance weight between the i-th case and each of the J initial similar cases based on the similarity information corresponding to the i-th case and the similarity information corresponding to the J initial similar cases includes: Based on the similarity information corresponding to the i-th case, determine the first unambiguous legal provision with the greatest similarity to the judgment information of the i-th case, and based on the similarity information corresponding to the j-th initial similar case among the J initial similar cases, determine the second unambiguous legal provision with the greatest similarity to the judgment information of the j-th initial similar case, where j≤J; When the first unambiguous legal provision is the same as the second unambiguous legal provision, the relevance weight between the i-th case and the j-th initial similar case is determined based on the degree of overlap between the set of unambiguous legal provisions involved in the i-th case and the set of unambiguous legal provisions involved in the j-th initial similar case, wherein the degree of overlap is positively correlated with the relevance weight; or, When the first unambiguous legal provision and the second unambiguous legal provision are different, the relevance weight between the i-th case and the j-th initial similar case is determined based on the similarity score between the similarity information corresponding to the i-th case and the similarity information corresponding to the j-th initial similar case, and the degree of overlap. The similarity score is positively correlated with the relevance weight.

6. The method according to claim 1, characterized in that, The case information of each case in the case set is partially masked according to a preset masking ratio, and the method further includes: Based on the case characteristics of each case in the case set, the real words that are blocked in the case information of each case are predicted, and the predicted words corresponding to each case are obtained. Based on the predicted words corresponding to each case and the real words that are masked in the case information of each case, the masked language loss is determined, and the text encoder is trained based on the masked language loss.

7. A method for retrieving similar cases, characterized in that, The method includes: Obtain case information of the target case to be searched; The case information of the target case is encoded using a text encoder to obtain the case features corresponding to the target case, wherein the text encoder is trained by the text encoder training method according to any one of claims 1 to 6; Based on the case characteristics corresponding to the target case and the case characteristics of multiple adjudicated cases in the target case database, similar cases to the target case are identified from the target case database.

8. The method according to claim 7, characterized in that, The method further includes: Based on the case characteristics corresponding to the target case, predict the original legal provisions and / or unambiguous legal provisions involved in the target case.

9. A text encoder training device, characterized in that, include: The acquisition module is used to acquire a case set, which includes multiple case groups. Each case group includes: case information of at least two cases that are of the same type and similarity information corresponding to each of the at least two cases. The similarity information is determined based on the similarity between the judgment information of the case and the set of unambiguous legal provisions involved in the case. Cases in different case groups are dissimilar to each other. The set of unambiguous legal provisions includes at least one unambiguous legal provision. The encoding module is used to encode the case information of each case in the multiple case groups using a text encoder to obtain the case features of each case in the multiple case groups. The determination module is used to determine, based on the case characteristics of each case in the plurality of case groups, the intra-class feature similarity between at least two cases in each case group, and the inter-class feature similarity between each of the at least two cases in each case group and out-of-class cases; wherein, the intra-class feature similarity between at least two cases in each case group includes the feature similarity of case characteristics between every two cases in each case group; the inter-class feature similarity between each of the at least two cases in each case group and out-of-class cases includes the feature similarity of case characteristics between each of the at least two cases in each case group and all or some out-of-class cases in other case groups; The training module is used to determine the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set, and to train the text encoder based on the contrastive learning loss; The step of determining the contrastive learning loss based on the intra-class feature similarity, the inter-class feature similarity, and the similarity information corresponding to at least two cases in each case group of the case set includes: Based on the similarity information of at least two cases in each case group, determine the relevance weight between each pair of cases in each case group; The first convergence control parameter is determined based on the correlation weight between every two cases in each case group and the intra-class feature similarity between every two cases in each case group. The second convergence control parameter is determined based on the inter-class feature similarity between at least two cases in each case group and the out-of-class cases. The contrastive learning loss is determined based on the intra-class feature similarity, the inter-class feature similarity, the first convergence control parameter, and the second convergence control parameter.

10. A case retrieval device, characterized in that, include: The information acquisition module is used to acquire case information of the target case to be searched; An information encoding module is used to encode the case information of the target case using a text encoder to obtain the case features corresponding to the target case, wherein the text encoder is trained by the text encoder training method according to any one of claims 1 to 6; The similarity determination module is used to determine similar cases from the target case database based on the case characteristics corresponding to the target case and the case characteristics of multiple adjudicated cases in the target case database.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 8 when executing instructions stored in the memory.

12. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 8.

Images