An administrative law enforcement post-event response effectiveness evaluation method and system

Abstract

The application relates to the technical field of data processing and text analysis, and provides an administrative law enforcement post-event response effectiveness evaluation method and system. The method comprises the following steps: analyzing administrative law enforcement post-event response related documents from multiple data sources based on a preset response element rule set, a target extraction model and a target reasoning model to obtain single-case structured response index data; homologous case collection is performed on the structured response index data of each case to identify records pointing to the same law enforcement event; for the conflicting fields existing in the same law enforcement event after the collection, a corresponding calibration strategy is selected according to a preset field type to determine a standard value, so that a standardized evaluation index library is formed; and the post-event response effectiveness of law enforcement subjects is quantitatively evaluated based on the standardized evaluation index library to obtain an evaluation score. The application automatically analyzes multi-source documents and calibrates conflicting value taking according to field types, and the evaluation result is used to determine corresponding law enforcement subjects.

Description

Technical Field

[0001] This invention relates to the field of data processing and text parsing technology, and in particular to a method and system for evaluating the effectiveness of post-event response to administrative law enforcement. Background Technology

[0002] In evaluating the effectiveness of post-event responses to administrative law enforcement, existing technologies typically employ: Manual case review, where relevant personnel review public complaint forms, administrative review decisions, and court judgments, manually compiling statistics on issues such as overdue responses and revocations of reviews, and relying on experience to create an assessment log. A general deduction scoring method is used, setting basic scoring items such as fairness, effectiveness, standardization, and scientific rigor, matching violation deduction factors, and deducting deduction items from a preset total score to obtain the final score, calculated based on a structured business log. A text extraction and statistical early warning method is also employed, extracting elements such as the complainant and address from public complaint texts through entity extraction, using vector matching to target objects and clustering to form complaint hotspots and early warning information.

[0003] In actual assessments, the same enforcement matter can generate multiple independent records across public complaint work orders, administrative review decisions, administrative litigation documents, and enforcement business ledgers. Key information such as response deadlines, review results, and dispute resolution status are presented in various ways, and the values ​​recorded for the same matter from different sources are often inconsistent. Manual review requires reviewing and extracting these documents one by one, which is time-consuming and susceptible to the subjective influence of the reviewers. General scoring rules are geared towards on-site enforcement scenarios and do not cover post-event responses, dispute resolution, and administrative review / litigation handling. When values ​​from different sources are inconsistent, only records from a single source or manually selected records can be used. This situation makes the assessment reliant on manual data processing; differences in values ​​from different sources directly affect the assessment results, making it difficult to automatically obtain standardized and verifiable response indicators. Furthermore, the assessment results are unlikely to drive improvements in post-event responses. Summary of the Invention

[0004] Based on this, it is necessary to propose an evaluation method and system for the effectiveness of post-event response to administrative law enforcement, addressing the technical problems of relying on manual processing of documents from multiple sources, the impact of differences in values ​​from various sources on the evaluation results, and the difficulty in automatically obtaining evaluation results with consistent standards and verifiable basis.

[0005] Firstly, a method for evaluating the effectiveness of post-event responses to administrative law enforcement is provided, including the following steps: The administrative law enforcement response documents from multiple data sources are analyzed based on a pre-set set of response element rules, a target extraction model, and a target reasoning model to obtain structured response indicator data for each case. The structured response indicator data of each case are grouped together to identify records that point to the same law enforcement event; For fields that conflict within the same law enforcement event after aggregation, the corresponding calibration strategy is selected according to the preset field type to determine the standard value, so as to form a standardized evaluation index library; Based on a standardized evaluation index library, the effectiveness of law enforcement entities' post-event response is quantitatively evaluated to obtain an evaluation score, which is used to determine the corresponding law enforcement entity.

[0006] Secondly, an evaluation system for the effectiveness of post-event response to administrative law enforcement is provided, including a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned evaluation method for the effectiveness of post-event response to administrative law enforcement.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-mentioned method for evaluating the effectiveness of post-event response to administrative law enforcement.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the aforementioned method for evaluating the effectiveness of post-event responses to administrative law enforcement.

[0009] The evaluation method and system for the effectiveness of post-event response to administrative law enforcement provided in this application analyzes relevant documents from multiple data sources. It transforms unstructured documents from various sources, such as public complaint work orders, administrative reconsideration decisions, administrative litigation documents, and law enforcement business ledgers, into structured response indicator data for each case with stable field structure and traceable source. The response indicators no longer rely on manual extraction of each document, providing a unified input that can be processed by machines for subsequent evaluation.

[0010] By grouping structured response indicator data from various cases into cases with the same source, records pointing to the same law enforcement event are identified. Records scattered from different sources but belonging to the same law enforcement event are grouped into the same event group. Conflict handling and indicator statistics are carried out within the same event group, so that data from different cases are no longer mixed up and the assessment criteria remain consistent at the event level.

[0011] By determining standard values ​​for conflicting fields within the same law enforcement event after aggregation, and processing fields with inconsistent values ​​from different sources within the same law enforcement event into standard values ​​through semantic matching, conflicting values ​​are no longer covered by a single source or simply averaged. Multi-source data form a standardized evaluation indicator library with unified caliber and verifiable sources, ensuring the accuracy of the evaluation input data.

[0012] By using a standardized evaluation indicator library, the effectiveness of law enforcement entities' post-event response is quantitatively assessed, resulting in an evaluation score. This score is used to identify the corresponding law enforcement entities. The evaluation is based on standardized and verifiable indicators. The evaluation results objectively reflect the performance of each law enforcement entity in post-event responses, dispute resolution, and handling of administrative review and litigation. Law enforcement entities with insufficient responses are directly identified by the evaluation results, creating a connection between evaluation and improvement. This process frees the evaluation of the effectiveness of administrative law enforcement post-event response from reliance on manual data processing. Differences in values ​​from multiple data sources no longer directly interfere with the evaluation results. The evaluation score is based on standardized and verifiable response indicators, and the evaluation results can be directly used to identify law enforcement entities that need rectification, thereby promoting improvements in the quality of post-event responses. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] in: Figure 1 This is a flowchart illustrating the method for evaluating the effectiveness of post-event response to administrative law enforcement in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of the evaluation system for the effectiveness of post-event response to administrative law enforcement in an embodiment of the present invention; Figure 3 This is a structural block diagram of the computer device server in an embodiment of the present invention; Figure 4 This is a structural block diagram of the computer device client in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 1 As shown, Figure 1 A flowchart illustrating the method for evaluating the effectiveness of post-event response to administrative law enforcement provided in this embodiment of the invention includes the following steps: Step 01: Analyze the administrative law enforcement response documents from multiple data sources based on the preset response element rule set, target extraction model and target reasoning model to obtain structured response indicator data for each case; Step 02: Collect cases with the same source from the structured response indicator data of each case and identify records that point to the same law enforcement event; Step 03: For fields that conflict within the same law enforcement event after aggregation, select the corresponding calibration strategy according to the preset field type to determine the standard value, so as to form a standardized evaluation index library; Step 04: Based on the standardized evaluation index library, the effectiveness of the law enforcement entity's post-event response is quantitatively evaluated to obtain an evaluation score, which is used to determine the corresponding law enforcement entity.

[0017] Specifically, in step 01, multiple data sources can be the origins of documents related to post-event responses to administrative law enforcement. These data sources can include public complaint platforms, administrative review agencies, judicial organs, and internal systems of law enforcement agencies. The document types and record formats output by different data sources vary. Documents related to post-event responses to administrative law enforcement can be public complaint work orders, administrative review decisions, administrative litigation documents, law enforcement business ledgers, and other documents recording post-event response information. The response element rule set can be a predefined set of element extraction rules, which limits the field boundaries, value constraints, and normalization methods of the elements to be extracted. The target extraction model can be a model that has been trained and used for batch extraction of element fields. The target inference model can be a model deployed on the server and involved in parsing, used to verify and confirm the extraction results.

[0018] Structured response indicator data for a single case can be structured indicator data corresponding to a single case. The structured response indicator data for a single case includes the values ​​of response element fields determined from the corresponding case documents. Step 01 parses the multi-source, unstructured response-related documents into single-case data with stable field structures for subsequent collection and evaluation.

[0019] In step 02, the same-source case aggregation can be a process of determining whether structured response indicator data from different sources point to the same law enforcement event. The same law enforcement event will generate multiple records from multiple data sources. Same-source case aggregation identifies and aggregates records pointing to the same law enforcement event, isolating data from different cases. Step 02 unifies the data organization method at the event level, providing a scope basis for conflict resolution within the same law enforcement event.

[0020] In step 03, conflicting fields can be fields with inconsistent values ​​from different sources within the same law enforcement event. Preset field types can be pre-configured field categories, used to select corresponding conflict handling methods. Calibration strategies can be conflict value handling methods determined for specific field types, and standard values ​​can be uniform values ​​determined after processing the conflicting fields. The standardized evaluation indicator library can be a set of evaluation indicators with uniform field values ​​and retained source information. Step 03 processes conflicting values ​​from different sources within the same law enforcement event into standard values ​​according to field type, forming a standardized evaluation indicator library.

[0021] In step 04, the enforcement entity can be a unit or individual undertaking administrative enforcement functions and making a post-event response. Quantitative assessment can be a method of calculating scores for the effectiveness of the enforcement entity's post-event response based on indicators. The assessment score can be a comprehensive score output by the quantitative assessment, reflecting the level of effectiveness of the enforcement entity's post-event response. The corresponding enforcement entity can be one whose assessment score meets preset conditions and requires rectification. Step 04 performs the assessment based on a standardized assessment indicator library. The assessment score is used to determine the corresponding enforcement entity, and the assessment results serve to improve post-event response work.

[0022] In some embodiments, step 01 above includes: Obtain relevant documents regarding the post-enforcement response; Standardized preprocessing is performed on documents related to post-enforcement responses to administrative law enforcement to obtain standard texts; Under the constraints of the response feature rule set, standard text is input into the target extraction model to obtain candidate feature fields; Based on the response element rule set, value constraint validation and synonym normalization are performed on the candidate element fields to obtain the pre-validated element fields. Input the standard text and pre-validated feature fields into the target inference model, and perform error correction validation on the pre-validated feature fields based on the original content of the standard text to obtain the corrected feature fields; Based on the revised feature fields, generate structured response indicator data for each case.

[0023] Specifically, when obtaining relevant documents for post-enforcement responses, documents from sources such as administrative reconsideration decisions, administrative litigation documents, public complaint work orders, and enforcement business ledgers can be collected simultaneously. Documents from different sources can exist in the form of text, images, or audio / video transcriptions. Scanned documents can be converted into processable text through optical character recognition. The obtained documents are associated with case identifiers for subsequent source tracing of indicator data.

[0024] Standardized preprocessing is applied to post-enforcement response documents to obtain standard text. This preprocessing may include removing redundant content such as headers, footers, garbled characters, and unnecessary line breaks, and converting documents of different formats into plain text. The standardized document text serves as the standard text. Structured records such as enforcement business ledgers can also be converted into a standard text format consistent with the documents for parsing.

[0025] Under the constraints of the response element rule set, standard text is input into the target extraction model to obtain candidate element fields. The response element rule set covers extraction rules for elements such as the complainant, response deadline, reconsideration results, dispute resolution status, duplicate complaint identifiers, and case-handling units and personnel. The response element rule set consists of field extraction rules and general processing rules. Field extraction rules are configured according to fields, value constraints, and synonym normalization triples. Fields limit the name and semantic boundaries of the elements to be extracted. Value constraints limit the trigger expression, evidence location, data type, legal values, and missing value handling methods of fields. Synonym normalization is used to map different textual expressions of the same matter to a unified field value.

[0026] The general processing rules include evidence binding rules, missing value rules, internal document conflict rules, and cross-source conflict transfer rules. Evidence binding rules require that each extraction result be associated with the original text evidence fragment and its location. Missing value rules require that fields not explicitly recorded in the original text be marked as unstated and prohibit inference or supplementation. Internal document conflict rules stipulate that when multiple candidate values ​​appear in the same document, they should be determined in the order of determining the main text, clarifying the content of the determination, stating the facts, and stating the parties' statements. Cross-source conflict transfer rules require that when different sources record the same field inconsistently, the original value, source identifier, and evidence fragment of each source should be retained and transferred for subsequent collection and conflict handling.

[0027] The target extraction model performs batch parsing of standard text under the constraints of the response feature rule set, and outputs candidate feature fields. The response feature rule set limits the field boundaries and value formats output by the target extraction model.

[0028] Based on the response element rule set, value constraint validation and synonym normalization are performed on candidate element fields to obtain pre-validated element fields. Value constraint validation checks whether candidate element fields meet the trigger expression and value constraints; candidate values ​​that do not meet the constraints are marked as semantically conflicting or unspecified. Synonym normalization maps different textual expressions to unified field values. For example, expressions such as "revoke the original administrative act" and "revoke the original decision" are uniformly mapped to "reconsideration revocation," and expressions such as "response deadline," "response deadline," and "completion period" are uniformly mapped to the "response deadline time" field. Date-type values ​​can be uniformly converted to standard date formats. Candidate element fields that have undergone value constraint validation and synonym normalization serve as pre-validated element fields, and these pre-validated element fields are bound to corresponding original text evidence fragments.

[0029] The target inference model inputs standard text and pre-validated element fields. Based on the original content of the standard text, it performs error correction and validation on the pre-validated element fields to obtain the corrected element fields. The target inference model can be deployed on a server, using the original content of the standard text as evidence to compare and verify each pre-validated element field, correcting values ​​inconsistent with the original text, deleting values ​​lacking supporting evidence, and verifying the correctness of synonym normalization. When performing error correction and validation, the target inference model is limited to correcting information solely based on the original text content, without supplementing case information based on irrelevant knowledge or inferences. The error correction and validation process is executed in batches on the server, and the resulting corrections are used as the corrected element fields.

[0030] After the revised element fields are aggregated, structured response indicator data for each case is generated. This data includes the revised element fields and their corresponding values. Basic fields for each case may include case number, enforcement body, summary of the disputed matter, response deadline, review result, and dispute resolution status. Additional elements such as the review application date and review decision date may be added based on the document content. The case number can only be obtained from document metadata, business system-related information, or case aggregation results; it is not generated by the model. Each element field in the structured response indicator data for each case is associated with a fragment of original evidence for source tracing. This step outputs structured data for cases from a single source; cross-source data conflicts are handled by subsequent aggregation and conflict calibration of cases from the same source.

[0031] In some implementations, the error correction verification in step 01 above includes: Compare each pre-validated feature field with the original text content in the standard text; When the pre-validation feature field is inconsistent with the corresponding original text content, the pre-validation feature field is corrected according to the original text content. Delete the pre-validation feature field if the pre-validation feature field lacks corresponding original text content in the standard text. The retained pre-validated feature fields and the corrected pre-validated feature fields are combined to obtain the corrected feature fields.

[0032] Specifically, each pre-verification element field is compared with the original content in the standard text. During error correction and verification, the target inference model performs evidence retrieval for each pre-verification element field, locating the original content of the corresponding item in the standard text, and comparing the value of the pre-verification element field with the original content item by item. The comparison scope covers the original expression, semantic content, and evidence location corresponding to the field value. If the original content corresponding to a pre-verification element field is missing, it can be determined that the pre-verification element field lacks original textual evidence.

[0033] When a pre-validation element field is inconsistent with the corresponding original text content, the pre-validation element field is corrected according to the original text content. Inconsistencies may include discrepancies in time descriptions, semantic ambiguity in addresses, text recognition errors, and erroneous values ​​generated by model illusions. During correction, the pre-validation element field is adjusted based on the original text content in the standard text. For example, a misidentified date is corrected to the actual date recorded in the original text, and a semantically ambiguous place name is corrected to the location indicated in the original text. The corrected field retains the original text evidence fragment.

[0034] If a pre-validation feature field lacks corresponding source text content in the standard text, the pre-validation feature field is deleted. If a pre-validation feature field cannot be found in the standard text, or if the source text content is insufficient to support the value of the pre-validation feature field, the pre-validation feature field is determined to be a field lacking source text support and is deleted. Deleted objects include false information generated out of thin air by the model and values ​​that cannot be supported by source text. No inference content is added after deletion.

[0035] During the aggregation process, pre-validated element fields that have not undergone correction after comparison and have passed the verification are retained and merged with the corrected pre-validated element fields into corrected element fields. All corrected element fields are based on the original text content and serve as the field source for the structured response indicator data for individual cases.

[0036] In one implementation, the parsing process is illustrated using an administrative reconsideration decision as an example. The standard text states that an applicant, dissatisfied with a district urban management bureau's decision to demolish a building within a specified time, applied for administrative reconsideration, and the reconsideration authority issued a reconsideration decision revoking the original decision. When the target extraction model outputs candidate element fields, it misidentifies the date of the demolition decision as the response deadline, retaining the reconsideration result as the revocation of the original decision without normalization. Value constraint verification reveals that the date of issuance does not meet the trigger expression for the response deadline, marking the date of issuance as a semantic conflict; synonym normalization maps the revocation of the original decision to the reconsideration revocation; the target reasoning model, based on the original text, confirms that the date of issuance is not the response deadline, and that the original text does not record a deadline directly related to the response obligation, correcting the response deadline to "not specified," and confirming that the revocation of the reconsideration is consistent with the main text of the original decision.

[0037] The final generated structured response indicator data for a single case includes fields such as the enforcement entity, a summary of the disputed matter, a response deadline that is not specified, and a reconsideration result that is revoked. Each field is associated with fragments of the original evidence.

[0038] In some embodiments, step 02 above includes: Extract the common source clustering characteristics of the structured response indicator data of each case. The common source clustering characteristics include case number, law enforcement entity, event time and request. When the case numbers of structured response indicator data from different sources are consistent, the structured response indicator data from different sources are identified as records of the same law enforcement event; When structured response indicator data from different sources lack consistent case numbers, calculate the similarity of structured response indicator data from different sources across each common-source aggregation feature dimension; The similarity scores across the various origin-based feature dimensions are weighted and fused according to preset matching feature weights to obtain a comprehensive similarity score. When the overall similarity reaches the first preset threshold, structured response indicator data from different sources are identified as records of the same law enforcement event; When the overall similarity is lower than the first preset threshold and reaches the second preset threshold, the corresponding structured response index data will be marked as a record to be reviewed from the same source.

[0039] Specifically, the common-source aggregation features of the structured response indicator data for each case are extracted. These features include case number, enforcement entity, event time, and the nature of the complaint. These features determine whether records from different sources point to the same enforcement event. The case number is the identifier of the enforcement event in the system; the enforcement entity is the entity that took enforcement action and assumes the responsibility for responding; the event time is the time when the enforcement event occurred or was handled; and the matter being complained about, appealed, or litigated is the content of the complaint. In another implementation, the common-source aggregation features may also include characteristics such as the case handlers, the parties involved, and the location of the incident, to further characterize the attributes of the enforcement event.

[0040] When case numbers from structured response indicator data from different sources are identical, the structured response indicator data from different sources are identified as records of the same law enforcement event. The case number is a strong identifier of the law enforcement event. When records from different sources contain the same case number that passes verification, the two records are directly determined to point to the same law enforcement event and are grouped into the same event group. When case numbers from different sources are different and both pass verification, they are not directly merged but marked as strong identifier conflicts. Further processing is conducted when there is clear evidence of case merging, related cases, or number changes.

[0041] When structured response indicator data from different sources lacks a consistent case number, the similarity of the structured response indicator data from different sources across each source aggregation feature dimension is calculated. Records lacking a unified case number or whose source relationship cannot be determined solely by the case number are similarized according to the source aggregation feature dimensions. The similarity across these dimensions includes at least the subject and case identifier dimension, the time dimension, and the matter and address dimension. The subject and case identifier dimension covers the degree of matching between law enforcement agencies, case handlers, parties involved, and case numbers; the time dimension covers the degree of matching between the time of the incident or the time of processing; and the matter and address dimension covers the degree of matching between the semantics of the claim and the address of the incident.

[0042] The text of the item can be calculated using vector cosine similarity. The number, subject, address, etc. can be calculated using exact matching, edit distance, or address level matching. Time can be mapped to similarity according to date intervals. The similarity value of each dimension is between 0 and 1.

[0043] The similarity scores across all common-origin feature dimensions are weighted and fused according to preset matching feature weights to obtain a comprehensive similarity score. The matching feature weights represent the importance of each common-origin feature dimension in the comprehensive similarity score, and the sum of the matching feature weights for each dimension is 1. In one implementation, the matching feature weight for the subject and case identifier dimension is 0.4, the matching feature weight for the time dimension is 0.3, and the matching feature weight for the event and address dimension is 0.3. When features are missing, the comprehensive similarity score is calculated after normalizing the matching feature weights within the remaining available dimensions. The comprehensive similarity score can be expressed as: (1) In equation (1), Indicates the overall similarity. Indicates similarity in terms of subject and case identifier dimensions. Indicates similarity over time. Indicates similarity in terms of both items and addresses. , , These represent the matching feature weights corresponding to the subject and case identifier dimension, the time dimension, and the matter and address dimension, respectively. The matching feature weights are configured by experts in advance and determined by calibration in combination with historical annotation samples. .

[0044] When the overall similarity reaches a first preset threshold, structured response index data from different sources are identified as records of the same law enforcement event. The first preset threshold is a similarity threshold used for automatic aggregation and judgment. When the overall similarity reaches the first preset threshold and there is no strong identifier conflict, the corresponding records are automatically grouped into the same law enforcement event group. In one embodiment, the first preset threshold is 0.85. When the overall similarity is lower than the first preset threshold but reaches a second preset threshold, the corresponding record is marked as a record from the same source awaiting review. The second preset threshold is lower than the first preset threshold. The second preset threshold is used to mark candidate aggregation results that require manual review. In one embodiment, the second preset threshold is 0.70. When the overall similarity is greater than or equal to 0.70 and lower than 0.85, the corresponding record is marked as a record from the same source awaiting review.

[0045] If the overall similarity is lower than the second preset threshold, the corresponding record will not be included in the current law enforcement event group and will continue to be matched with other candidate record groups. If the conditions are still not met after the entire process, a separate event group will be created or the case will be transferred to manual review. Each preset threshold can be adjusted based on historical samples.

[0046] In one implementation, the similarity between the complaint work order and the law enforcement business ledger is set to 0.80 for the subject and case identifier dimensions, 0.90 for the time dimension, and 0.95 for the matter and address dimensions. The matching feature weights are set to 0.4, 0.3, and 0.3, respectively. The overall similarity is 0.4×0.80+0.3×0.90+0.3×0.95, which is 0.875. This is greater than the first preset threshold of 0.85, and the two records are grouped into the same law enforcement event when there is no strong identifier conflict. If the review case file and the representative record of the law enforcement event group also meet the automatic grouping conditions, the records from the three sources will enter the subsequent conflict calibration processing.

[0047] In some embodiments, step 03 above includes: Obtain the data source confidence coefficient for each data source; Element fields with inconsistent values ​​from different sources within the same law enforcement event are identified as conflict fields; Based on the preset field type mapping relationship, identify the field type of the conflicting field. The field types include numeric, legal conclusion, and enumeration types, as well as identifier or fact types. Based on the field type of the conflict field, select the corresponding target calibration strategy from the preset calibration strategies; Determine the standard values ​​for the conflict fields based on the target calibration strategy; A standardized evaluation index library is formed based on the consistent values ​​of element fields from various sources and the standard values ​​of conflicting fields.

[0048] Specifically, the confidence coefficient of each data source is obtained. The data source confidence coefficient represents the degree of truthfulness and accuracy of the content recorded by the data source. When different sources within the same law enforcement event have inconsistent values ​​for the same field, the data source confidence coefficient is used to measure the credibility of the values ​​from different sources. The data source confidence coefficient serves as a conflict calibration tool after the completion of same-source aggregation and is distinct from the matching feature weights used in same-source aggregation. The data source confidence coefficient is determined by a combination of expert evaluation and historical error statistics, and can be expressed as: (2) In equation (2), This represents the data source confidence coefficient for the i-th data source. This represents the expert confidence score for the i-th data source, which is obtained by domain experts rating the reliability of the data source. This represents the historical data confidence score of the i-th data source. The historical data confidence score is calculated from the historical error rate of the data source. The historical data confidence score can be obtained by subtracting the error rate from 1. α represents the combination coefficient, which is used to adjust the proportion of expert confidence score and historical data confidence score in the data source confidence coefficient. The data source confidence coefficient is cropped to the range of 0 to 1. In one implementation, the combination coefficient is set to 0.5. Taking administrative law enforcement business ledgers, review or litigation files, and public complaint work orders as examples, the data source confidence coefficients can be configured as 0.95, 0.90, and 0.65, respectively.

[0049] Element fields with inconsistent values ​​from different sources within the same law enforcement event are identified as conflicting fields. Conflict detection is performed only within the same law enforcement event group, comparing the values ​​of fields with the same name from different sources. In one implementation, numeric fields are determined to be conflicting based on the degree of value difference; a conflict is determined when the difference between the maximum and minimum values ​​from different sources is greater than the product of a preset ratio and the average value. This can be expressed as: (3) In equation (3), This indicates the value of a numerical field across records from various sources. `max` represents the maximum value, `min` represents the minimum value, `mean` represents the average value, and `δ` represents a preset percentage; in one implementation, `δ` is 20%. Fields of legal conclusion type, enumeration type, identifier type, or fact type are considered conflicting fields if their values ​​differ between two or more source records. Conflict detection is used to trigger conflict calibration or manual review prompts; it does not directly delete data from any source.

[0050] Based on the pre-defined field type mapping relationship, the field types of conflicting fields are identified. These field types include numerical, legal conclusion, enumeration, and identifier / factual. Field types are pre-configured according to field semantics, data structure, and standard value determination methods. Numerical fields include the number of complaints, processing time, overdue days, and response time; legal conclusion fields include review results and litigation judgment results; enumeration fields include dispute handling status, duplicate complaint indicators, and whether a response was given on time; and identifier / factual fields include case number, enforcement entity name, address, response deadline, and dates of various actions. The field type mapping relationship corresponds to the element fields output in the parsing phase, and conflicting fields select the corresponding standard value determination method according to their field type.

[0051] Based on the field type of the conflicting field, a corresponding target calibration strategy is selected from the preset calibration strategies. The preset calibration strategies include at least a confidence-weighted fusion strategy for numerical fields, a strategy for determining the basis of legally binding legal documents for statutory conclusion fields, a confidence-weighted voting strategy for enumerated fields, and a source priority verification strategy for identifier or factual fields. Each calibration strategy corresponds to one field type. Based on the target calibration strategy, the standard value of the conflicting field is determined. The method for determining the standard value for each field type will be described in subsequent implementations.

[0052] The standardized assessment indicator library is formed by conflict-free element fields and standard values ​​of conflicting fields. Conflict-free element fields retain consistent values ​​after alignment and standardization, and retain source evidence information; conflicting fields are processed according to standard values. When forming the standardized assessment indicator library, the indicator names, data types, statistical definitions, and numerical formats of each field are unified. Fields with identifiable standard values ​​output their standard values, field types, calibration methods, and confidence information. Fields whose standard values ​​cannot be automatically determined are marked as requiring manual review, and the original values, evidence fragments, conflict types, and calibration records from each source are retained. In one implementation, fields marked as requiring manual review are not included in the corresponding field-related deductions and maturity calculations; instead, they are handled by the missing value rule during the assessment phase.

[0053] In some implementations, determining the standard value of the conflict field in step 03 above includes: When the field type of the conflicting field is numeric, the values ​​from each source are weighted and merged according to the confidence coefficient of the corresponding data source to obtain the standard value of the conflicting field. When the field type of the conflict field is statutory conclusion type, the conclusion of the main text of the effective legal document of the case corresponding to the conflict field is determined as the standard value of the conflict field; When the field type of the conflicting field is an enumeration type, the confidence coefficients of the data sources corresponding to the values ​​from the same source are summed, and the source value with the largest summation result is determined as the standard value of the conflicting field. When the field type of a conflicting field is an identifier or a fact, the standard value of the conflicting field is determined based on the preset source priority and the original text content of each source value.

[0054] Specifically, when the conflicting field is numeric, the values ​​from each source are weighted and merged according to the confidence coefficient of the corresponding data source to obtain the standard value of the conflicting field. The merging standard value of a numeric field can be expressed as: (4) In equation (4), This represents the fusion standard value for numeric conflict fields, where n represents the number of sources providing available values, and i represents the source sequence number. This represents the value of the i-th source record. Σ represents the data source confidence coefficient corresponding to the i-th source, and Σ represents summation. The fusion standard value is a weighted sum of the values ​​from each source based on the data source confidence coefficient, and then divided by the sum of the data source confidence coefficients. The fusion result can be rounded down or retained as a decimal according to the field rules, and the original values ​​of each source and conflict alarms are recorded. In one implementation, the number of complaints in the law enforcement business ledger, review or litigation case files, and public complaint work orders are 10, 13, and 7, respectively, corresponding to data source confidence coefficients of 0.95, 0.90, and 0.65, respectively. The fusion standard value is (0.95×10+0.90×13+0.65×7)÷(0.95+0.90+0.65), which is approximately 10.30. It is rounded down to 10 according to the field's integer rule, and conflict records are retained.

[0055] When the conflict field is of the statutory conclusion type, the conclusion in the main text of the effective legal document corresponding to the case is determined as the standard value of the conflict field. Statutory conclusion types are based on the conclusion stated in the main text of the effective administrative reconsideration decision or effective judgment, and are not subject to numerical weighting. When different effective legal documents still conflict on the same matter, verification is performed according to the legal document's level of jurisdiction, the document's effective date, and the correspondence between the document and the case; those that cannot be determined are marked as awaiting manual review. In one implementation, a public complaint work order records that the public believes the original decision should be revoked, the law enforcement business log records that the original decision is upheld, and the main text of the effective administrative reconsideration decision records that the original decision is revoked. For statutory conclusion types of conflict fields, the main text of the effective decision is used as the standard, and the reconsideration result is standardized as revocation. Records from other sources are retained as conflict evidence.

[0056] When the conflict field is of enumeration type, the confidence coefficients of the data sources corresponding to the same source values ​​are summed, and the source value with the largest sum is determined as the standard value of the conflict field. Since the set of legal values ​​for an enumeration field is finite, and the values ​​in each source record belong to a preset enumeration value, the confidence coefficients of the data sources corresponding to the same enumeration value are added together. Each candidate enumeration value is compared according to the sum of its confidence coefficients, and the source value with the largest sum of confidence coefficients is determined as the standard value. When the sums of confidence coefficients corresponding to different candidate enumeration values ​​are close, i.e., the difference between the confidence coefficients of the highest candidate and the second highest candidate is less than a preset difference, it is marked as pending manual review, and evidence from both parties is retained. In one implementation, when the dispute resolution status is recorded as "completed" in one source record and "unresolved" in another, the standard value is determined by confidence-weighted voting, and when the confidence levels of the two are close, it is marked as pending manual review.

[0057] The standard value for conflict-type identifier or factual fields is determined based on a preset source priority and the original text content of each source's value. Source priority indicates the credible order of different sources in the identifier or factual field. Each source's value undergoes a consistency check with the original text content, and the standard value is determined based on the check result of the source priority and the original text; an arithmetic average is not used. In one implementation, the source priority for identifier or factual fields decreases sequentially from law enforcement business ledger number, document record, to work order entry. When the original text content of a source value is inconsistent with the field's semantics, or when a unique standard value cannot be determined, multi-source evidence is retained and marked for manual review.

[0058] In some implementations, step 04 above includes: Based on the standardized evaluation indicator library, the indicator values ​​for law enforcement entities on each evaluation indicator are determined. The evaluation indicators include response timeliness, dispute resolution, handling of reconsideration, and public satisfaction. Based on the pre-set specific deduction rules and indicator values, determine the deduction values ​​for the evaluation indicators; The actual evaluation value of the evaluation indicator is determined based on the benchmark score and deduction score of the evaluation indicator. Based on the analytic hierarchy process, the combined weights of each evaluation indicator are determined according to their importance. The maturity level of each evaluation indicator is determined based on the position of its actual measured value in the historical data distribution. According to the preset grading rules, the maturity level of each evaluation indicator is converted into a grade score. The evaluation score is obtained by weighting and combining the grade scores of each evaluation indicator with their corresponding combined weights.

[0059] Specifically, based on a standardized evaluation indicator library, the indicator values ​​for each evaluation indicator are determined for the law enforcement entity. These indicators include response timeliness, dispute resolution, administrative review handling, and public satisfaction. The evaluation indicators are set around the scenario of post-event response to administrative law enforcement. Response timeliness measures the timeliness of the law enforcement entity's response to the public and the handling of matters; dispute resolution measures the resolution of complaints and disputes; administrative review handling measures the post-event correction of errors such as revocation or modification of administrative review decisions and losses in administrative litigation; and public satisfaction measures the public's satisfaction with the response process.

[0060] In one implementation, the evaluation indicators are set under four primary dimensions: fairness, effectiveness, standardization, and scientific rigor. Response timeliness falls under the effectiveness dimension, dispute resolution under the fairness dimension, administrative review handling under the standardization dimension, and public satisfaction under the scientific rigor dimension. These primary dimensions are used for indicator system classification and result display. The indicator values ​​for administrative review handling are determined by considering both the revocation or modification of administrative review and the loss in administrative litigation.

[0061] Based on the pre-set specific deduction rules and indicator values, the deduction values ​​for the evaluation indicators are determined. The specific deduction rules are pre-set for post-event response scenarios and may include: a late response rule (2 points deducted for each day overdue, with an additional 1 point deducted for each additional day overdue); a repeated complaint rule (5 points deducted when the public submits more than 3 complaints about the same matter); a reconsideration or modification rule (10 points deducted when the reconsideration agency revokes or modifies the original enforcement decision); a litigation loss rule (15 points deducted when the court rules against the administrative enforcement agency); and a failure to provide timely feedback on rectification (3 points deducted when feedback is not submitted on time after receiving a rectification notice). The deduction values ​​for each evaluation indicator are determined based on whether the indicator values ​​in the standardized evaluation indicator library match the specific deduction rules.

[0062] The correspondence between deduction items and evaluation indicators is as follows: overdue response corresponds to response timeliness, repeated complaints correspond to dispute resolution, revocation or modification of reconsideration corresponds to reconsideration handling, losing a lawsuit corresponds to reconsideration handling, and failure to provide timely feedback on rectification corresponds to public satisfaction.

[0063] The actual evaluation value of each evaluation indicator is determined based on its baseline score and deduction value. A baseline score is set for each indicator; the actual evaluation value is the baseline score minus the deduction value, deducting until the baseline score is reached. In one implementation, the baseline score for each evaluation indicator is 100 points. The higher the actual evaluation value of an indicator, the better the post-event response performance. The actual evaluation value of the evaluation indicators is a direct input for subsequent maturity assessment and overall score calculation.

[0064] Based on the Analytic Hierarchy Process (AHP), the combined weights of each evaluation indicator are determined according to their importance. The AHP constructs a judgment matrix by having experts compare the importance of each indicator at the same level pairwise, calculating the weights of each level's indicators and combining them layer by layer. When constructing the judgment matrix, domain experts are invited to score the importance of each indicator at the same level pairwise using a 1-9 scale. The elements in the judgment matrix represent the multiple of importance of an indicator in one row relative to an indicator in a column. The geometric mean is calculated for each row of the judgment matrix; the geometric mean of the evaluation indicators in the i-th row can be expressed as: (5) In equation (5), Let represent the geometric mean of the i-th row of the judgment matrix, n represent the total number of evaluation indicators at the same level, and j represent the column number. Let represent the element in the i-th row and j-th column of the judgment matrix, i.e., the score given by experts for the importance of the i-th indicator relative to the j-th indicator. ∏ denotes multiplication, and the superscript 1 / n indicates the nth power. The single-layer weight of the i-th evaluation indicator can be expressed as: (6) In equation (6), This represents the single-layer weight of the i-th evaluation index. Let represent the geometric mean of the k-th row of the judgment matrix, and Σ represent the sum of the geometric means over all n rows. The sum of the weights in a single layer is 1. and the combined weights used in subsequent comprehensive evaluations, along with the confidence coefficients of the aforementioned data sources. The weight β of the same-origin set matching feature belongs to different calculation stages, and their meanings should not be mixed.

[0065] The judgment matrix needs to undergo consistency verification. The consistency metric can be expressed as: (7) In equation (7), CI represents the consistency index. represents the largest eigenvalue of the judgment matrix, and n represents the order of the judgment matrix. The consistency ratio is the ratio of the consistency index to the average random consistency index of the same order. When the consistency ratio is less than a preset consistency threshold, the judgment matrix is ​​considered to have passed the consistency check. The average random consistency index of the same order is obtained by looking up a table. In one implementation, the consistency threshold is 0.1. When the judgment matrix passes the consistency check, the calculated single-level weights have reference value. When there are multiple levels of evaluation indicators, the weights of each level are multiplied layer by layer to obtain the final combined weights of the evaluation indicators. In one implementation, the combined weights of the four evaluation indicators—response timeliness, dispute resolution, reconsideration handling, and public satisfaction—are 0.25, 0.20, 0.30, and 0.25, respectively, and the sum of the combined weights is 1.

[0066] The maturity level of each evaluation indicator is determined based on its position in the historical data distribution. The historical data distribution is obtained by statistically analyzing the actual evaluation values ​​of each indicator within a historical period. The higher the position of the actual evaluation value of an evaluation indicator in the historical data distribution, the higher its corresponding maturity level. The process of classifying maturity levels will be explained in subsequent implementation methods. According to a preset level scoring rule, the maturity level of each evaluation indicator is converted into a level score. The rules for determining the level score will be explained in subsequent implementation methods.

[0067] The scores of each evaluation indicator are weighted and combined with their corresponding combined weights to obtain the comprehensive evaluation score, which can be expressed as: (8) In equation (8), This represents the overall evaluation score, where n represents the total number of evaluation indicators, and i represents the indicator number. Let represent the combined weight of the i-th evaluation indicator. The combined weights have been normalized and the sum of the weights is 1. This represents the grade score of the i-th evaluation indicator. The comprehensive evaluation score is obtained by weighted summation of the grade scores of each evaluation indicator according to their combined weights. The actual evaluation value of the evaluation indicator is first formed by specific deductions and then converted by maturity level. The deduction results directly affect the comprehensive evaluation score and are not deducted again in the weighted formula. In one implementation, the range of the comprehensive evaluation score is: A score of 1 to 1, with a comprehensive evaluation score greater than 0, indicates that the post-event response effectiveness is basically up to standard. For ease of presentation, the comprehensive evaluation score can be linearly mapped to a display score, which can be expressed as: (9) In equation (9), This indicates that the score is being displayed. This indicates the overall evaluation score, with the displayed score ranging from 0 to 100.

[0068] In one implementation, the combined weights of the four evaluation indicators for a certain law enforcement department are as follows: response timeliness 0.25, dispute resolution 0.20, reconsideration handling 0.30, and public satisfaction 0.25. The maturity levels of each evaluation indicator are: excellent response timeliness, excellent dispute resolution, poor reconsideration handling, and average public satisfaction, with scores of 1, 1, and 1 respectively. 1. 0, the overall evaluation score is 0.25×1 + 0.20×1 + 0.30×( 1) +0.25×0, the calculation result is 0.15, and the displayed score is 50 + 50×0.15 = 57.5. The maturity level corresponding to the reconsideration is poor, reflecting that the reconsideration is a weakness, which is used to determine the rectification targets in the future.

[0069] In some implementations, determining the maturity level of each evaluation indicator in step 04 above includes: Obtain the actual evaluation values ​​of each evaluation indicator within the historical time period; The distribution of historical data is determined based on the actual evaluation values ​​within the historical period. Based on the preset grade ratio, the historical data distribution is divided into three intervals: the first interval, the second interval, and the third interval. The actual evaluation value in the first interval is higher than the actual evaluation value in the second interval, and the actual evaluation value in the second interval is higher than the actual evaluation value in the third interval. When the actual measured value of the evaluation indicator falls into the first interval, the maturity level of the evaluation indicator is determined as the first maturity level. When the actual measured value of the evaluation indicator falls into the second interval, the maturity level of the evaluation indicator is determined as the second maturity level. When the actual measured value of the evaluation indicator falls into the third interval, the maturity level of the evaluation indicator is determined as the third maturity level. The first maturity level is higher than the second maturity level, and the second maturity level is higher than the third maturity level.

[0070] Specifically, the actual evaluation values ​​of each evaluation indicator within a historical period are obtained. The historical period can be a fixed timeframe used to trace back from the initial assessment. The actual evaluation values ​​are derived from the actual evaluation values ​​of the evaluation indicators calculated at each evaluation point within the historical period, and are collected separately for each evaluation indicator. In one implementation, the historical period is the past three years, and the actual evaluation values ​​of the administrative law enforcement post-event response indicators from the past three years are retrieved as the historical sample.

[0071] Based on the actual evaluation values ​​within historical periods, the historical data distribution is determined. Statistical analysis is performed on the historical actual evaluation values ​​of the same evaluation indicator to obtain the historical data distribution. The historical data distribution reflects the distribution of the corresponding evaluation indicator's actual evaluation value among law enforcement entities and is used to determine the relative position of the current actual evaluation value.

[0072] Based on preset grading percentages, historical data distribution is divided into three intervals: a first interval, a second interval, and a third interval. The actual evaluation values ​​in the first interval are higher than those in the second interval, and vice versa. The grading percentage represents the proportion of each interval in the historical sample. The historical sample is divided into three intervals according to the actual evaluation values ​​from highest to lowest: the first interval has the highest actual evaluation value, and the third interval has the lowest. In one implementation, the grading percentages are set at 30% for the first interval, 40% for the second interval, and 30% for the third interval. That is, the first interval represents the top 30% of the historical sample in terms of actual evaluation values, the second interval represents the middle 40%, and the third interval represents the bottom 30%. The actual evaluation values ​​of each evaluation indicator are divided into intervals according to a positive benefit principle where higher values ​​indicate better performance.

[0073] When the actual measured value of an evaluation indicator falls into the first interval, its maturity level is determined as the first maturity level; when it falls into the second interval, it is determined as the second maturity level; and when it falls into the third interval, it is determined as the third maturity level. The first maturity level is higher than the second maturity level, and the second maturity level is higher than the third maturity level. The maturity level reflects the relative position of the actual measured value of the evaluation indicator in the historical data distribution.

[0074] In one implementation, the rating rules are as follows: the rating score for the first maturity level is 1, the rating score for the second maturity level is 0, and the rating score for the third maturity level is... 1. The grade score is used to weight and merge with the combined weights to obtain the comprehensive evaluation score. The maturity level is determined based on the relative position of the actual evaluation value in the historical data distribution. Even if the law enforcement entity has no violation points, the actual evaluation value may still fall into the third interval because the actual evaluation values ​​of other law enforcement entities in the same industry are higher.

[0075] To avoid excessive pressure on law enforcement entities without violations due to relative ranking, a protective rule is set: when a law enforcement entity has 0 points for each specific deduction corresponding to each evaluation indicator, the maturity level of the law enforcement entity shall not be lower than the second maturity level. Relative ranking is used to further distinguish the performance of law enforcement entities with violations.

[0076] In some embodiments, the above method further includes: Based on the assessment scores, law enforcement entities whose assessment scores are below the third preset threshold are identified as weak law enforcement entities; The evaluation indicators corresponding to the third maturity level of weak law enforcement entities are identified as the weakest link indicators. Based on the indicators of shortcomings and the weak enforcement entities, generate rectification and supervision work orders; The corresponding shortcomings, rectification deadlines, and responsible persons for rectification should be written into the rectification supervision work order; Based on the management authority of the weak enforcement entities, the rectification and supervision work orders will be pushed to the corresponding enforcement management terminals; Obtain the rectification results of the rectification supervision work order; The rectification results are reviewed, and the confirmed rectification cases are fed back to update the response element rule set and target extraction model. Based on the rectification results, weak law enforcement entities and shortcomings were re-identified.

[0077] Specifically, based on the assessment scores, law enforcement entities whose assessment scores are below a third preset threshold are identified as weak law enforcement entities. The third preset threshold is the assessment score threshold used to determine weak law enforcement entities; when an enforcement entity's overall assessment score is below the third preset threshold, it is identified as a weak law enforcement entity. In one implementation, when the overall assessment score is represented by a displayed score, the displayed score corresponding to the third preset threshold is 40, meaning that an overall assessment score below... Enforcement entities with a score of 0.2 are identified as weak enforcement entities. The indicator-level maturity level is used to identify shortcomings in indicators; it belongs to a different category from the weakness assessment based on the comprehensive evaluation score and should not be used interchangeably.

[0078] Evaluation indicators corresponding to the third maturity level of weak law enforcement entities are identified as bottleneck indicators. The actual measured values ​​of the third maturity level evaluation indicators are located at the lower end of the historical data distribution, reflecting insufficient ex-post response performance for the corresponding evaluation indicators. In one implementation, when the maturity level of the reconsideration handling is the third maturity level, the reconsideration handling is identified as a bottleneck indicator, corresponding to response issues such as reconsideration revocation or modification, and litigation losses.

[0079] Based on the identified weaknesses in key performance indicators (KPIs) and weak enforcement entities, rectification and supervision work orders are generated. These work orders are used to urge weak enforcement entities to rectify the issues corresponding to the KPIs. The work orders should include the specific KPIs, rectification deadlines, and responsible parties. The KPI description should outline the specific problems corresponding to the KPIs, the rectification deadline should specify the timeframe for completion, and the responsible party should clearly identify the entity bearing the responsibility for rectification. Rectification and supervision work orders can also include rectification judgment criteria for reviewing and assessing the rectification results.

[0080] Based on the management authority of the weak enforcement entities, rectification supervision work orders are pushed to the corresponding enforcement management terminals. Rectification supervision work orders are distributed according to the management authority of the enforcement entities, and pushed to the enforcement management terminals corresponding to the weak enforcement entities to avoid cross-departmental mis-distribution of work orders. The process of pushing and circulating rectification supervision work orders is recorded in a ledger, which records the work order issuance time, rectification feedback time, and review and acceptance results, ensuring that the operation process is traceable and auditable.

[0081] Obtain the rectification results from the rectification supervision work orders. Weak enforcement entities complete rectification within the specified timeframe and return the rectification results, recording the handling of shortcomings. Review the rectification results, and feed back the confirmed rectification cases to update the response element rule set and target extraction model. Rectification cases confirmed to meet the judgment criteria are fed back as high-quality samples to expand the field configuration, synonym normalization mapping, and value constraints of the response element rule set, and to iteratively update the target extraction model. High-confidence corrected samples generated from the target inference model's error correction and verification can also be fed back to update the target extraction model. After rectification, re-evaluate, and re-select weak enforcement entities and shortcomings based on the updated evaluation results, forming a continuous closed loop of identification, supervision, rectification, review, and re-identification.

[0082] This application also provides an evaluation system for the effectiveness of post-event response to administrative law enforcement. The system includes a processor and a memory. The memory stores a computer program. When the processor executes the computer program, it implements the steps of the above-mentioned evaluation method for the effectiveness of post-event response to administrative law enforcement.

[0083] Please see Figure 2 As shown, Figure 2This is a schematic diagram of the structure of an evaluation system for the effectiveness of post-event response to administrative law enforcement provided in an embodiment of the present invention. In one embodiment, the evaluation system for the effectiveness of post-event response to administrative law enforcement includes a multi-source relief document intelligent extraction module, a multi-source feedback data confidence fusion and typified conflict calibration module, an administrative law enforcement post-event response effectiveness evaluation engine, and an effectiveness analysis closed-loop supervision output module, which are connected in sequence. The output of the former module serves as the input of the latter module. The rectification feedback output by the effectiveness analysis closed-loop supervision output module is fed back to the multi-source relief document intelligent extraction module to update the response element rule set and target extraction model.

[0084] The multi-source relief document intelligent extraction module receives administrative law enforcement post-event response documents from multiple data sources. Under the constraints of the response element rule set, the target extraction model extracts candidate element fields in batches, and after value constraint verification, synonym normalization, and error correction verification by the target inference model, it outputs structured response indicator data for each case. The multi-source feedback data confidence fusion and typified conflict calibration module receives structured response indicator data for each case, configures the data source confidence coefficients of each data source, performs homogeneous case aggregation, performs conflict calibration on conflicting fields according to field type, and outputs a standardized evaluation indicator library.

[0085] The special evaluation engine for the effectiveness of post-event response to administrative law enforcement is used to receive a standardized evaluation indicator library, determine the indicator values, deduction values, actual evaluation values, combined weights and maturity levels of the evaluation indicators, and output evaluation scores and shortcomings information.

[0086] The performance analysis closed-loop supervision output module is used to receive evaluation scores and shortcomings information, identify weak law enforcement entities and shortcomings indicators, generate rectification supervision work orders and push them to the corresponding law enforcement management terminals, track rectification results and provide feedback on rectification.

[0087] The modules exchange data through a standardized data structure, and the evaluation data and rectification feedback form a closed loop.

[0088] In one implementation, when the multi-source relief document intelligent extraction module performs parsing, the target extraction model is a contrastive learning model (Lo-Rank Adaptation, LoRA) fine-tuned using a low-rank adaptation method. The low-rank adaptation method trains a small number of adaptation parameters while keeping the main parameters of the pre-trained model unchanged, while the contrastive learning method enables the model to learn to distinguish the representations of different feature fields.

[0089] The target inference model is a large-scale, powerful inference model deployed on the server. During error correction and verification, the target inference model uses the original content of the standard text as evidence for comparison and review. The server-side deployment ensures that the processing of the target inference model does not occupy the front-end real-time processing link. The combined weights of the evaluation indicators are determined using the Analytic Hierarchy Process (AHP). AHP involves domain experts comparing the importance of indicators pairwise to construct a judgment matrix and calculate the weights.

[0090] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side method for evaluating the effectiveness of post-event responses in administrative law enforcement.

[0091] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a method for evaluating the effectiveness of post-event responses in administrative law enforcement.

[0092] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0093] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for evaluating the effectiveness of post-event response in administrative law enforcement, characterized in that, Includes the following steps: Obtain relevant administrative enforcement response documents from multiple data sources; The relevant documents for post-enforcement responses to administrative law enforcement are subjected to standardized preprocessing to obtain standard texts; Under the constraints of a preset set of response element rules, the standard text is input into the target extraction model to obtain candidate element fields; According to the response element rule set, the candidate element fields are subjected to value constraint verification and synonym normalization to obtain pre-verified element fields; The standard text and the pre-validation element field are input into the target inference model. Based on the original content of the standard text, error correction and validation are performed on the pre-validation element field to obtain the corrected element field. Based on the modified element fields, generate structured response indicator data for each case; The structured response indicator data of each case are grouped together to identify records that point to the same law enforcement event; For fields that conflict within the same law enforcement event after aggregation, the corresponding calibration strategy is selected according to the preset field type to determine the standard value, so as to form a standardized evaluation index library; Based on the standardized evaluation index library, the effectiveness of the law enforcement entity's post-event response is quantitatively evaluated to obtain an evaluation score, which is used to determine the corresponding law enforcement entity.

2. The method according to claim 1, characterized in that, The step of inputting the standard text and the pre-validation element field into the target inference model, and performing error correction validation on the pre-validation element field based on the original content of the standard text to obtain the corrected element field includes: The target inference model compares each of the pre-verified element fields with the original text content in the standard text; When the pre-validation element field is inconsistent with the corresponding original text content, the pre-validation element field is corrected according to the original text content. When the pre-validation element field lacks corresponding original text content in the standard text, the pre-validation element field is deleted. The retained pre-validated feature fields and the corrected pre-validated feature fields are combined to obtain the corrected feature fields.

3. The method according to claim 1, characterized in that, The process of grouping structured response indicator data from various cases into records pointing to the same law enforcement event includes: Extract the common source aggregation features of the structured response indicator data of each case. The common source aggregation features include case number, law enforcement entity, event time, and matters requested. When the case numbers of structured response indicator data from different sources are consistent, the structured response indicator data from different sources are identified as records of the same law enforcement event; When structured response indicator data from different sources lack consistent case numbers, calculate the similarity of structured response indicator data from different sources across each common-source aggregation feature dimension; The similarity scores across the various origin-based feature dimensions are weighted and fused according to preset matching feature weights to obtain a comprehensive similarity score. When the overall similarity reaches a first preset threshold, structured response index data from different sources are identified as records of the same law enforcement event; When the overall similarity is lower than the first preset threshold and reaches the second preset threshold, the corresponding structured response index data is marked as a record to be reviewed from the same source.

4. The method according to claim 1, characterized in that, For fields that conflict within the same law enforcement event after aggregation, a standard value is determined by selecting the corresponding calibration strategy based on the preset field type to form a standardized evaluation index library, including: Obtain the data source confidence coefficient for each data source; Element fields with inconsistent values ​​from different sources within the same law enforcement event are identified as conflict fields; Based on a preset field type mapping relationship, the field type of the conflicting field is identified, including numeric, legal conclusion, enumeration, and identifier or fact types. Based on the field type of the conflict field, select the corresponding target calibration strategy from the preset calibration strategies; Based on the target calibration strategy, determine the standard value of the conflict field; The standardized evaluation index library is formed based on the values ​​of the element fields that have consistent values ​​from various sources and the standard values ​​of the conflicting fields.

5. The method according to claim 4, characterized in that, Determining the standard value of the conflict field according to the target calibration strategy includes: When the field type of the conflicting field is numeric, the values ​​from each source are weighted and merged according to the confidence coefficient of the data source corresponding to each source to obtain the standard value of the conflicting field; When the field type of the conflict field is a statutory conclusion type, the main text conclusion of the effective legal document of the case corresponding to the conflict field is determined as the standard value of the conflict field; When the field type of the conflict field is an enumeration type, the confidence coefficients of the data sources corresponding to the same source values ​​are summed, and the source value with the largest summation result is determined as the standard value of the conflict field. When the field type of the conflicting field is an identifier or a fact, the standard value of the conflicting field is determined according to the preset source priority and the original text content of each source value.

6. The method according to claim 1, characterized in that, The standardized evaluation index library is used to quantitatively evaluate the effectiveness of law enforcement entities' post-event response, resulting in an evaluation score, including: Based on the standardized evaluation index library, the index values ​​of law enforcement entities on each evaluation index are determined. The evaluation indexes include response timeliness, dispute resolution, reconsideration handling, and public satisfaction. Based on the preset specific deduction rules and the values ​​of the indicators, the deduction values ​​of the evaluation indicators are determined; The actual evaluation value of the evaluation indicator is determined based on the benchmark score and the deduction value of the evaluation indicator. Based on the analytic hierarchy process, the combined weights of each evaluation indicator are determined according to their importance. The maturity level of each evaluation indicator is determined based on the position of its actual measured value in the historical data distribution. According to the preset grading rules, the maturity level of each evaluation indicator is converted into a grade score. The evaluation score is obtained by weighting and integrating the grade scores of each evaluation indicator with their corresponding combined weights.

7. The method according to claim 6, characterized in that, The process of determining the maturity level of each evaluation indicator based on its actual measurement value within the historical data distribution includes: Obtain the actual evaluation values ​​of each evaluation indicator within the historical time period; The distribution of the historical data is determined based on the actual evaluation values ​​within the historical period. According to the preset grade ratio, the historical data distribution is divided into a first interval, a second interval, and a third interval. The actual evaluation value in the first interval is higher than the actual evaluation value in the second interval, and the actual evaluation value in the second interval is higher than the actual evaluation value in the third interval. When the actual measured value of the evaluation indicator falls into the first interval, the maturity level of the evaluation indicator is determined as the first maturity level. When the actual measured value of the evaluation indicator falls into the second interval, the maturity level of the evaluation indicator is determined as the second maturity level. When the actual measured value of the evaluation indicator falls into the third interval, the maturity level of the evaluation indicator is determined as the third maturity level. The first maturity level is higher than the second maturity level, and the second maturity level is higher than the third maturity level.

8. The method according to claim 7, characterized in that, The method further includes: Based on the assessment scores, law enforcement entities whose assessment scores are below the third preset threshold are identified as weak law enforcement entities; The evaluation index corresponding to the third maturity level of the weak law enforcement entity is identified as the shortcoming index; Based on the aforementioned shortcomings and the weak enforcement entities, a rectification and supervision work order is generated. The corresponding shortcomings, rectification deadlines, and responsible persons for rectification shall be written into the rectification supervision work order; Based on the management authority of the weak enforcement entity, the rectification and supervision work order will be pushed to the corresponding enforcement management terminal; Obtain the rectification results of the rectification supervision work order; The rectification results are reviewed, and the confirmed rectification cases are fed back to update the response element rule set and the target extraction model. Based on the rectification results, weak law enforcement entities and shortcomings were re-identified.

9. An evaluation system for the effectiveness of post-event response to administrative law enforcement, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method for evaluating the effectiveness of post-event response to administrative law enforcement as described in any one of claims 1 to 8.

Images