Multi-agent collaborative evaluation system, method, device and storage medium

Abstract

This application proposes a multi-agent collaborative evaluation system, method, device, and storage medium, relating to the field of artificial intelligence technology. The system includes: a visual agent, a text agent, and a conditional review agent. The visual agent performs multimodal analysis on input image data, identifies various types of elements in the image data, extracts key information from the image data based on element recognition information, and outputs a structured document based on the key information. The text agent performs semantic understanding and information extraction on input text data, and outputs a standardized document based on the extracted information. The conditional review agent receives the structured document and / or the standardized document, performs multi-dimensional evaluation based on the received document and the input request, and generates an evaluation report corresponding to the input request. This multi-agent collaborative evaluation system can effectively improve document review efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a multi-agent collaborative evaluation system, method, device and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI agents are being used more and more widely in complex decision-making scenarios.

[0003] Currently, in various application scenarios such as human resource management, it is necessary to review and evaluate qualification certification materials. However, manual review is often inefficient and prone to omissions. Therefore, reviewers often use document recognition systems to conduct preliminary analysis and screening of qualification certification materials.

[0004] Traditional document recognition systems rely on a single dimension for credibility assessment in their decision-making process. They often rely solely on screening based on a single indicator and lack a deep understanding of the content of qualification and certification materials and the ability to make conditional judgments. As a result, the review and evaluation tasks still need to be completed manually, which limits the improvement of review efficiency. Summary of the Invention

[0005] In view of this, this application proposes a multi-agent cooperative evaluation system, method, apparatus and storage medium.

[0006] In a first aspect, this application provides a multi-agent collaborative evaluation system, including: a visual agent, a text agent, and a condition review agent; The visual agent is used to perform multimodal analysis on the input image data, identify various types of elements in the image data, extract key information from the image data based on the element identification information, and output a structured document based on the key information. The text agent is used to perform semantic understanding and information extraction on the input text data, and output standardized documents based on the extracted information; The condition review agent is used to receive the structured document and / or the standardized document, and perform multi-dimensional evaluation based on the received document and input request to generate an evaluation report corresponding to the input request.

[0007] In one embodiment, the visual agent is further configured to: when handwritten text is recognized in the image data, perform contrast enhancement processing on the handwritten text, determine the stroke connection based on the distance between stroke points and the stroke width, and split the connected characters.

[0008] In one embodiment, the visual agent is further configured to: perform color feature recognition in the image data to determine candidate areas for seals, perform shape matching on the candidate areas for seals, and determine whether a seal exists in each candidate area for seals based on the matching results.

[0009] In one embodiment, the visual agent is further configured to: identify the image data based on a preset blurry image recognition algorithm, determine whether there is a blurry image in the image data and the degree of blurriness, and when there is a blurry image, adjust the core parameters of the sharpening filter algorithm based on the degree of blurriness, and use the sharpening filter algorithm to process the blurry image.

[0010] In one embodiment, the text agent is further configured to: determine the information density of each paragraph based on the proportion of key information in the paragraph; determine the information gradient of the current detected paragraph based on the information density of the current detected paragraph and the information density of adjacent paragraphs; identify the current detected paragraph as an information surge point if the information gradient of the current paragraph is greater than a preset threshold; optimize paragraphs not identified as information surge points based on preset optimization rules, wherein the optimization process includes at least one of deletion processing and paragraph merging processing; vectorize the sentences in each paragraph; calculate the vector distance of each sentence vector using cosine similarity; identify similar sentences based on the vector distance of each sentence vector and a similarity threshold; and retain sentences with higher information density among similar sentences.

[0011] In one embodiment, the condition review agent is further configured to, upon receiving the structured document and the standardized document, and detecting a conflict in the key type information of the structured document and the standardized document, generate viewpoint adversarial samples and text adversarial samples for the visual agent and the text agent respectively, using gradient perturbation and semantic preservation constraints, with the conflicting field as the core; send the viewpoint adversarial samples back to the viewpoint agent and the text adversarial samples back to the text agent; determine the robustness scores of the viewpoint agent and the text agent based on the output drift of the viewpoint agent and the text agent; determine the confidence levels of the viewpoint agent and the text agent based on the entropy of the output distribution of each agent; use the robustness scores and confidence levels of the viewpoint agent and the text agent as observations; use a Bayesian algorithm to determine the weight coefficients of the robustness scores and confidence levels; and, based on the weight coefficients of the robustness scores and confidence levels, and the robustness scores and confidence levels of the viewpoint agent and the text agent, determine one of the output fields of the viewpoint agent and the text agent as the target field of the conflicting field.

[0012] In one embodiment, the condition review agent is used to perform semantic recognition on the input request, determine the first condition information contained in the input request, and obtain the second condition information corresponding to the condition keywords based on the condition keywords of the input request; perform condition review evaluation on the received document based on the first condition information and the second condition information, and generate an evaluation report corresponding to the input request based on the condition review results.

[0013] Secondly, this application also provides a multi-agent cooperative evaluation method, which is performed based on the multi-agent cooperative evaluation system as described in the first aspect; the method includes: Obtain input data and determine the type of the input data; When the input data includes image data, a visual intelligent agent performs multimodal analysis on the input image data, identifies various types of elements in the image data, extracts key information from the image data based on the element identification information, and outputs a structured document based on the key information. When the input data includes text data, a text intelligence agent is used to perform semantic understanding and information extraction on the input text data, and the extracted information is used to input standardized documents. The condition review agent receives the structured document and / or the standardized document, performs a multi-dimensional evaluation based on the received document and the input request, and generates an evaluation report corresponding to the input request.

[0014] Thirdly, this application also provides an electronic device, including a processor and a memory; the memory has a computer program stored thereon, wherein the computer program, when executed by the processor, implements the multi-agent cooperative evaluation method as described in the first aspect.

[0015] Fourthly, this application also provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multi-agent cooperative evaluation method as described in the first aspect.

[0016] The multi-agent cooperative evaluation system proposed in this application has the following advantages over related technologies: 1. The multi-agent collaborative evaluation system of this application uses a visual agent to perform multimodal analysis on the input image data, identify various types of elements in the image data, and extract key information from the image data based on the element recognition information, outputting a structured document based on the key information; and uses a text agent to perform semantic understanding and information extraction on the input text data, outputting a standardized document based on the extracted information. Thus, the visual agent performs focused analysis on the image data, and the text agent performs focused analysis on the text data, achieving effective analysis of specific types of materials through specialized agents, with high accuracy and efficiency. Finally, the output results of the visual agent and the text agent are transmitted to the conditional review agent, which performs multi-dimensional evaluation based on the received document and input request, generating an evaluation report corresponding to the input request. This multi-agent collaborative evaluation system can effectively process input data and perform multi-dimensional comprehensive evaluation, obtaining the final evaluation result based on the input request, achieving in-depth understanding, conditional judgment, and final evaluation of the content of qualification certification materials, effectively improving review efficiency.

[0017] 2. The text agent can determine the information density of each paragraph based on the proportion of key information. It determines the information gradient of the currently detected paragraph based on its information density and that of adjacent paragraphs. If the information gradient of the current paragraph exceeds a preset threshold, it is identified as an information surge point. Based on preset optimization rules, paragraphs not identified as information surge points are optimized to determine key information and refine and integrate the text. Building on this, the text agent can further focus on redundancy at the sentence level. First, it uses sentence vectorization technology to convert each sentence into a computer-recognizable multi-dimensional numerical vector. Then, it calculates the vector distance between any two sentence vectors using a cosine similarity algorithm and compares the result with a similarity threshold. If the similarity between two sentences exceeds the threshold, they are considered similar sentences. The text agent then compares the information density of similar sentences, retaining the sentence with higher density, thus completely eliminating redundancy at the sentence level. Through the above processing, the conciseness and information acquisition efficiency of the text can be greatly improved while ensuring that the core information (keywords, numbers, terms) is not lost. This avoids the core content being buried and solves the problems of fragmentation of non-critical paragraphs and repetition of sentences, ultimately outputting high-quality text with logical coherence and highlighting key points. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure of a multi-agent collaborative evaluation system in one embodiment of this application; Figure 2 This is a schematic diagram of the interactive interface of a multi-agent collaborative evaluation system in one embodiment of this application; Figure 3 This is a flowchart illustrating a multi-agent collaborative evaluation method in one embodiment of this application.

[0020] Explanation of reference numerals in the attached figures: 10-Multi-agent collaborative evaluation system, 11-Visual agent, 12-Text agent, 13-Condition review agent. Detailed Implementation

[0021] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0022] In some embodiments, such as Figure 1 As shown, this application provides a multi-agent collaborative evaluation system 10, which includes a visual agent 11, a text agent 12, and a condition review agent 13.

[0023] The visual agent 11 is used to perform multimodal analysis on the input image data, identify various types of elements in the image data, extract key information from the image data based on the element identification information, and output a structured document based on the key information.

[0024] The visual agent 11 can perform comprehensive and in-depth analysis of image data using multimodal technologies such as image recognition, semantic understanding, and feature extraction. The visual agent 11 can accurately identify various types of elements contained in the image data, including text, tables, graphics, and symbols. Subsequently, based on the identified element information and combined with the system's preset evaluation goals or application requirements, the visual agent 11 filters and extracts key information that has practical value for subsequent evaluation, such as related information in text and key data nodes in charts. Finally, the visual agent 11 organizes this extracted key information according to a standardized format, outputting a structured document containing fixed fields such as an element type list and key information descriptions. This structured document ensures both the clear presentation and efficient storage of key information and provides standardized data support for the subsequent condition review agent 13 to conduct relevant condition reviews, thereby ensuring the smooth operation of the entire multi-agent collaborative evaluation system 10.

[0025] The text agent 12 is used to perform semantic understanding and information extraction on the input text data, and output standardized documents based on the extracted information.

[0026] In the application, the visual agent 11 can identify the format of the input data and then transmit the text data to the text agent 12.

[0027] Text Agent 12, when given input text materials (such as application materials and resumes), utilizes Natural Language Processing (NLP) technology, deep learning models, and domain-specific corpora to conduct multi-dimensional semantic understanding. This enables the identification of basic linguistic elements (such as word meanings and grammatical structures) and in-depth analysis of the underlying logical relationships and professional context. Based on this semantic understanding, Text Agent 12 can filter and extract key information with practical evaluation value from the text according to evaluation requirements. Multiple rounds of cross-validation (such as comparing the consistency of the same information in different paragraphs and verifying the accuracy of professional terminology using a domain-standard dictionary) ensure the authenticity and completeness of the extracted information. Finally, Text Agent 12 structures the extracted key information according to a unified document specification, outputting a standardized document. This document not only eliminates differences in format and expression between different original texts, making key information clearly traceable, but also directly connects to the review rules of the subsequent condition review agent 13, providing unified and standardized text data support for the efficient operation of the entire collaborative evaluation system.

[0028] The condition review agent 13 receives structured and / or standardized documents, performs multi-dimensional evaluation based on the received documents and input requests, and generates an evaluation report corresponding to the input requests. For example... Figure 2 As shown, the interactive interface of the multi-agent collaborative evaluation system 10 can be referenced. Figure 2 The interface shown.

[0029] The conditional review agent 13 receives structured documents from the visual agent 11 and / or standardized documents from the text agent 12. Simultaneously, it acquires user input requests (e.g., whether the user meets the requirements for a specific profession) through a system interface, determining the review conditions and assessment objectives based on these requests. Then, based on the received document data and guided by the assessment objectives of the input requests, the conditional review agent 13 initiates a multi-dimensional assessment process. During the assessment, the conditional review agent 13 automatically matches compliance clauses using a rule engine and intelligently predicts key points using a machine learning model. Finally, a structured assessment report is generated based on the results of the multi-dimensional assessment. The assessment report may include assessment conclusions, specific scores for each assessment dimension, key assessment criteria, detailed explanations of key points, and recommendations.

[0030] The aforementioned multi-agent collaborative evaluation system 10 utilizes a visual agent 11 to perform multimodal analysis on the input image data, identifying various types of elements within the images and extracting key information based on element recognition. A structured document is then output based on this key information. Conversely, a text agent 12 performs semantic understanding and information extraction on the input text data, outputting a standardized document based on the extracted information. This approach, employing both visual and text agents, enables focused analysis of specific types of materials, achieving high accuracy and efficiency. Finally, the outputs from both agents are transmitted to a conditional review agent 13. This agent performs a multi-dimensional evaluation based on the received document and input request, generating an evaluation report corresponding to the input request. This multi-agent collaborative evaluation system 10 effectively processes input data and performs multi-dimensional comprehensive evaluation, obtaining the final evaluation result based on the input request. It achieves in-depth understanding, conditional judgment, and final evaluation of the content of qualification certification materials, effectively improving review efficiency.

[0031] In some embodiments, the visual agent 11 is also used to: when handwritten text is recognized in image data, perform contrast enhancement processing on the handwritten text, determine the connection of strokes based on the distance between stroke points and the width of strokes, and split the connected characters.

[0032] In applications, image data may contain handwritten text. Therefore, the visual agent 11 needs to recognize and process handwritten text. This embodiment uses the method described above to achieve this recognition and processing. The contrast enhancement formula can be: New pixel value = Original pixel value × Contrast coefficient + Brightness adjustment.

[0033] It is understandable that when the visual agent 11 detects handwritten text in an image using image recognition technology, it can perform contrast enhancement processing. By dynamically adjusting the pixel brightness and color contrast of the handwritten text area, background interference (such as paper texture, stains, and shadows) is weakened, and the clarity of the character strokes is enhanced, making the originally blurry handwritten trajectory recognizable, laying the foundation for subsequent stroke analysis. Based on this, to address potential issues of broken or excessively connected strokes in handwritten text, the visual agent 11 compares the spatial distance between adjacent stroke points and the stroke width to determine stroke connectivity. When the distance between two stroke points is less than half the stroke width, it is judged as connected strokes. For connected text, the visual agent 11 can first binarize the handwritten text area, converting the color image into a binary image to eliminate background interference and clarify the stroke outlines; then, it can perform noise reduction optimization on the binary image, using median filtering or Gaussian filtering to remove noise generated during handwriting, preventing noise from being misjudged as "connected strokes"; finally, a preset algorithm is used to split the connected text. Through this series of targeted processing, the visual intelligent agent 11 can effectively overcome the difficulties in recognizing handwritten text, significantly improve the recognition accuracy of handwritten content, ensure that the handwritten information extracted from the image is complete and accurate, and thus provide more comprehensive raw data for the subsequent generation of structured documents, enhancing the entire evaluation system's adaptability to diverse information carriers.

[0034] It should be noted that decomposing cursive characters can capture their decomposition features. On one hand, image edge detection algorithms (such as the Canny operator) can be used to extract the outline information of the strokes, calculate the direction of the strokes, local curvature, and changes in stroke width, and record the coordinates of stroke intersections and forks. On the other hand, character-level structural features can be combined with the inherent structural rules of the character type (such as the typical forms of left-right structure, top-bottom structure, and radicals in Chinese characters) to establish a character structure template library, providing a reference for subsequent decomposition.

[0035] Based on the features extracted earlier, various algorithms can be used for splitting cursive characters. First, a rule-based splitting method can be employed, pre-setting splitting rules based on the extracted features. For example, when the angle between adjacent strokes exceeds 120° (indicating a sudden change in stroke direction, possibly a character boundary) or the stroke width difference exceeds 30% (indicating strokes of different characters), the splitting point is automatically marked. Next, pre-trained machine learning models can be used to improve the splitting accuracy of complex cursive characters. For instance, convolutional neural networks (CNNs) can learn the splitting features of a large number of cursive text samples to accurately identify the boundary features of "multi-character cursives." For extremely complex cursives, semantic segmentation networks (such as U-Net) can be used to directly segment the cursive region at the pixel level, clearly defining the pixel range of different characters. Finally, contextual semantics can be combined to assist in splitting. Scene semantics can be used to narrow down the candidate character range (e.g., cursive in the "Contact Information" field is more likely to be numbers or letters than complex Chinese characters), while also referring to adjacent, clearly identified characters to help determine the splitting boundaries.

[0036] In some embodiments, the visual agent 11 is further configured to: perform color feature recognition in the image data to determine the candidate region of the seal, perform shape matching on the candidate region of the seal, and determine whether a seal exists in each candidate region of the seal based on the matching result.

[0037] In applications, seal recognition can be performed using both color recognition and shape matching. Based on the typical color attributes of seals in real-world scenarios, a color thresholding algorithm can be used to extract pixel regions matching the color characteristics from the image by using a preset range of target seal colors (e.g., red value > 150, green < 100, and blue value < 100 for a red seal). This process ultimately identifies multiple candidate seal regions. Furthermore, the shape of the seal can be used to determine whether a seal exists in each candidate region. For example, seals are typically circular, and a simple formula can be used to determine this: Circularity = (Circumference × Circumference) ÷ (Area × 4 × π). If the circularity is close to 1, it indicates a very round shape and is likely a seal; if the circularity is far from 1, it indicates a non-circular shape and is not a seal.

[0038] In some embodiments, the visual agent 11 is further configured to: identify image data based on a preset fuzzy image recognition algorithm, determine whether there is a fuzzy image in the image data and the degree of fuzziness, and when there is a fuzzy image, adjust the core parameters of the sharpening filter algorithm based on the degree of fuzziness, and use the sharpening filter algorithm to process the fuzzy image.

[0039] It is understandable that images in image data may be blurry. Therefore, when a blurry image is detected, it needs to be processed for easier recognition. For blurry image recognition, the visual agent 11 can call a preset blurry image recognition algorithm to perform a full-area scan of the image data. For example, since the essence of blurriness is a decrease in the rate of gray-level change at the edges (a clear image has a sudden change in gray-level at the edges, while a blurry image has a slow transition in gray-level at the edges), and the gradient is the core indicator for measuring the rate of gray-level change, the smaller the gradient value, the more blurry the image. Therefore, the intensity of gray-level change (gradient value) of each pixel in the image can be calculated using a gradient operator (such as the Sobel operator), and whether the image is blurry is determined based on whether the gradient value is within the blurry range. After completing the blur determination, the visual agent 11 will further quantify the degree of blur based on the gradient value, usually dividing it into three levels: mild, moderate, and severe, and recording the coordinate range of the blurry area.

[0040] When a blurred image exists, the core parameters of the Unsharp Filter (USM) algorithm are dynamically adjusted based on the degree of blur. The USM algorithm is essentially a process of "original image - low-pass image" to obtain high-frequency residuals, and then adding the residuals back to the original image to highlight the edges.

[0041] Let the original image be Gaussian low-pass kernel is ,but: It is a two-dimensional convolution. The "Amount" coefficient controls the magnitude of the enhancement. The bigger and sharper, (or radius r) determines the low-pass cutoff frequency. This means that by applying a "slight blur" to the original image I, we get a negative that "has no details and only large color blocks"; in the frequency domain, this corresponds to "cutting out the high frequencies". This means that by subtracting this "blurred negative" from the original image, what remains is the "high-frequency part that was cut off" (commonly known as the detail layer or residual layer, where edges, textures, and noise are located). This means that by enlarging the detail layer by α times and then adding it back, the edges become steeper and the textures become more prominent.

[0042] After testing with a large amount of experimental data, it was found that the image processing results are optimal when the α value is in the range of 1.1-1.3 and the σ value is in the range of 1.6-1.8 (the experimental results may vary slightly depending on the image clarity).

[0043] In some embodiments, the text agent 12 is further configured to: determine the information density of each paragraph based on the proportion of key information in the paragraph; determine the information gradient of the current detected paragraph based on the information density of the current detected paragraph and the information density of adjacent paragraphs; identify the current detected paragraph as an information surge point if the information gradient of the current paragraph is greater than a preset threshold; optimize the paragraphs not identified as information surge points based on preset optimization rules, the optimization process including at least one of deletion processing and paragraph merging processing; vectorize the sentences in each paragraph; calculate the vector distance of each sentence vector through cosine similarity; identify similar sentences based on the vector distance of each sentence vector and a similarity threshold; and retain the sentences with higher information density among the similar sentences.

[0044] The formula for calculating paragraph information density is: Paragraph Information Density = (Number of keywords × Keyword weight + Number of numerical information × 2 + Number of technical terms × 1.5) ÷ Total number of words in the paragraph; the formula for calculating information gradient is: Information Gradient = Current paragraph density - Average density of adjacent paragraphs. The sentence vectorization formula is: S1 represents the statement vector; the cosine similarity calculation formula can be: S1 represents the statement vector of the first sentence, and S2 represents the statement vector of the second sentence.

[0045] It is understandable that the text agent 12, based on the above formula, identifies key information elements (such as keywords, numerical information, and technical terms) in a paragraph, calculates the total value of key information according to a preset weighting rule, and finally divides this total value by the total number of words in the paragraph to obtain the information density, which reflects the richness of the paragraph's core information. Then, the text agent 12 can further calculate the information gradient of the currently detected paragraph based on the information density. The information gradient value is obtained by subtracting the average density of adjacent paragraphs from the current paragraph density, and compared with a preset threshold. If the gradient value is greater than the threshold, it means that the core information content of the current paragraph is much higher than that of the surrounding paragraphs, and the system marks it as an "information surge point". Such paragraphs will be preferentially and completely preserved as the core content skeleton of the text. For paragraphs not identified as information spikes, the system will process them according to preset optimization rules. Specifically, it will choose between "deletion processing" or "paragraph merging processing" based on the actual situation of the paragraph: If the paragraph is an independent paragraph containing redundant sentences (such as explanatory sentences with semantic repetition > 0.8 with the preceding text, transitional sentences without key information, or irrelevant detailed descriptions), deletion processing will be performed, retaining only short sentences containing a small amount of key information; if the paragraph has a thematic relevance ≥ 50% with adjacent paragraphs and its existence alone is likely to lead to fragmentation, paragraph merging processing will be performed.

[0046] After optimizing at the paragraph level, Text Agent 12 further focuses on redundancy at the sentence level. First, it uses sentence vectorization technology to convert each sentence into a computer-recognizable multi-dimensional numerical vector. Then, it calculates the vector distance between any two sentence vectors using a cosine similarity algorithm and compares the result with a similarity threshold (e.g., 0.8). If the similarity between two sentences is greater than 0.8, they are considered similar (e.g., "The model accuracy reached 92%" and "The model's accuracy in this experiment was 92%"). At this point, Text Agent 12 compares the information density of similar sentences, retaining the sentence with the higher density, thus completely eliminating redundancy at the sentence level. Through this process, while ensuring no loss of core information (keywords, numbers, terminology), it significantly improves the conciseness and information retrieval efficiency of the text. This avoids the core content being buried and solves the problems of fragmented non-critical paragraphs and repetitive sentences, ultimately outputting high-quality text that is logically coherent and highlights key points.

[0047] In some embodiments, the condition review agent 13 is further configured to, upon receiving a structured document and a standardized document, and detecting a conflict in the key type information of the structured document and the standardized document, generate viewpoint adversarial samples and text adversarial samples for the visual agent 11 and the text agent 12 respectively, using gradient perturbation and semantic preservation constraints, with the conflicting field as the core; feed the viewpoint adversarial samples back to the viewpoint agent and the text adversarial samples back to the text agent 12; determine the robustness scores of the viewpoint agent and the text agent 12 based on the output drift of the viewpoint agent and the text agent 12; determine the confidence of the viewpoint agent and the text agent 12 based on the entropy of the output distribution of each agent; use the robustness scores and confidence of the viewpoint agent and the text agent 12 as observations; use a Bayesian algorithm to determine the weight coefficients of the robustness scores and confidence; and, based on the weight coefficients of the robustness scores and confidence, the robustness scores and confidence of the viewpoint agent and the text agent 12, determine one of the output fields of the viewpoint agent and the text agent 12 as the target field of the conflicting field.

[0048] Specifically, a Bayesian algorithm is used to determine the weight coefficients of robustness scores and confidence scores. The one with the higher weight coefficient is used as the primary reference condition, and the one with the lower weight coefficient as the secondary reference condition. First, the parameters corresponding to the primary reference conditions are compared. If the parameters corresponding to the primary reference conditions are the same, then the parameters corresponding to the secondary reference conditions are compared. For example, if the weight coefficient of the robustness score is greater than the weight coefficient of the confidence score, then the robustness scores of the viewpoint agent and the text agent 12 are compared first, and the output field of the agent with the higher score is determined as the target field. If the robustness scores of the viewpoint agent and the text agent 12 are the same, then the confidence scores of the viewpoint agent and the text agent 12 are compared, and the output field of the agent with the higher confidence score is determined as the target field.

[0049] For example, if the field set F = {graduation year, degree type, ...}, then for any field... If | ( This indicates the output of visual agent 11 for this field. This indicates the output of text agent 12 for this field, where εf is the field-related tolerance (e.g., triggering if the year difference is greater than 1 year). Then, condition review agent 13 initiates the arbitration process.

[0050] For example, the arbitration process may specifically include steps one through five as described below.

[0051] Step 1: Adversarial Example Generation. If field f triggers a conflict, the conditional review agent 13 generates adversarial examples for both the visual agent 11 and the text agent 12, forcing either the visual agent 11 or its output to approximate the opponent's current value while satisfying perceptual / semantic consistency constraints. Visual adversarial examples : in, Let be the objective function. Let be the perturbation vector. For cosine similarity, For vector dot product, Norm (Euclidean norm) It is a loss function. The output of visual agent 11 against adversarial examples. This is the current output of text agent 12 (as "attack target").

[0052] The formula has two constraints: (1) (Perturbation vector) Norm constraint: Limits the maximum perturbation amplitude for each pixel. Within the range of 0-255 pixel values, each pixel can change by a maximum of 2 units; (2) This indicates that the adversarial sample is 97% structurally similar to the original image.

[0053] Text adversarial examples :

[0054] Rewrite the document text using synonyms (e.g., “June 2019” → “Summer 2019”), so that T-Agent(f) is pushed toward the value of V-Agent(f).

[0055] Constraints: Edit distance ≤ 3, semantic similarity (SimCSE) ≥ 0.95.

[0056] Adversarial example generation formula:

[0057] in, () indicates edit distance. The output of the text agent against 12 adversarial examples. This is the current output of visual agent 11. The adversarial example generation formula has two constraints: (1) This means a maximum of 3 editing operations are allowed.

[0058] (2) The threshold indicates that the adversarial text semantics are 95% similar to the original text.

[0059] Step two, robustness testing. The adversarial examples obtained in Step one are fed back into the original agent, and its output drift is calculated to obtain the robustness score. The higher the score, the more robust it is.

[0060] Visual agent 11 robustness score:

[0061] Text-based agent 12 robustness score:

[0062] in, This represents the output drift of visual agent 11. This represents the output drift of text agent 12. The output of the visual agent 11 to the original image. For the visual agent 11, Range(f) represents the output of the adversarial image, where Range(f) represents the normalization of the output range.

[0063] Output Completely robust, ineffective against attacks.

[0064] Output : Not robust at all, output drifts to the maximum.

[0065] Output Relatively robust, with output drifting by 20% of the possible range.

[0066] Step 3, Confidence Calculation. Each agent gives a confidence score based on the entropy of its own output distribution. .

[0067] Confidence level of visual agent 11: .

[0068] Text agent 12 confidence level: .

[0069] in , Entropy is the average distribution of outputs with higher entropy and greater uncertainty. Conversely, lower entropy results in a more concentrated and deterministic output distribution. For temperature parameters, It can amplify the effect of entropy, making small entropy differences significantly amplified, and can also make confidence differences more obvious.

[0070] Step 4, Bayesian fusion. Treating the outputs of both as observations, construct a likelihood function and calculate the posterior maximum to obtain the final field value y. Then, apply the Bayesian multi-agent fusion decision formula, as follows: in, Given two agent observations, the posterior probability of the true field value y is given by ∝, which represents a direct proportional relationship (ignoring the normalization constant).

[0071] In this formula, It is the likelihood function of the visual agent 11. This indicates that the output of visual agent 11 is the average. It is variance, determined by robustness and confidence level. It is the likelihood function of text agent 12, based on the same principle as visual agent 11, but with independently calculated parameters. Additionally, in middle, A value close to 1 indicates that the agent is both robust and confident → small variance → strong constraints. A value close to 0 indicates that the agent is not robust and lacks confidence → large variance → weak constraints.

[0072] Step 5, online learning. Learn the conflict triples. Store in playback buffer B; every time |B|=100, use Fine-tuning the local parameters of V and T continuously improves robustness. Among these, : Original image input, x t Raw text input, : The final value predicted by the system (Bayesian fusion result).

[0073] And for The use of fine-tuning formulas is reflected in: (1) Visual agent 11 parameter update: (2) Text agent 12 parameter update: in, Is it visual agent 11 in terms of parameters? The following is a prediction of the input x, where y is the true label. It is a loss function that measures the difference between the prediction and the true value. This represents the sum of batch losses. The gradient is calculated as the partial derivative of the loss function with respect to the model parameters, which indicates the direction and magnitude of parameter updates. This is the learning rate.

[0074] In some embodiments, the condition review agent 13 is used to perform semantic recognition on the input request, determine the first condition information contained in the input request, and obtain the second condition information corresponding to the condition keywords according to the condition keywords of the input request; perform condition review evaluation on the received document based on the first condition information and the second condition information, and generate an evaluation report corresponding to the input request based on the condition review results.

[0075] In the application, the condition review agent 13 can obtain the user's input request and then initiate semantic recognition based on the user's input request to accurately extract the "constraints explicitly required by the user" from the input request, i.e., the first condition information. During recognition, it will prioritize core content containing qualifiers such as "satisfied" and "compliant". At the same time, the system will automatically trigger a preset "condition information knowledge base" matching based on the condition keywords (such as occupation) in the input request to obtain the second condition information corresponding to the keywords. The second condition information is not actively mentioned by the user, but is an "objective reference standard" strongly associated with the keyword. The source can include industry compliance norms, internal review benchmarks of the enterprise, common requirements in historical review cases, etc. For example, for the keyword "warranty period", the second condition information may be "the industry default warranty period is ≥6 months".

[0076] Subsequently, the conditional review agent 13 can enter the document conditional review and evaluation stage. This stage uses "first conditional information as the core constraint and second conditional information as a supplementary reference" to conduct a precise comparison of the received document: First, text retrieval technology is used to locate the content in the document corresponding to the conditional keywords. Then, information density rules are used to prioritize the extraction of core expressions containing key numbers and qualifiers from the document. Next, the extracted information from the document is compared and evaluated with the first and second conditional information respectively, and an evaluation report is output based on the comparison and evaluation results. For example, if the document information meets the first conditional information, the output is "Meets the first conditional information and meets the user's core needs." If the second condition is also met, the output can be supplemented with "Meets the second conditional information and meets industry reference standards." Based on the above review, the conditional review agent 13 generates an evaluation report corresponding to the input request. The report does not simply list the comparison results, but outputs the conclusion, evaluation results of each dimension, and key information in sequence. For dimensions that are not met or lack information, further actionable suggestions can be given. This report can clearly reflect the review results and can significantly improve review efficiency and accuracy.

[0077] Based on the same inventive concept, this application also provides a multi-agent cooperative evaluation method, which is executed based on a multi-agent cooperative evaluation system of any of the above schemes. Figure 3 As shown, the multi-agent collaborative evaluation method includes the following steps S301 to S304.

[0078] S301: Obtain input data and determine the type of input data.

[0079] S302: When the input data includes image data, a visual intelligent agent performs multimodal analysis on the input image data, identifies various types of elements in the image data, extracts key information from the image data based on the element identification information, and outputs a structured document based on the key information.

[0080] S303: When the input data includes text data, a text agent is used to perform semantic understanding and information extraction on the input text data, and the extracted information is used to input a standardized document.

[0081] S304: The condition review agent receives structured and / or standardized documents, performs multi-dimensional evaluation based on the received documents and input requests, and generates an evaluation report corresponding to the input requests.

[0082] It should be noted that the multi-agent cooperative evaluation method and the multi-agent cooperative evaluation system provided in this application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned multi-agent cooperative evaluation method, and the repeated parts will not be described again.

[0083] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the above-described multi-agent cooperative evaluation method.

[0084] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0085] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0086] This application also provides a computer-readable medium storing a computer program thereon, which, when executed by a processor, implements the multi-agent cooperative evaluation method described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0087] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0088] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A multi-agent collaborative evaluation system, characterized in that, include: Visual agents, text-based agents, and conditional review agents; The visual agent is used to perform multimodal analysis on the input image data, identify various types of elements in the image data, extract key information from the image data based on the element identification information, and output a structured document based on the key information. The text agent is used to perform semantic understanding and information extraction on the input text data, and output standardized documents based on the extracted information; The conditional review agent is used to receive the structured document and / or the standardized document, and perform multi-dimensional evaluation based on the received document and input request to generate an evaluation report corresponding to the input request. The conditional review agent is also used to, upon receiving the structured document and the standardized document, and detecting a conflict in key type information between the structured document and the standardized document, generate viewpoint adversarial samples and text adversarial samples respectively for the visual agent and the text agent, using gradient perturbation and semantic preservation constraints, with the conflicting field as the core. The viewpoint adversarial samples are then fed back to the visual agent, and the text adversarial samples are fed back to the text agent. This agent determines the robustness scores of the visual agent and the text agent based on the output drift of the visual agent and the text agent, determines the confidence scores of the visual agent and the text agent based on the entropy of the output distribution of each agent, uses the robustness scores and confidence scores of the visual agent and the text agent as observations, and uses a Bayesian algorithm to determine the weight coefficients of the robustness scores and confidence scores. Based on the weight coefficients of the robustness scores and confidence scores, and the robustness scores and confidence scores of the visual agent and the text agent, one of the output fields of the visual agent and the text agent is determined as the target field of the conflict field.

2. The multi-agent cooperative evaluation system as described in claim 1, characterized in that, The visual intelligent agent is also used to: when handwritten text is recognized in the image data, perform contrast enhancement processing on the handwritten text, determine the stroke connection based on the distance between stroke points and the stroke width, and split the connected characters.

3. The multi-agent cooperative evaluation system as described in claim 1, characterized in that, The visual agent is also used to: identify color features in the image data to determine candidate areas for seals, perform shape matching on the candidate areas for seals, and determine whether a seal exists in each candidate area for seals based on the matching results.

4. The multi-agent cooperative evaluation system as described in claim 1, characterized in that, The visual intelligent agent is also used to: identify the image data based on a preset blurry image recognition algorithm, determine whether there is a blurry image in the image data and the degree of blurriness, and when there is a blurry image, adjust the core parameters of the sharpening filtering algorithm based on the degree of blurriness, and use the sharpening filtering algorithm to process the blurry image.

5. The multi-agent cooperative evaluation system as described in claim 1, characterized in that, The text intelligence agent is also used to: determine the information density of each paragraph based on the proportion of key information in the paragraph; determine the information gradient of the current detected paragraph based on the information density of the current detected paragraph and the information density of adjacent paragraphs; identify the current detected paragraph as an information surge point if the information gradient of the current paragraph is greater than a preset threshold; optimize paragraphs not identified as information surge points based on preset optimization rules, the optimization process including at least one of deletion and paragraph merging; vectorize the sentences in each paragraph; calculate the vector distance of each sentence vector using cosine similarity; identify similar sentences based on the vector distance of each sentence vector and a similarity threshold; and retain sentences with higher information density among similar sentences.

6. The multi-agent cooperative evaluation system as described in any one of claims 1 to 5, characterized in that, The condition review agent is used to perform semantic recognition on the input request, determine the first condition information contained in the input request, and obtain the second condition information corresponding to the condition keywords based on the condition keywords of the input request. The received document is evaluated based on the first and second condition information, and an evaluation report corresponding to the input request is generated based on the evaluation results.

7. A multi-agent collaborative evaluation method, characterized in that, The method is performed based on the multi-agent cooperative evaluation system as described in any one of claims 1 to 6; the method includes: Obtain input data and determine the type of the input data; When the input data includes image data, a visual intelligent agent performs multimodal analysis on the input image data, identifies various types of elements in the image data, extracts key information from the image data based on the element identification information, and outputs a structured document based on the key information. When the input data includes text data, a text intelligence agent is used to perform semantic understanding and information extraction on the input text data, and the extracted information is used to input standardized documents. The condition review agent receives the structured document and / or the standardized document, and performs a multi-dimensional evaluation based on the received document and the input request to generate an evaluation report corresponding to the input request. When the structured document and the standardized document are received, and a conflict is detected between the key type information of the structured document and the standardized document, the conflict field is used as the core to generate viewpoint adversarial samples and text adversarial samples for the visual agent and the text agent respectively using gradient perturbation and semantic preservation constraints. The adversarial sample from the viewpoint is fed back to the visual agent, and the adversarial sample from the text is fed back to the text agent. Robustness scores for the visual and text agents are determined based on their output drift. Confidence levels for the visual and text agents are determined based on the entropy of their output distributions. Using the robustness scores and confidence levels of the visual and text agents as observations, a Bayesian algorithm is used to determine the weighting coefficients for the robustness scores and confidence levels. Based on these weighting coefficients, the robustness scores, and confidence levels of the visual and text agents, one of the output fields of the visual and text agents is selected as the target field for the conflict field.

8. An electronic device, characterized in that, It includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the multi-agent cooperative evaluation method as described in claim 7.

9. A computer storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the multi-agent cooperative evaluation method as described in claim 7.

Images