A multi-level auditing and shelving management system, method and medium for an intelligent agent

Abstract

The application relates to a multi-level review and shelving management system, method and medium for an intelligent agent, and relates to the technical field of intelligent agent review management. The multi-level review and shelving management system comprises a state conversion module, a double review module, an analysis and shelving module, a traceability management module and a permission verification module. The state conversion module is used for detecting an intelligent agent according to a preset node state machine model, verifying a state conversion path of the intelligent agent and determining a state of the intelligent agent. The double review module is used for verifying stage state elements of the intelligent agent according to a state review rule in combination with the state of the intelligent agent, and constructing an intelligent agent review work order. The analysis and shelving module is used for quantifying the intelligent agent that has been shelved according to a multi-dimensional risk assessment mechanism in combination with the intelligent agent review work order, calculating a comprehensive risk score and determining a shelving type. Through a seven-state node state machine model, a double review mechanism, a forced shelving identification technology and an audit dimension quantitative scoring algorithm, accurate management and safety control of the intelligent agent in the whole life cycle are realized.

Description

Technical Field

[0001] This application relates to the field of intelligent agent review and management technology, and in particular to a multi-level review and listing management system, method and medium for intelligent agents. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI intelligent agent service platforms are gradually becoming an important bridge connecting service providers and enterprise users.

[0003] Existing app store review models (Apple App Store, Google Play) primarily employ a one-time review mechanism; once approved, the app can be listed, lacking dynamic monitoring of already listed apps. E-commerce platform product management systems (Taobao, JD.com), while possessing listing and delisting functions, mainly focus on physical goods and lack review dimensions specific to AI services (such as algorithm security, data privacy protection, and model interpretability). SaaS service marketplaces (Salesforce AppExchange, Microsoft AppSource) offer basic review and listing functions, but their status transition logic is simplistic, lacking multi-level review and mandatory delisting mechanisms.

[0004] Existing patents disclose an optimization method, system, terminal, and medium for a change scheme review intelligent agent based on a large model. The method includes: receiving and parsing change schemes; identifying the scheme type and classifying it into multiple categories based on preset classification rules; allocating parallel processing thread pools to each category, and calling a large model inference framework to review the schemes within each thread pool; generating phased review results during the review process and dividing the review task into multiple stages according to a preset workflow; transmitting the phased results to the user terminal in real time via a streaming output mechanism, and summarizing them after all stages are completed to form a comprehensive review report containing risk point association information. This invention can improve processing efficiency while ensuring review accuracy, and enhance the user's real-time perception of the review process and the interpretability of the results. It is applicable to change scheme review and decision support in various complex scenarios.

[0005] The existing technical solutions mentioned above have the following defects: 1. Existing systems usually only provide a one-time review mechanism. Once the review is passed, the system can be put on the platform. They lack the ability to continuously monitor the intelligent agents after they are put on the platform. They cannot cope with issues such as changes in service quality and dynamic evolution of security risks. As a result, there may be potential risks in the intelligent agents that have been put on the platform, but they cannot be intervened in a timely manner. 2. The lack of clear rule definitions and state machine model support for the state transitions of intelligent agents leads to inconsistent understanding of the current state between service providers and platform administrators, unclear state transition paths, easy occurrence of illegal state transitions, chaotic operation permissions, and poor system stability. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this application is to provide a multi-level review and listing management system, method, and medium for intelligent agents. Through a seven-state node state machine model, a dual review mechanism, a forced delisting identification technology, and a review dimension quantitative scoring algorithm, it achieves precise management and security control of the entire lifecycle of intelligent agents.

[0007] This was achieved using the following technical solutions: In a first aspect, this application provides a multi-level review and listing management system for intelligent agents, comprising: The state transition module is used to detect agents based on a preset node state machine model, verify the state transition path of the agents, and determine the state of the agents. The dual-review module is used to verify the stage state elements of the agent based on the status review rules and the agent's status, and to construct the agent review work order. The delisting module is used to quantify the listed agents based on a multi-dimensional risk assessment mechanism and agent review work orders, calculate a comprehensive risk score, and determine the delisting type. The traceability management module is used to monitor the entire process of the intelligent agent's review operation, create lifecycle review paths, mark status change nodes, and trigger change event notifications. The permission verification module is used to verify and allocate the execution operation permissions of the agent according to the agent's state and operator type, based on the dynamic permission control matrix.

[0008] By adopting the above technical solutions, based on the state machine model and multi-dimensional risk assessment algorithm, and through dual review and dynamic permission control matrix, closed-loop management of the entire life cycle of intelligent agents is achieved, which significantly improves the standardization of review and risk control capabilities.

[0009] Furthermore, the state transition module includes: The model parsing unit is used to parse the preset node state machine model and extract the semantic labels of state nodes and the skeleton of state transition rules. The state matching unit is used to perform semantic matching on the agent based on the semantic labels of the state nodes to obtain the source state of the agent; The state evolution unit is used to transform and evolve the source state of the intelligent agent according to the state transition rule skeleton and the target state to obtain the state evolution matrix; The path verification unit is used to verify the state transition path of the agent based on the state transition request and the state evolution matrix. If the test passes, the current state evolution matrix is ​​deemed compliant, and the agent's source state is transformed into the agent's target state. If not, then the current state evolution matrix is ​​determined to be in violation, abnormal transition nodes are extracted, the state transition rule skeleton is optimized, and the current agent state is maintained.

[0010] By adopting the above technical solution, based on state machine model parsing and semantic matching algorithms, the automatic verification and rule optimization of agent state transitions are realized through state evolution matrix and path verification, which significantly improves the accuracy, compliance and adaptability of state transitions.

[0011] Furthermore, the dual-verification module includes: The instruction triggering unit is used to deconstruct the state of the intelligent agent according to the business scenario and extract the audit type identifier and the service provider credit rating; The status verification unit is used to verify the status of the intelligent agent based on the audit type identifier. If the review type is marked as "Release Review", then the current agent status is determined to be "Pending Submission". If the review type is marked as "listing review", then the current agent status is determined to be "approved". No, the review request will be rejected and an illegal operation log will be recorded; The audit configuration unit is used to filter status audit rules and extract audit verification items based on the type of intelligent agent and the credit rating of the service provider. The stage verification unit is used to verify each audit verification item according to the audit type identifier and mark the pre-verification result; If it is a release review, then the format completeness, content compliance and validity period of the current review verification items will be verified. If all verifications pass, update the current agent status to pending listing and generate a listing review instruction. If it is for listing review, then based on historical operation data and the current review verification items, the functional availability, service response time and user experience score of the intelligent agent will be analyzed according to the listing review instructions; If all verifications pass, the current agent status will be updated to "listed" and services will be available to the public. The work order generation unit is used to associate and aggregate the pre-verification results, audit type identifiers, and items to be audited based on the agent identifier to generate an agent audit work order.

[0012] By adopting the above technical solution, the review rules are dynamically matched based on the review type and credit rating. Through two-stage verification of release and uploading, the format, content, function and service quality of the intelligent agent are automatically verified and a review work order is generated. This achieves refined and standardized closed-loop management of the review process, which significantly improves the efficiency and reliability of the review.

[0013] Furthermore, the module for parsing and removing apps includes: The target filtering unit is used to filter agents based on their status tags and extract agents that have been listed, their review dimension scores, and their historical risk coefficients. The data acquisition unit is used to detect the operation process of the deployed intelligent agents according to the intelligent agent review work order and collect multi-source dynamic operation data; The single-dimensional assessment unit is used to score multi-source dynamic operating data based on the review dimension scoring combined with the multi-dimensional risk assessment mechanism, and calculate the single-dimensional risk score. The comprehensive judgment unit is used to calculate the comprehensive risk score by weighting and correcting the single-dimensional risk score based on the agent type and historical risk coefficient. The risk level matching unit is used to match the comprehensive risk score according to the preset risk level threshold to determine the risk level of the agent. The delisting judgment unit is used to determine the delisting type based on the risk level of the intelligent agent and the delisting request response; If the risk level of the intelligent agent is low or medium and there is a delisting request response, then the delisting type of the current intelligent agent is determined to be active delisting; If the risk level of the agent is high, then the current agent's removal type is determined to be forced removal.

[0014] By adopting the above technical solution, based on multi-source dynamic operation data and historical risk coefficients, a comprehensive risk score is calculated through weighted correction, and risk levels are matched according to risk level thresholds. Combined with the automatic determination of active or forced delisting in response to delisting requests, the automation, multi-dimensional quantification and risk classification management of intelligent delisting are realized, which significantly improves the accuracy of decision-making and operational safety.

[0015] Furthermore, the multidimensional risk assessment mechanism includes: The multi-source dynamic operating data is quantitatively evaluated based on data dimensions, and a single-dimensional score is calculated. If the data is security data, the agent is scored based on the vulnerability level and the level decay coefficient, and the security dimension score is calculated. If the data is functional, the agent is evaluated based on the duration of service anomalies and the time-period decay coefficient, and a functional dimension score is calculated. If the data is compliant, the compliance assessment of the agent is performed based on the number of service evaluations combined with the abnormal attenuation deduction coefficient, and the compliance dimension score is calculated. The single-dimensional scores are weighted and normalized according to the type of agent to obtain the final comprehensive risk score. If the final comprehensive risk score is within the first risk threshold range, or if the score of any dimension is within the first scoring threshold range, then the current agent's risk level is determined to be high. If the final overall risk score is within the second risk threshold range, but at least two dimension scores are within the second scoring threshold range, then the current agent's risk level is determined to be medium. If the final comprehensive risk score is within the third risk threshold range, and the scores of each dimension are also within the third scoring threshold range, then the current agent's risk level is determined to be low.

[0016] By adopting the above technical solutions, based on multi-dimensional quantitative data of security, functionality, and compliance, the risk level of intelligent agents is automatically assessed through weighted fusion and threshold grading algorithms, realizing refined and automated graded management of risks, and significantly improving the accuracy of risk assessment and decision-making efficiency.

[0017] Furthermore, the traceability management module includes: The intelligent capture unit is used to capture the entire process of the intelligent agent's review operation and extract state transition events; The record creation unit is used to associate state transition events with intelligent agent review work orders based on state transition type, and generate independent review records; The path building unit is used to sort independent audit records according to time series and build lifecycle paths; The change marking unit is used to transform and mark the lifecycle path to obtain the state change node; The state management unit is used to fuse and map the agent's state, lifecycle path, and state change nodes to generate a state transition mapping diagram. The notification tracing unit is used to associate and push state transition events with state transition maps according to event priority, and generate change event notifications.

[0018] By adopting the above technical solution, based on the event capture and association mapping algorithm, the intelligent agent state transition events are automatically recorded and associated with the audit work order, a full life cycle path and state transition mapping diagram is constructed, and change event notifications are triggered through a priority push mechanism. This achieves traceable and auditable closed-loop management of the entire audit operation process, significantly improving operation and maintenance transparency and event response efficiency.

[0019] Furthermore, the dynamic access control matrix includes a creator access matrix and a manager access matrix; among which, The creator permission matrix is ​​as follows: If the agent status is pending submission, the edit, delete, and submit for review operations are set to 1; if the agent status is pending review, the withdraw operation is set to 1; if the agent status is not approved, the edit, delete, and resubmit operations are set to 1; if the agent status is approved, the edit, delete, and submit for listing review operations are set to 1; if the agent status is listed, the active delisting operation is set to 1. The administrator permission matrix is ​​as follows: If the agent status is "removed from the market", the "submit for review" operation is set to 1; If the agent status is "pending review", the "approved", "rejected", and "view details" operations are set to 1; If the agent status is "approved", "rejected", or "removed from the market", the "view details" operation is set to 1; If the agent status is "listed", the "view details" and "force removal" operations are set to 1.

[0020] By adopting the above technical solution, and through a dual permission matrix of creators and administrators, the system dynamically grants operation permissions such as editing, reviewing, and delisting based on the state of the intelligent agent. This achieves refined and differentiated access control based on the state machine model, improving the security, compliance, and flexibility of permission management.

[0021] Secondly, this application also provides a multi-level review and listing management method for intelligent agents, which adopts the following technical solution; A method for multi-level review and listing management of intelligent agents includes: The agent is detected based on the preset node state machine model, and the state transition path of the agent is verified to determine the state of the agent. Based on the status review rules and the agent's status, verify the agent's stage status elements and construct an agent review work order. Based on the multi-dimensional risk assessment mechanism and the intelligent agent review work order quantification of the intelligent agents that have been put on the shelves, the comprehensive risk score is calculated to determine the type of delisting. Monitor the entire audit process of the intelligent agent, create lifecycle audit paths, mark status change nodes, and trigger change event notifications; Based on the agent's state and operator type, the execution operation permissions of the agent are verified and allocated according to the dynamic permission control matrix.

[0022] By adopting the above technical solution, QR code areas are accurately extracted from production line data through data fusion and template matching algorithms. Based on multi-dimensional quantitative analysis and weighted evaluation, and combined with threshold grading, the quality of QR codes is automatically and standardizedly rated, thereby improving the accuracy and efficiency of detection.

[0023] Thirdly, this application also provides a storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the multi-level review and listing management method for intelligent agents as described above.

[0024] In summary, the beneficial technical effects of this application are as follows: The quality of intelligent agents is ensured through a dual review mechanism, including the release of review and verification qualifications and the implementation of review and verification functions, with multiple layers of checks; the quantitative scoring system transforms subjective review into objective scoring, ensuring unified review standards and high consistency of results; and the complete record of review history supports review decision analysis and process optimization, continuously improving review efficiency. The system monitors the operational status of the intelligent agent in real time through a risk assessment mechanism, proactively identifying potential problems and transforming passive response into proactive defense; the state machine model ensures the legality of state transitions, avoids illegal operations, and ensures high system stability; the permission separation design clearly separates the permissions of service providers and the platform, avoids unauthorized operations, and ensures strong security. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the multi-level review and listing management system in this application; Figure 2 This is a schematic diagram of the dual-review module in this application; Figure 3 This is a flowchart illustrating the multidimensional risk assessment mechanism in this application; Figure 4 This is a flowchart illustrating the multi-level review and listing management method in this application. Detailed Implementation

[0026] The present application will be further described in detail below with reference to the accompanying drawings.

[0027] Reference Figure 1 This application discloses a multi-level review and listing management system for intelligent agents, comprising: The state transition module is used to detect agents based on a preset node state machine model, verify the state transition path of the agents, and determine the state of the agents. The dual-review module is used to verify the stage state elements of the agent based on the status review rules and the agent's status, and to construct the agent review work order. The delisting module is used to quantify the listed agents based on a multi-dimensional risk assessment mechanism and agent review work orders, calculate a comprehensive risk score, and determine the delisting type. The traceability management module is used to monitor the entire process of the intelligent agent's review operation, create lifecycle review paths, mark status change nodes, and trigger change event notifications. The permission verification module is used to verify and allocate the execution operation permissions of the agent according to the agent's state and operator type, based on the dynamic permission control matrix.

[0028] In this embodiment, after a smart agent service provider completes the basic information and functional configuration of the smart agent on the platform, it submits the application for release review. The system uses a seven-state machine to transition the smart agent from the "pending submission" state to the "pending review" state and automatically triggers a release review work order. Based on a quantitative scoring algorithm of four dimensions—security, functional completeness, document completeness, and compliance—a preliminary scoring radar chart is generated for reviewers' reference. After the review is passed, the smart agent enters the "passed" state but is not publicly visible. The service provider then submits the application for listing review. The system conducts a second evaluation based on indicators such as functional availability and service quality. After the review is passed, the smart agent enters the "listed" state and the service is made available. During operation, the platform uses a risk assessment algorithm to comprehensively consider security vulnerabilities and user complaints. The system dynamically calculates risk levels based on data such as service quality. If the risk level reaches "high," it automatically enforces a forced removal, setting the intelligent entity to "removed" and marking `forcedOffline` as true. This restricts its relisting to require platform re-review. If the service provider voluntarily removes the entity, it is simply changed to a normal removal status. Throughout the process, the permission verification module dynamically controls editing, withdrawal, review, and removal permissions based on the intelligent entity's current status and operator type. The traceability management module fully records every status change and review history and pushes information such as status changes, review results, and risk warnings to relevant parties through a real-time notification mechanism. This ensures that the entire lifecycle management of the intelligent entity from creation to operation is standardized, traceable, and risk-controllable.

[0029] Preferably, the state transition module includes: The model parsing unit is used to parse the preset node state machine model and extract the semantic labels of state nodes and the skeleton of state transition rules. The state matching unit is used to perform semantic matching on the agent based on the semantic labels of the state nodes to obtain the source state of the agent; The state evolution unit is used to transform and evolve the source state of the intelligent agent according to the state transition rule skeleton and the target state to obtain the state evolution matrix; The path verification unit is used to verify the state transition path of the agent based on the state transition request and the state evolution matrix. If the test passes, the current state evolution matrix is ​​deemed compliant, and the agent's source state is transformed into the agent's target state. If not, then the current state evolution matrix is ​​determined to be in violation, abnormal transition nodes are extracted, the state transition rule skeleton is optimized, and the current agent state is maintained.

[0030] In this embodiment, seven state nodes are established through the state definition module: TO_BE_SUBMITTED (pending submission), PENDING (pending review), APPROVED (approved), REJECTED (not approved), LAUNCHED (listed), OFFLINE (removed), and CANCELLED (cancelled). Each state node has a clear semantic definition and allowed transition paths. State transitions must follow predefined rules, and the agent can only be in one definite state at any given time.

[0031] The state machine engine controls the legal transition paths through a state transition matrix: TO_BE_SUBMITTED can transition to PENDING or CANCELLED; PENDING can transition to APPROVED, REJECTED, or TO_BE_SUBMITTED; REJECTED can transition to PENDING or CANCELLED; APPROVED can transition to PENDING or OFFLINE; LAUNCHED can transition to OFFLINE; OFFLINE can transition to PENDING. The system verifies the legality of each state transition, records the transition history (including source state, target state, operator, operation time, and transition reason), and triggers a state change event to notify relevant parties.

[0032] In this embodiment, a platform operator triggers a forced removal operation for a smart agent that has already been listed due to a surge in user complaints and a sharp drop in security vulnerability scores. When the system calls the state transition module, the model parsing unit first extracts the transition rules from "LAUNCHED" to "OFFLINE" in the node state machine model and the semantic constraints of the forcedOffline identifier. The state matching unit matches the source state of the smart agent to "LAUNCHED" based on the actual running data of the current smart agent. The state evolution unit generates a state evolution matrix containing the forcedOffline field by combining the target state "OFFLINE" with the operation type of forced removal. After receiving the state transition request, the path verification unit checks the legality of the transition based on the matrix. It confirms that the transition from "LAUNCHED" to "OFFLINE" with forcedOffline set to true conforms to the predefined rule skeleton. After the verification is passed, the smart agent's state is successfully updated to "OFFLINE" and the reason for forced removal is marked.

[0033] Subsequently, the service provider attempted to submit the application directly for review. The system called the state transition module again. The model parsing unit read the constraint in the rule skeleton that under the "Removed (forcedOffline=true)" state, only platform re-review is allowed and the service provider is prohibited from submitting directly. The state matching unit identified the source state as "Removed (Forced)". The state evolution unit generated a target matrix pointing to "Pending Review". The path verification unit found that the transition path was illegal, determined that the state evolution matrix was abnormal, and extracted the abnormal transition node as "Service Provider Unauthorized Submission". The system automatically kept the current agent in the "Removed" state and recorded the abnormal path and fed it back to the state transition rule skeleton optimization queue for the platform to improve the review process rules in the future.

[0034] Preferably, refer to Figure 2 The dual-verification module includes: The instruction triggering unit is used to deconstruct the state of the intelligent agent according to the business scenario and extract the audit type identifier and the service provider credit rating; The status verification unit is used to verify the status of the intelligent agent based on the audit type identifier. If the review type is marked as "Release Review", then the current agent status is determined to be "Pending Submission". If the review type is marked as "listing review", then the current agent status is determined to be "approved". No, the review request will be rejected and an illegal operation log will be recorded; The audit configuration unit is used to filter status audit rules and extract audit verification items based on the type of intelligent agent and the credit rating of the service provider. The stage verification unit is used to verify each audit verification item according to the audit type identifier and mark the pre-verification result; If it is a release review, then the format completeness, content compliance and validity period of the current review verification items will be verified. If all verifications pass, update the current agent status to pending listing and generate a listing review instruction. If it is for listing review, then based on historical operation data and the current review verification items, the functional availability, service response time and user experience score of the intelligent agent will be analyzed according to the listing review instructions; If all verifications pass, the current agent status will be updated to "listed" and services will be available to the public. The work order generation unit is used to associate and aggregate the pre-verification results, audit type identifiers, and items to be audited based on the agent identifier to generate an agent audit work order.

[0035] In this embodiment, the release review stage verifies the basic qualifications, material completeness (avatar, security commitment letter, intellectual property certificate), and information compliance of the intelligent agent. After passing the review, the status changes to APPROVED, but the agent is not immediately listed. The listing review stage verifies the functional usability, service quality, and user experience of the intelligent agent. After passing the review, the status changes to LAUNCHED, and the agent begins providing services.

[0036] The two reviews utilize independent review work orders, with each review generating a separate review record containing a review type identifier (INITIAL_REVIEW or LAUNCH_REVIEW). The system supports flexible configuration of the review process, allowing adjustments to review strategies based on agent type and service provider credit rating. This dual review mechanism ensures that only fully verified agents can provide services, reducing platform risk.

[0037] In this embodiment, when a service provider submits a request for intelligent agent release review, the instruction triggering unit deconstructs the review type identifier as "Release Review" based on the business scenario and reads the service provider's credit rating as "High". After the status verification unit verifies that the current intelligent agent status is indeed "Pending Submission", the review configuration unit filters simplified review verification items (only including basic information completeness and agreement signing validity) based on the intelligent agent type "Data Analysis" and the high credit rating. After the stage verification unit verifies the format completeness, content compliance, and validity period one by one and all pass, the intelligent agent status is updated to "Pending Listing" and a listing review instruction is generated; then the listing review stage begins, and the instruction triggering unit again... The review type identifier is identified as "Listing Review". The status verification unit confirms that the agent is in the "Passed" state. The review configuration unit selects three core indicators based on the credit level and agent type: functional availability, service response time, and user experience score. The stage verification unit automatically analyzes the agent's running data in the sandbox environment for 48 consecutive hours to determine if the functional availability reaches 100%, the average response time is less than 200ms, and the user experience score is 92 points. After all verifications are passed, the agent status is updated to "Listed" and the agent is opened to external services. The work order generation unit associates and aggregates the pre-verification results, review type identifier, and items to be reviewed to form a complete agent review work order for archiving and future reference.

[0038] Preferably, the parsing and delisting module includes: The target filtering unit is used to filter agents based on their status tags and extract agents that have been listed, their review dimension scores, and their historical risk coefficients. The data acquisition unit is used to detect the operation process of the deployed intelligent agents according to the intelligent agent review work order and collect multi-source dynamic operation data; The single-dimensional assessment unit is used to score multi-source dynamic operating data based on the review dimension scoring combined with the multi-dimensional risk assessment mechanism, and calculate the single-dimensional risk score. The comprehensive judgment unit is used to calculate the comprehensive risk score by weighting and correcting the single-dimensional risk score based on the agent type and historical risk coefficient. The risk level matching unit is used to match the comprehensive risk score according to the preset risk level threshold to determine the risk level of the agent. The delisting judgment unit is used to determine the delisting type based on the risk level of the intelligent agent and the delisting request response; If the risk level of the intelligent agent is low or medium and there is a delisting request response, then the delisting type of the current intelligent agent is determined to be active delisting; If the risk level of the agent is high, then the current agent's removal type is determined to be forced removal.

[0039] In this embodiment, when the service provider actively removes the product, `forcedOffline` is set to `false`, and the agent can resubmit for review at any time. When the platform performs a forced removal, `forcedOffline` is set to `true`, and the agent must undergo platform review before it can be listed again.

[0040] The forced removal data structure includes: agentId (agent ID), launchStatus (launch status, 0 indicates removal), forcedOffline (forced removal identifier), offlineReason (removal reason, structured text), operatorType (operator type, 0 indicates platform, 1 indicates service provider), offlineTime (removal timestamp), and riskLevel (risk level, HIGH / MEDIUM / LOW). The risk level is automatically calculated using a risk assessment algorithm, comprehensively considering multiple dimensions such as the number of security vulnerabilities, the number of user complaints, service quality scores, and compliance violation records.

[0041] In this embodiment, after a data analysis AI agent has been running for three months, its service provider proactively submitted a delisting application due to business adjustments. The target screening unit of the system's delisting module first filters out the AI ​​agent based on its status tag, extracts its historical review dimension scores (security 88 points, functional integrity 92 points, document integrity 85 points, compliance 90 points) and historical risk coefficient of "low". The data collection unit then collects dynamic operation data from the monitoring system for nearly 30 days (including an average daily call volume of 12,000 times, an average response time of 180ms, 0 user complaints, and no vulnerabilities found in security scans). The single-dimensional evaluation unit scores the data of each dimension according to the multi-dimensional risk assessment mechanism, obtaining security risk of 9 points, functional risk of 8 points, and compliance risk of 7 points.

[0042] The comprehensive judgment unit calculates a comprehensive risk score of 28 points (out of 100) after weighting and correcting the intelligent agent type "data analysis type" and historical risk coefficient. The level matching unit matches this score with a preset threshold and determines it as a "low" risk level. After receiving the service provider's delisting request response, the delisting judgment unit determines that the current intelligent agent's delisting type is voluntary delisting and sets the forcedOffline flag to false, allowing the service provider to resubmit for listing review at any time.

[0043] Preferably, refer to Figure 3 The multidimensional risk assessment mechanism includes: The multi-source dynamic operating data is quantitatively evaluated based on data dimensions, and a single-dimensional score is calculated. If the data is security data, the agent is scored based on the vulnerability level and the level decay coefficient, and the security dimension score is calculated. If the data is functional, the agent is evaluated based on the duration of service anomalies and the time-period decay coefficient, and a functional dimension score is calculated. If the data is compliant, the compliance assessment of the agent is performed based on the number of service evaluations combined with the abnormal attenuation deduction coefficient, and the compliance dimension score is calculated. The single-dimensional scores are weighted and normalized according to the type of agent to obtain the final comprehensive risk score. If the final comprehensive risk score is within the first risk threshold range, or if the score of any dimension is within the first scoring threshold range, then the current agent's risk level is determined to be high. If the final overall risk score is within the second risk threshold range, but at least two dimension scores are within the second scoring threshold range, then the current agent's risk level is determined to be medium. If the final comprehensive risk score is within the third risk threshold range, and the scores of each dimension are also within the third scoring threshold range, then the current agent's risk level is determined to be low.

[0044] In this embodiment, four core audit dimensions are defined: Security, Functionality, Documentation, and Compliance. Each dimension has a score range of 0-100, with 60 being the passing score.

[0045] Security Dimension Assessment: Algorithm testing includes data encryption mechanism integrity, user privacy protection measures, API security authentication mechanism, abnormal access monitoring capabilities, and security vulnerability scan results. Functional Integrity Dimension Assessment includes core function usability testing, boundary condition handling, error handling mechanisms, and performance metrics (response time, concurrency capabilities). Documentation Integrity Dimension Assessment includes functional documentation, API interface documentation, user tutorials, application cases, and frequently asked questions. Compliance Dimension Assessment includes service agreement integrity, intellectual property rights certification, data usage authorization, and compliance with industry standards.

[0046] The system automatically calculates scores for each dimension and generates radar charts for visualization. Reviewers can adjust scores and add comments. The scoring results are stored in a structured manner, supporting historical comparative analysis and trend prediction.

[0047] In this embodiment, during the operation of a financial risk control intelligent agent, the system monitored three high-risk security vulnerability alerts over seven consecutive days, a cumulative service anomaly duration of 12 hours, and a surge in user complaints to 15. A multi-dimensional risk assessment mechanism evaluated the data based on predefined threshold ranges (the first risk threshold range is set as a comprehensive risk score ≥ 80 or any dimension score < 60; the second risk threshold range is set as a comprehensive risk score of 60-79 with at least two dimension scores between 60-79; the third risk threshold range is set as a comprehensive risk score ≤ 59 with all dimension scores ≥ 80). Security data was categorized by vulnerability level. The combined risk score is 28 points (far below the first risk threshold of 60 points) calculated by the attenuation coefficient of the service level. The functional data score is 35 points calculated by the attenuation coefficient of the service anomaly duration and time period. The compliance data score is 40 points calculated by the number of service evaluations and the attenuation deduction coefficient of the anomaly. After weighted calculation by normalization of the agent type weight, the comprehensive risk score is 89 points, which falls into the first risk threshold range. The comprehensive risk level is judged to be high. The delisting judgment unit automatically executes the forced delisting operation, sets the forcedOffline flag to true and records the high risk reason as "frequent security vulnerabilities and serious service quality deficiencies". The agent is restricted to be reviewed by the platform again before it can be restored to the listing.

[0048] Preferably, the traceability management module includes: The intelligent capture unit is used to capture the entire process of the intelligent agent's review operation and extract state transition events; The record creation unit is used to associate state transition events with intelligent agent review work orders based on state transition type, and generate independent review records; The path building unit is used to sort independent audit records according to time series and build lifecycle paths; The change marking unit is used to transform and mark the lifecycle path to obtain the state change node; The state management unit is used to fuse and map the agent's state, lifecycle path, and state change nodes to generate a state transition mapping diagram. The notification tracing unit is used to associate and push state transition events with state transition maps according to event priority, and generate change event notifications.

[0049] In this embodiment, an independent audit record is created for each audit operation, which includes: audit record ID, agent ID, audit work order ID, audit status, auditer's name and account, audit timestamp, rejection reason (structured text), audit comments, audit dimension scoring details, and audit attachment list.

[0050] The review records are stored in time series and support multi-dimensional queries by agent ID, reviewer, time range, review result, etc. The system provides a visual display of the review history timeline, clearly showing the complete review path of the agent from creation to the current status. It supports the generation of review reports, including analytical data such as review frequency statistics, average review time, scoring trends of various dimensions, and classification statistics of rejection reasons.

[0051] The system achieves real-time data synchronization across multiple devices and users through a unified status management module. It manages intelligent agent monitoring data in a unified manner through objects such as real-time table mapping data, real-time chart mapping data, and host status data, supporting data isolation and parallel monitoring.

[0052] A notification service is automatically triggered when the status changes, using a WebSocket long-lived connection to achieve millisecond-level message push. The notification content includes: status change type, status before and after the change, operator information, reason for the change, and suggested next steps. The system supports multiple notification channels: in-app messages, email notifications, and SMS notifications (high-priority events). Notification templates are configurable and support customization.

[0053] In this embodiment, from the moment an intelligent agent submits for publication review, the intelligent capture unit of the traceability management module captures each state transition event in real time (such as from "pending submission" to "pending review", from "not approved" after publication review rejection, from "pending review" after correction and resubmission, from "approved" after publication review approval, from "approved" after publication review approval, from "pending review for publication" after submission for listing review, and from "listed" after listing review approval). The record creation unit associates each event with the corresponding review work order to generate an independent review record, and the path construction unit sorts them according to the time sequence to form a complete lifecycle path, and changes are marked. The unit marks key nodes on the path (such as initial review rejection, listing approval), and the status management unit integrates the status and path to generate a visual status transition mapping. At the same time, the notification traceability unit, based on the preset event priority threshold range (setting abnormal events such as forced delisting and review rejection as "high priority" to trigger dual notifications via SMS and in-app notification, and ordinary status changes to "low priority" to trigger only in-app notification), automatically pushes the high-priority event association mapping to the platform administrator and service provider when the intelligent agent triggers forced delisting due to risk scoring after listing, thus achieving full-process traceability and accurate notification.

[0054] Preferably, the dynamic access control matrix includes a creator access matrix and an administrator access matrix; wherein, The creator permission matrix is ​​as follows: If the agent status is pending submission, the edit, delete, and submit for review operations are set to 1; if the agent status is pending review, the withdraw operation is set to 1; if the agent status is not approved, the edit, delete, and resubmit operations are set to 1; if the agent status is approved, the edit, delete, and submit for listing review operations are set to 1; if the agent status is listed, the active delisting operation is set to 1. The administrator permission matrix is ​​as follows: If the agent status is "removed from the market", the "submit for review" operation is set to 1; If the agent status is "pending review", the "approved", "rejected", and "view details" operations are set to 1; If the agent status is "approved", "rejected", or "removed from the market", the "view details" operation is set to 1; If the agent status is "listed", the "view details" and "force removal" operations are set to 1.

[0055] In this embodiment, the executable operation permissions are dynamically calculated based on the current state of the intelligent agent and the operator type (service provider or platform administrator).

[0056] Service provider permission matrix definition: In the TO_BE_SUBMITTED state, the service provider can perform edit, delete, and submit for review operations; in the PENDING state, the service provider can perform withdraw operations; in the REJECTED state, the service provider can perform edit, delete, and resubmit operations; in the APPROVED state, the service provider can perform edit, delete, and submit for listing review operations; in the LAUNCHED state, the service provider can perform proactive delisting operations; in the OFFLINE state, the service provider can perform submit for listing review operations.

[0057] Platform administrator permission matrix definition: In the PENDING state, the administrator can perform operations such as approval, rejection, and viewing details; in the APPROVED, REJECTED, and OFFLINE states, the administrator can perform the operation of viewing details; in the LAUNCHED state, the administrator can perform the operation of viewing details and forcibly removing the product.

[0058] Before each operation, the system checks the legality of the operation through the permission verification module. If the permission verification fails, it returns a clear error message, and all permission check results are recorded in the operation log.

[0059] Reference Figure 4 This application discloses a multi-level review and listing management method for intelligent agents, comprising: S1: Detect the agent according to the preset node state machine model, verify the state transition path of the agent, and determine the state of the agent; S2: Based on the status review rules and the agent's status, verify the agent's stage status elements and construct the agent review work order; S3: Based on the multi-dimensional risk assessment mechanism and the intelligent agent review work order, quantify the intelligent agents that have been put on the shelves, calculate the comprehensive risk score, and determine the type of delisting; S4: Monitor the entire process of the intelligent agent's audit operation, create lifecycle audit paths, mark status change nodes, and trigger change event notifications; S5: Based on the agent's state and operator type, verify and allocate the agent's execution operation permissions according to the dynamic permission control matrix.

[0060] In this embodiment: After a high-credit-rating intelligent agent service provider submits a release review application, the system verifies the legality of the transition path from the source state "pending submission" to the target state "pending review" based on a preset seven-state state machine model, and updates the intelligent agent state to "pending review". After verifying the intelligent agent state as "pending submission" based on the review type identifier "release review", the system filters and simplifies the review verification items based on the service provider's "high" credit rating, completes the format integrity and content compliance checks, generates a release review work order, and updates the status to "passed". The service provider then submits for listing review, and after verifying the status as "passed" again, the system analyzes the functional availability, service response time, and user experience score one by one according to the listing review instructions, and updates the status to "listed" after passing the review.

[0061] During the launch and operation period, two medium-risk security vulnerability alerts were detected within five consecutive days, and there were eight user complaints. The multi-dimensional risk assessment mechanism calculated the following based on predefined threshold ranges (a comprehensive risk score ≥75 is considered high risk, 60-74 is considered medium risk, and ≤59 is considered low risk; security dimension score <65 is considered high risk threshold, functional dimension score <70 is considered medium risk threshold, and compliance dimension score <60 is considered low risk threshold): security score 58 (below the high risk threshold of 65), functional score 72, and compliance score 55. After weighting by normalizing the intelligent agent type weights, the comprehensive risk score was 82, falling into the high risk range. The forced removal type was determined, and forcedOffline was set to true. The state transition event was captured, an independent review record was created, and a lifecycle path including nodes such as release review, launch review, and forced removal was constructed according to the time sequence, generating a state transition mapping diagram.

[0062] When verifying the user's submission approval permissions based on the current status "removed" and the operator type "service provider", if the forced removal flag is detected as true, the operation is deemed illegal according to the dynamic permission control matrix, the request is rejected and logged. At the same time, the forced removal event is pushed to the platform administrator and service provider according to the predefined high priority threshold range (set to trigger dual notifications via SMS and in-app notification when forced removal, review rejection, and risk level upgrade to high priority), thus achieving closed-loop management of the entire process.

[0063] This application discloses a storage medium storing at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the multi-level review and listing management method for intelligent agents as described above.

[0064] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A multi-level review and listing management system for intelligent agents, characterized in that, include: The state transition module is used to detect agents based on a preset node state machine model, verify the state transition path of the agents, and determine the state of the agents. The dual-review module is used to verify the stage state elements of the agent based on the status review rules and the agent's status, and to construct an agent review work order. The delisting module is used to quantify the listed intelligent agents based on the multi-dimensional risk assessment mechanism and the intelligent agent's review work order, calculate the comprehensive risk score, and determine the delisting type. The traceability management module is used to monitor the entire process of the intelligent agent's review operation, create lifecycle review paths, mark status change nodes, and trigger change event notifications.

2. The multi-level review and listing management system for intelligent agents according to claim 1, characterized in that, The state transition module includes: The model parsing unit is used to parse the preset node state machine model and extract the semantic labels of state nodes and the skeleton of state transition rules. The state matching unit is used to perform semantic matching on the agent based on the semantic labels of the state nodes to obtain the source state of the agent; The state evolution unit is used to transform and evolve the source state of the intelligent agent according to the state transition rule skeleton and the target state to obtain the state evolution matrix. The path verification unit is used to verify the state transition path of the agent based on the state transition request and the state evolution matrix. If the verification passes, the current state evolution matrix is ​​determined to be compliant, and the agent source state is converted into the agent target state. If not, it is determined that the current state evolution matrix violates the rules, abnormal transition nodes are extracted, the state transition rule skeleton is optimized, and the current agent state is maintained.

3. The multi-level review and listing management system for intelligent agents according to claim 1, characterized in that, The dual verification module includes: The instruction triggering unit is used to deconstruct the state of the intelligent agent according to the business scenario and extract the audit type identifier and the service provider credit rating; The status verification unit is used to verify the status of the intelligent agent according to the audit type identifier; If the review type is identified as "Release Review", then the current agent status is determined to be "Pending Submission". If the review type is identified as "listing review", then the current agent status is determined to be "approved". No, the review request will be rejected and an illegal operation log will be recorded; The audit configuration unit is used to filter the status audit rules and extract audit verification items based on the intelligent agent type and the service provider's credit rating. The stage verification unit is used to verify each of the audit verification items according to the audit type identifier and mark the pre-verification result; If it is a release review, then the format completeness, content compliance and validity period of the current review verification items will be verified. If all verifications pass, update the current agent status to pending listing and generate a listing review instruction. If it is for listing review, then based on historical operation data and the current review verification items, the functional availability, service response time and user experience score of the intelligent agent are analyzed according to the listing review instructions; If all verifications pass, the current agent status will be updated to "listed" and services will be provided externally. The work order generation unit is used to associate and aggregate the pre-verification result, the audit type identifier, and the item to be audited based on the agent identifier to generate an agent audit work order.

4. The multi-level review and listing management system for intelligent agents according to claim 1, characterized in that, The parsing and delisting module includes: The target filtering unit is used to filter agents based on their status tags and extract agents that have been listed, their review dimension scores, and their historical risk coefficients. The data acquisition unit is used to detect the operation process of the deployed intelligent agent according to the intelligent agent review work order and collect multi-source dynamic operation data; A single-dimensional assessment unit is used to score the multi-source dynamic operating data based on the review dimension scoring combined with the multi-dimensional risk assessment mechanism, and to calculate a single-dimensional risk score. The comprehensive judgment unit is used to calculate the comprehensive risk score by weighting and correcting the single-dimensional risk score based on the agent type and the historical risk coefficient. A risk level matching unit is used to match the comprehensive risk score according to a preset risk level threshold to determine the risk level of the agent. The delisting determination unit is used to determine the delisting type based on the risk level of the intelligent agent and the delisting request response; If the risk level of the intelligent agent is low or medium and there is a delisting request response, then the delisting type of the current intelligent agent is determined to be active delisting. If the risk level of the intelligent agent is high, then the current removal type of the intelligent agent is determined to be forced removal.

5. The multi-level review and listing management system for intelligent agents according to claim 1 or 4, characterized in that, The multidimensional risk assessment mechanism includes: Quantitatively evaluate multi-source dynamic operating data based on data dimensions and calculate single-dimensional scores; If the data is security data, the agent is given a security score based on the vulnerability level and the level decay coefficient, and the security dimension score is calculated. If the data is functional, the agent is evaluated based on the duration of service anomalies and the time-period decay coefficient, and a functional dimension score is calculated. If the data is compliant, the compliance assessment of the agent is performed based on the number of service evaluations combined with the abnormal attenuation deduction coefficient, and the compliance dimension score is calculated. The single-dimensional scores are weighted and normalized according to the type of agent to obtain the final comprehensive risk score. If the final comprehensive risk score is within the first risk threshold range, or if any dimension score is within the first scoring threshold range, then the current agent's risk level is determined to be high. If the final comprehensive risk score is within the second risk threshold range, but at least two dimension scores are within the second scoring threshold range, then the current agent's risk level is determined to be medium. If the final comprehensive risk score is within the third risk threshold range, and the scores of each dimension are also within the third scoring threshold range, then the current agent's risk level is determined to be low.

6. The multi-level review and listing management system for intelligent agents according to claim 1, characterized in that, The traceability management module includes: The intelligent capture unit is used to capture the entire process of the intelligent agent's review operation and extract state transition events; The record creation unit is used to associate the state transition event with the intelligent agent's review work order according to the state transition type, and generate an independent review record; A path construction unit is used to sort the independent audit records according to the time series and construct a lifecycle path; A change marking unit is used to convert and mark the lifecycle path to obtain a state change node; The state management unit is used to fuse and map the agent's state, the lifecycle path, and the state change nodes to generate a state transition mapping diagram. The notification tracing unit is used to associate and push the state transition event with the state transition mapping according to the event priority, and generate a change event notification.

7. The multi-level review and listing management system for intelligent agents according to claim 1, characterized in that, The management system also includes: The permission verification module is used to verify and allocate the execution operation permissions of the agent according to the agent's state and operator type, based on the dynamic permission control matrix.

8. The multi-level review and listing management system for intelligent agents according to claim 7, characterized in that, The dynamic permission control matrix includes a creator permission matrix and a manager permission matrix; wherein... The creator permission matrix is ​​as follows: if the agent status is pending submission, the edit, delete, and submit for review operations are set to 1; if the agent status is pending review, the withdraw operation is set to 1; if the agent status is not approved, the edit, delete, and resubmit operations are set to 1; if the agent status is approved, the edit, delete, and submit for listing review operations are set to 1; if the agent status is listed, the active delisting operation is set to 1. The administrator permission matrix is ​​as follows: if the intelligent agent's status is "removed from the market", then the "submit for review" operation is set to 1; if the intelligent agent's status is "pending review", then the "approved", "rejected", and "view details" operations are set to 1; if the intelligent agent's status is "approved", "rejected", or "removed from the market", then the "view details" operation is set to 1; if the intelligent agent's status is "listed", then the "view details" and "force removal" operations are set to 1.

9. A multi-level review and listing management method for intelligent agents, applied to the system described in any one of claims 1-8, characterized in that, include: The agent is detected based on the preset node state machine model, and the state transition path of the agent is verified to determine the state of the agent. Based on the status review rules and the agent's status, verify the agent's stage status elements and construct an agent review work order. Based on the multi-dimensional risk assessment mechanism and the intelligent agent review work order quantification of the intelligent agents that have been put on the shelf, a comprehensive risk score is calculated to determine the type of delisting. Monitor the entire audit process of the intelligent agent, create lifecycle audit paths, mark status change nodes, and trigger change event notifications; Based on the agent's state and operator type, the execution operation permissions of the agent are verified and allocated according to the dynamic permission control matrix.

10. A storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the multi-level review and listing management method for intelligent agents as described in claim 9.

Images