A Method and System for Institutional Qualification Verification Based on Knowledge Graphs and Multimodal Collaboration
By constructing a two-layer graph structure and multimodal feature mapping, the problem of reliance on human experience in the qualification review of acne care institutions is solved, realizing rapid and accurate qualification compliance verification and adaptive expansion capabilities, thereby improving review efficiency and transparency.
Patent Information
- Application Number
- CN202611293332.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-25
- Publication Date
- 2026-11-10
AI Technical Summary
The current qualification review of acne care institutions relies on human experience, which makes it difficult to quickly and accurately handle the compliance judgment of complex qualification materials, and lacks cross-modal correlation capabilities, resulting in insufficient review efficiency and accuracy.
A knowledge graph-based and multimodal collaborative approach is adopted to construct a two-layer graph structure. By establishing the association between static rule layers and dynamic instance layers through cross-layer anchor points, a unified semantic mapping of certificate documents and environmental image features and cross-modal semantic vector generation are achieved, and compliance verification is automatically performed.
It achieves semantic verification between certificate text and environmental details, reduces the amount of manual review, expands the scope of acceptable materials, improves review efficiency and accuracy, and provides a structured chain of evidence to support regulatory transparency and adaptive expansion capabilities.
Smart Images

Figure CN123070972A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of qualification verification technology for acne care institutions, and in particular to a method and system for qualification verification based on knowledge graphs and multimodal collaboration. Background Technology
[0002] In current practices for verifying the qualifications of acne care clinics, auditors often rely on manually reviewing certificates and documents along with images of the actual operating environment. The audit process typically involves a step-by-step comparison of written materials such as business licenses, health permits, and staff health certificates, combined with manual interpretation of on-site photos or videos to confirm compliance with basic hygiene and service standards. This manual auditing model demands a high level of industry experience from auditors and has a lengthy audit cycle, making it ill-suited to the current industry landscape of rapidly growing and frequently changing acne care clinics.
[0003] With the widespread adoption of information systems in government and industry supervision, some review processes have begun to use electronic forms and scanned document uploads to assist manual judgment. The review system can perform basic keyword searches and validity period reminders on certificate documents, while environmental images are mainly retained as attachments for future reference and are not included in the automated review process. This semi-automated approach reduces the cost of paper document circulation to some extent, but it still requires reviewers to manually review and cross-reference information from different sources. The objectivity and timeliness of the review conclusions are limited by individual subjective judgment and fatigue. When there are discrepancies between the textual information in the institution's qualification materials and the actual situation on-site, when the authenticity of certificates is questionable, or when the content of images is difficult to directly correlate, the conventional review path cannot effectively highlight potential compliance risks, easily leading to misjudgments or omissions.
[0004] Furthermore, existing review methods utilize material information in a fragmented manner. The structured descriptions in certificate documents and the unstructured visual information in on-site images lack a unified semantic association mechanism, and the corroboration relationships between different modalities can only be inferred through human experience. This conventional approach, which relies on human experience and lacks cross-modal correlation capabilities, presents significant bottlenecks in both review efficiency and accuracy when dealing with the compliance judgment of complex qualification materials. Summary of the Invention
[0005] This invention provides a method and system for auditing institutional qualifications based on knowledge graphs and multimodal collaboration, which can solve the problems in the prior art.
[0006] A first aspect of this invention provides a method for institutional qualification verification based on knowledge graphs and multimodal collaboration, comprising:
[0007] We receive qualification materials uploaded by acne care institutions awaiting review, including certificate documents and environmental images;
[0008] A two-layer map structure for acne-prone skin care qualifications is constructed, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to the regulatory requirements, and the dynamic instance layer stores the qualification instances submitted by the institution. The relationship between the static rule layer and the dynamic instance layer is established through cross-layer anchor points.
[0009] Multimodal feature extraction is performed on the qualification materials, and the text features of the certificate document and the visual features of the environmental image are mapped to a unified semantic space to generate cross-modal semantic vectors.
[0010] The cross-modal semantic vector is constructed as an instance node and inserted into the dynamic instance layer. The semantic distance between the instance node and the rule node in the static rule layer is calculated, and rule nodes with a semantic distance less than the semantic threshold are selected as the applicable rule set.
[0011] For the applicable rule set, cross-layer anchor points are established between the instance node and each rule node. The corresponding audit logic is triggered through the cross-layer anchor points to perform compliance verification and generate an audit report. The audit report and its verification path in the two-layer graph structure are then output.
[0012] A two-layer map structure for acne-prone skin care qualifications is constructed, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to regulatory requirements, and the dynamic instance layer stores qualification instances submitted by institutions. The association between the static rule layer and the dynamic instance layer is established through cross-layer anchor points, including:
[0013] Compliance elements are extracted from regulatory documents and an ontology model is constructed. Based on the ontology model, a hierarchical representation is organized in the static rule layer. The hierarchical representation includes upper-level concept nodes and lower-level detailed rule nodes. Inheritance constraints are established for the upper-level concept nodes and the lower-level detailed rule nodes. The inheritance constraints stipulate that the lower-level detailed rule nodes satisfy the preconditions of the upper-level concept nodes.
[0014] The qualification instances are structured and organized into an instance diagram, and the supporting material nodes of the instance diagram are represented in the dynamic instance layer;
[0015] Semantic encoding is performed on the evidence material nodes, and the semantic distance between the encoding results and the nodes in the static rule layer is calculated. The node combination with the smallest semantic distance is selected as the candidate anchor set. Constraint compatibility checks are performed on the candidate anchor set to verify whether the evidence material nodes meet the inheritance constraints. The candidate anchors that pass the verification are established as cross-layer anchors, and the static rule layer and the dynamic instance layer are connected into a unified two-layer graph structure.
[0016] Hierarchical path information is attached to the cross-layer anchor points. When the dynamic instance layer changes, the path information is used to trace back to the static rule layer and trigger the corresponding level of review rules.
[0017] Perform constraint compatibility checks on the candidate anchor point set to verify whether the proof material nodes conform to inheritance constraints. Establish the verified candidate anchor points as cross-layer anchor points. Connect the static rule layer and the dynamic instance layer into a unified two-layer atlas structure, including:
[0018] Extract the corresponding static rule layer nodes from the candidate anchor point set, traverse upwards along the static rule layer nodes to the upper-level concept node in the hierarchical representation, and aggregate the inherited constraints on the traversal path in hierarchical order to form a constraint propagation chain.
[0019] The preconditions are extracted from the constraint propagation chain and arranged in hierarchical order to form a verification sequence;
[0020] The semantic encoding result of the proof material node is matched and verified with the preconditions in the verification sequence in turn. When the semantic encoding result satisfies all the preconditions in the verification sequence, the corresponding candidate anchor point is marked as compliant. When the semantic encoding result does not satisfy any precondition, the verification is terminated and the corresponding candidate anchor point is marked as non-compliant.
[0021] Candidate anchor points marked as compliant are selected and established as the cross-layer anchor points;
[0022] In the dual-layer graph structure, a connection relationship is established from the proof material node to the cross-layer anchor point. The constraint propagation chain is attached to the connection relationship as a verification basis, thus completing the connection between the static rule layer and the dynamic instance layer.
[0023] Multimodal feature extraction is performed on the qualification materials, mapping the textual features of the certificate document and the visual features of the environmental image to a unified semantic space to generate cross-modal semantic vectors, including:
[0024] Perform layout parsing on the certificate document, extract the qualification type identifier from the title area, extract key fields from the content area, and semantically group them according to the qualification type identifier;
[0025] Construct a field dependency tree for the key fields in each semantic group according to their hierarchical relationship, with the qualification type identifier as the root node and the key fields as child nodes;
[0026] The semantic information of the field dependency tree is aggregated layer by layer from the leaf node to the root node. Each node integrates the information passed from the lower layer with the semantics of the current field and then passes it upward. The root node outputs the text feature vector.
[0027] The environmental image is used to perform semantic segmentation to identify facility areas, detect facility objects and extract boundary coordinates, calculate the spatial proximity between facility objects based on the boundary coordinates, and combine facility objects with a proximity greater than the proximity threshold into functional units.
[0028] Local visual features are extracted from facility objects in functional units, and global visual features are extracted from the area covered by functional units. These features are then weighted and fused according to spatial proportion to generate a visual feature vector.
[0029] The text feature vector and the visual feature vector are projected into a shared semantic space, the projection parameters are adjusted to minimize the distance between them, and the resulting fusion generates a cross-modal semantic vector.
[0030] The cross-modal semantic vectors are constructed as instance nodes and inserted into the dynamic instance layer. The semantic distance between the instance nodes and the rule nodes in the static rule layer is calculated. Rule nodes with a semantic distance less than a semantic threshold are selected as the applicable rule set, including:
[0031] The cross-modal semantic vector is converted into a graph node representation and node attribute fields are configured. Feature parameters are extracted from the vector dimension of the cross-modal semantic vector and mapped to the node attribute fields to generate instance nodes carrying structured attributes.
[0032] A node index structure is constructed in the dynamic instance layer. An index position is assigned to the instance node in the node index structure according to the node attribute field. The neighboring nodes in the dynamic instance layer are retrieved through the index position. A bidirectional link is established between the instance node and each of the neighboring nodes. The instance node carrying the bidirectional link is inserted into the dynamic instance layer.
[0033] Obtain the semantic vectors of the rule nodes in the static rule layer, measure the distance between the cross-modal semantic vectors and each of the semantic vectors to obtain the initial semantic distance, and obtain the historical distance data of the neighboring nodes based on the bidirectional links and perform correction to obtain the semantic distance;
[0034] Filter rule nodes whose semantic distance is less than the semantic threshold, and group and integrate the filtered rule nodes according to the node index structure to form the applicable rule set.
[0035] For the applicable rule set, cross-layer anchor points are established between the instance nodes and each rule node. These anchor points trigger corresponding audit logic to perform compliance verification and generate an audit report. The audit report and its verification path within the two-layer graph structure are then output, including:
[0036] Traverse each rule node in the applicable rule set and construct a cross-layer anchor point between the instance node and the corresponding rule node. Configure a bidirectional transmission channel for the cross-layer anchor point and transmit the semantic features of the instance node to the rule node through the bidirectional transmission channel to obtain the encapsulated audit logic.
[0037] Based on the bidirectional transmission channel, a data mapping relationship is established between the audit logic and the instance node. The audit logic parses the semantic features to extract compliance verification elements and performs rule matching to generate verification tags. The verification tags are then sent back to the instance node through the bidirectional transmission channel.
[0038] At the instance node, the returned verification tags are aggregated to construct a verification result set, and the verification result set is integrated with the connection information of the cross-layer anchor point to form an audit report.
[0039] Tracing back from the instance node along the bidirectional transmission channel to the node index position of the dynamic instance layer, and tracing to the storage position of the rule node in the static rule layer through cross-layer anchor points, the connection sequence of the node index position, the storage position, and the bidirectional transmission channel is concatenated to generate a verification path and output with the audit report.
[0040] A second aspect of this invention provides an institutional qualification verification system based on knowledge graphs and multimodal collaboration, comprising:
[0041] The material receiving unit is used to receive qualification materials uploaded by acne care institutions to be reviewed, including certificate documents and environmental images;
[0042] The graph construction unit is used to construct a two-layer graph structure for acne skin care qualifications, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to the regulatory requirements, and the dynamic instance layer stores the qualification instances submitted by the institution. The association between the static rule layer and the dynamic instance layer is established through cross-layer anchor points.
[0043] The feature extraction unit is used to extract multimodal features from the qualification materials, map the text features of the certificate document and the visual features of the environmental image to a unified semantic space, and generate cross-modal semantic vectors.
[0044] The rule matching unit is used to construct the cross-modal semantic vector into instance nodes and insert them into the dynamic instance layer, calculate the semantic distance between the instance nodes and the rule nodes in the static rule layer, and filter the rule nodes with semantic distance less than the semantic threshold as the applicable rule set.
[0045] The audit generation unit is used to establish cross-layer anchor points between the instance node and each rule node for the applicable rule set, trigger the corresponding audit logic through the cross-layer anchor points to perform compliance verification and generate an audit report, and output the audit report and its verification path in the two-layer graph structure.
[0046] A third aspect of the present invention provides an electronic device, comprising:
[0047] processor;
[0048] Memory used to store processor-executable instructions;
[0049] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.
[0050] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.
[0051] The multimodal feature extraction of this invention maps certificate text and environmental images into a unified semantic space, enabling the review system to simultaneously understand structured documents and on-site visual information, eliminating misjudgments caused by the lack of a single modality. Certificate text and environmental details corroborate each other in the semantic space, and even if materials are blurry or missing, the true qualification status can still be restored through cross-modal association, significantly reducing the amount of manual review.
[0052] The two-layer graph structure organizes static regulatory logic and dynamic instance data in layers, establishing stable connections between cross-layer anchor points so that new review tasks do not require rule reconstruction. After instance nodes are inserted into the dynamic instance layer, they are automatically matched with applicable rule sets based on semantic distance, avoiding the mechanical nature of keyword matching. Based on business semantics, equivalent qualification certificates or similar compliance conditions are identified, expanding the scope of acceptable materials and maintaining consistent review standards.
[0053] When cross-layer anchor points trigger the audit logic to perform compliance verification, each verification step leaves a trace along the graph path, and the output verification path fully presents the conclusion basis. Auditors can backtrack to the corresponding rule nodes and instance nodes at any time to quickly locate the qualification deficiencies, providing a structured chain of evidence for appeals and reviews, and significantly improving regulatory transparency.
[0054] The dynamic instance layer continuously accumulates historical audit data, enabling cross-modal semantic vectors and semantic thresholds to be progressively optimized in actual audit feedback. It has adaptive expansion capabilities for new qualification templates and changed regulatory requirements. Local rule updates only require adjusting the static layer nodes and anchor point relationships without changing the overall framework, maintaining high-precision audit results and reducing maintenance costs in long-term operation. Attached Figure Description
[0055] Figure 1 This is a flowchart illustrating the institutional qualification review method based on knowledge graph and multimodal collaboration according to an embodiment of the present invention.
[0056] Figure 2 This is a flowchart illustrating the method for constructing a two-layer atlas structure for acne-prone skin care qualifications according to an embodiment of the present invention.
[0057] Figure 3 A schematic diagram comparing multimodal feature fusion and cross-modal semantic vector generation;
[0058] Figure 4 This diagram illustrates the semantic distance distribution between instance nodes and rule nodes, as well as threshold filtering. Detailed Implementation
[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0061] Figure 1 This is a flowchart illustrating the institutional qualification verification method based on knowledge graphs and multimodal collaboration according to an embodiment of the present invention. The present invention provides an institutional qualification verification method based on knowledge graphs and multimodal collaboration, comprising:
[0062] We receive qualification materials uploaded by acne care institutions awaiting review, including certificate documents and environmental images;
[0063] A two-layer map structure for acne-prone skin care qualifications is constructed, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to the regulatory requirements, and the dynamic instance layer stores the qualification instances submitted by the institution. The relationship between the static rule layer and the dynamic instance layer is established through cross-layer anchor points.
[0064] Multimodal feature extraction is performed on the qualification materials, and the text features of the certificate document and the visual features of the environmental image are mapped to a unified semantic space to generate cross-modal semantic vectors.
[0065] The cross-modal semantic vector is constructed as an instance node and inserted into the dynamic instance layer. The semantic distance between the instance node and the rule node in the static rule layer is calculated, and rule nodes with a semantic distance less than the semantic threshold are selected as the applicable rule set.
[0066] For the applicable rule set, cross-layer anchor points are established between the instance node and each rule node. The corresponding audit logic is triggered through the cross-layer anchor points to perform compliance verification and generate an audit report. The audit report and its verification path in the two-layer graph structure are then output.
[0067] Figure 2 This is a flowchart illustrating a method for constructing a two-layer atlas structure for acne-prone skin care qualifications according to an embodiment of the present invention. The two-layer atlas structure for acne-prone skin care qualifications includes a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to regulatory requirements, and the dynamic instance layer stores qualification instances submitted by the institution. The association between the static rule layer and the dynamic instance layer is established through cross-layer anchor points, including:
[0068] Compliance elements are extracted from regulatory documents and an ontology model is constructed. Based on the ontology model, a hierarchical representation is organized in the static rule layer. The hierarchical representation includes upper-level concept nodes and lower-level detailed rule nodes. Inheritance constraints are established for the upper-level concept nodes and the lower-level detailed rule nodes. The inheritance constraints stipulate that the lower-level detailed rule nodes satisfy the preconditions of the upper-level concept nodes.
[0069] The qualification instances are structured and organized into an instance diagram, and the supporting material nodes of the instance diagram are represented in the dynamic instance layer;
[0070] Semantic encoding is performed on the evidence material nodes, and the semantic distance between the encoding results and the nodes in the static rule layer is calculated. The node combination with the smallest semantic distance is selected as the candidate anchor set. Constraint compatibility checks are performed on the candidate anchor set to verify whether the evidence material nodes meet the inheritance constraints. The candidate anchors that pass the verification are established as cross-layer anchors, and the static rule layer and the dynamic instance layer are connected into a unified two-layer graph structure.
[0071] Hierarchical path information is attached to the cross-layer anchor points. When the dynamic instance layer changes, the path information is used to trace back to the static rule layer and trigger the corresponding level of review rules.
[0072] When constructing a two-layer graph structure for acne-prone skin care qualifications, the primary task is to systematically extract compliance elements from regulatory documents. These documents typically encompass various texts, including health administrative licensing regulations, skin care industry standards, practitioner qualification requirements, and workplace hygiene requirements. For these texts, a combination of named entity recognition and dependency parsing is used to extract core compliance concepts such as "professional qualification certificate," "disinfection facilities," and "operating procedures." Simultaneously, the logical relationships between these concepts are identified, including inclusion, premise, and parallel relationships. Based on this, an ontology model for the acne-prone skin care field is constructed, organizing the extracted compliance elements into a directed acyclic graph structure according to semantic hierarchy, ensuring that each node has a clear semantic definition and hierarchical affiliation within the ontology model.
[0073] Based on the constructed ontology model, a hierarchical representation is established in the static rule layer. This hierarchical representation consists of two levels: upper-level concept nodes carry macro-level compliance categories, such as "personnel qualification compliance," "venue environment compliance," and "equipment compliance"; lower-level detailed rule nodes correspond to specific audit details, such as "holding a valid beautician qualification certificate," "sterilizer temperature meets standards," and "operating table surface material meets hygiene standards." An inheritance constraint is established between upper-level concept nodes and lower-level detailed rule nodes. This constraint formally stipulates that if an instance node claims to meet the compliance category represented by a certain upper-level concept node, it must simultaneously meet the preconditions of all lower-level detailed rule nodes attached to that upper-level concept node. This mechanism ensures the completeness of the audit logic and prevents organizations from circumventing detailed rule-level checks by only meeting macro-level conditions.
[0074] When performing structured parsing on the qualification instances submitted by institutions, the original qualification materials (including certificate documents, business licenses, health permits, and related supporting documents) are parsed according to attribute fields such as material type, issuing authority, validity period, and certificate number, forming a structured instance graph. Each supporting document in the instance graph corresponds to a supporting document node, and the node attributes include fields such as material type label, issuing entity, validity period, and original text summary. These supporting document nodes are organized and stored in a dynamic instance layer, and directed edges are established between nodes based on the logical relationship between the materials (e.g., one certificate is a prerequisite for another certificate), forming a complete instance graph topology.
[0075] For each proof material node in the dynamic instance layer, semantic encoding is performed. Semantic encoding uses a pre-trained domain-adaptive language model to transform the text content of the proof material node into a fixed-dimensional semantic vector, denoted as proof material node. The semantic vector is In the static rule layer, the first The semantic vector of each node is The semantic distance between the two Calculated using cosine distance:
[0076]
[0077] For each supporting document node Traverse all rule nodes in the static rule layer, according to The nodes are sorted in ascending order, and a group of nodes with the smallest semantic distance is selected as the candidate anchor set. The size of the candidate anchor set is determined by a preset candidate number parameter. Control, meaning each supporting documentation node corresponds to at most Candidate rule nodes.
[0078] After the candidate anchor point set is determined, a constraint compatibility check is performed on each pair of candidate anchor points. The core logic of the constraint compatibility check is to verify whether the attribute information carried by the current proof material node satisfies the preconditions of all upper-level concept nodes on the hierarchical path of the candidate rule node, based on the inheritance constraints in the static rule layer. Specifically, starting from the candidate rule node, the system traverses upwards along the inheritance relationship to the root node, extracting preconditions layer by layer, and comparing these preconditions with the attribute fields of the proof material node item by item. If the attribute fields of the proof material node can cover the preconditions of all levels on the path, the candidate anchor point passes the constraint compatibility check; otherwise, the candidate anchor point is removed from the candidate set. The candidate anchor points that pass the check are finally established as formal cross-layer anchor points, thereby establishing a connection between the corresponding proof material node and the rule node in the cross-layer dimension, integrating the static rule layer and the dynamic instance layer into a unified two-layer graph structure.
[0079] After the cross-layer anchor point is established, hierarchical path information is attached to it. The path information records the complete hierarchical path from the current rule node to the root node of the static rule layer, including the node identifier, the level number, and the corresponding audit rule identifier for each node on the path. When the dynamic instance layer changes—for example, an organization updates a certificate document or adds a new equipment compliance material—the change trigger event propagates along the cross-layer anchor point to the static rule layer. Based on the attached path information, the system precisely traces back to the level of the affected rule node and triggers the re-execution of the audit rules corresponding to that level and all its lower-level detailed rule nodes. This mechanism avoids the computational redundancy of a full re-audit and achieves incremental, precise audit responses.
[0080] The addition of path information also supports the interpretability of audit conclusions. When an audit conclusion is generated, the path information along the cross-layer anchors can be traced backwards to reconstruct the complete reasoning chain from the supporting material node, through the cross-layer anchors, traversing the rule nodes layer by layer, to the final audit conclusion. This reasoning chain, as a verification path, is output along with the audit report, enabling auditors to intuitively understand the basis for each compliance conclusion, and also providing traceable evidence support for subsequent objections and appeals.
[0081] In actual deployment scenarios, the content of the static rule layer is maintained synchronously with the updates to regulatory policies. When regulatory documents are revised, the compliance element extraction and ontology model update process is re-executed, the affected nodes and their inherited constraints in the static rule layer are locally corrected, and the constraint compatibility checks of dynamic instance layer nodes that have cross-layer anchor point associations with the corrected nodes are triggered in batches to ensure that the two-layer graph structure is always consistent with the latest regulatory requirements.
[0082] Perform constraint compatibility checks on the candidate anchor point set to verify whether the proof material nodes conform to inheritance constraints. Establish the verified candidate anchor points as cross-layer anchor points. Connect the static rule layer and the dynamic instance layer into a unified two-layer atlas structure, including:
[0083] Extract the corresponding static rule layer nodes from the candidate anchor point set, traverse upwards along the static rule layer nodes to the upper-level concept node in the hierarchical representation, and aggregate the inherited constraints on the traversal path in hierarchical order to form a constraint propagation chain.
[0084] The preconditions are extracted from the constraint propagation chain and arranged in hierarchical order to form a verification sequence;
[0085] The semantic encoding result of the proof material node is matched and verified with the preconditions in the verification sequence in turn. When the semantic encoding result satisfies all the preconditions in the verification sequence, the corresponding candidate anchor point is marked as compliant. When the semantic encoding result does not satisfy any precondition, the verification is terminated and the corresponding candidate anchor point is marked as non-compliant.
[0086] Candidate anchor points marked as compliant are selected and established as the cross-layer anchor points;
[0087] In the dual-layer graph structure, a connection relationship is established from the proof material node to the cross-layer anchor point. The constraint propagation chain is attached to the connection relationship as a verification basis, thus completing the connection between the static rule layer and the dynamic instance layer.
[0088] After constructing the candidate anchor point set, a constraint compatibility check needs to be performed on each candidate anchor point to confirm whether the corresponding proof material node truly satisfies the inherited constraints specified by the static rule layer. The core of this process lies in tracing back up along the hierarchical representation of the static rule layer, aggregating all inherited constraints along the path into an ordered constraint propagation chain, and then comparing the semantic encoding results of the proof material node with this chain level by level to determine the compliance of the candidate anchor point.
[0089] Each candidate anchor point is sequentially retrieved from the candidate anchor point set, and its corresponding static rule layer node is extracted, denoted as [node name missing]. In the hierarchical representation of static rule layers, It does not exist in isolation, but rather within an inheritance system that refines downwards from higher-level conceptual nodes. Along... The hierarchical representation is traversed upwards, passing through several intermediate level nodes, until the parent concept node of the current concept branch is reached. During the traversal, the inheritance constraints attached to each node are read and collected sequentially according to the order of the traversed levels, ultimately forming a path from the parent concept node to... Constraint propagation chain Constraining the propagation chain The earlier constraints in the China-Vietnam framework originate from higher-level conceptual nodes, reflecting more general regulatory requirements; the later constraints originate from closer... The hierarchy itself reflects the detailed requirements for that specific rule node. Preserving this hierarchical order is crucial for the construction of subsequent verification sequences, because higher-level constraints are often prerequisites for lower-level constraints. If the higher-level constraints are no longer satisfied, the verification of the lower-level constraints becomes meaningless.
[0090] Constraint propagation chain Once constructed, the prerequisites corresponding to each constraint are extracted and arranged strictly according to the hierarchical order of the constraints in the chain to form a verification sequence. ,in To verify the total length of the sequence, The preconditions corresponding to the highest level of constraints. The prerequisites correspond to the lowest level of constraints. The content of the prerequisites covers multiple dimensions, such as the type of institution, the scope of business, the validity period of certificates, the qualification level of practitioners, and the environmental hygiene compliance status. All of these are stored in the constraint propagation chain in the form of structured semantic descriptions and can be directly compared with the semantic encoding results of the supporting documentation nodes.
[0091] The semantic encoding result of the proof material node comes from the cross-modal semantic vector generated during the multimodal feature extraction stage. This vector has uniformly mapped the textual features of the certificate document and the visual features of the environmental image to the semantic space, enabling semantic matching calculations with structured preconditions. This semantic encoding result is then sequentially compared with the verification sequence. Prerequisites Perform matching verification. Specifically, for each prerequisite... (in This is the sequence number of the current verification step. ), calculate semantic encoding results and The semantic similarity between the described compliance states is used to determine whether a precondition is met when the similarity exceeds a preset matching threshold, and the verification of the next precondition continues. If the similarity is below the matching threshold, the precondition is deemed not met, the verification process for the current candidate anchor point is immediately terminated, and the candidate anchor point is marked as a violation. This "termination upon non-compliance" verification strategy effectively reduces invalid computations, and can significantly improve overall verification efficiency, especially when there are a large number of candidate anchor points.
[0092] When the semantic encoding results of the proof material nodes pass through the verification sequence in sequence All During the verification of the matching of the preconditions, the corresponding candidate anchor points are marked as compliant. After marking, all candidate anchor points with compliant status are selected from the candidate anchor point set and formally established as cross-layer anchor points. The establishment of cross-layer anchor points means that the corresponding proof material nodes have satisfied the requirements of the corresponding rule nodes in the static rule layer at both the semantic and constraint levels, and are qualified to assume the responsibility of cross-layer connection in the two-layer graph structure.
[0093] In the two-layer graph structure, for each established cross-layer anchor point, a connection relationship is established from the proof material node to that cross-layer anchor point. This connection relationship not only records the association between the two nodes, but also the corresponding constraint propagation chain. This constraint propagation chain serves as the basis for verification and is attached to the connection. The addition of the constraint propagation chain means that the connection itself carries the complete compliance derivation process. When generating the audit report, the constraint propagation chain can be directly read along the connection to reconstruct the complete verification path from high-level regulatory requirements to specific supporting materials, providing a traceable basis for the audit conclusion.
[0094] Through the aforementioned constraint compatibility check and cross-layer anchor point establishment process, the connection between the static rule layer and the dynamic instance layer is completed, forming a unified two-layer graph structure. Rule nodes in the static rule layer are interconnected with supporting document nodes in the dynamic instance layer through cross-layer anchor points. The semantic and constraint relationships between the two layers are explicitly stored in a structured form, supporting precise triggering of subsequent review logic and efficient execution of compliance verification. For candidate anchor points marked as non-compliant, the corresponding precondition number is used to terminate verification. The specific details of any non-compliance are also recorded, serving as the basis for providing feedback to the organization on rectification measures, thus ensuring that the entire qualification review process is fully explainable and traceable.
[0095] In a real-world scenario of acne-prone skin care institution qualification verification, when an institution submits a nurse qualification certificate and the corresponding supporting document node undergoes constraint compatibility checks, if its semantic encoding result is in the first position of the verification sequence... If the prerequisite condition (corresponding to "the nurse's qualification level is not lower than level two") is not met, the candidate anchor point is immediately marked as non-compliant, the verification is terminated, and subsequent verification of more detailed prerequisite conditions such as operating procedures and hygiene standards will not be performed. This design ensures the rigor of the review while avoiding the continued consumption of computing resources on paths that have been identified as non-compliant, demonstrating the practical value of the hierarchical design of the constraint propagation chain.
[0096] Multimodal feature extraction is performed on the qualification materials, mapping the textual features of the certificate document and the visual features of the environmental image to a unified semantic space to generate cross-modal semantic vectors, including:
[0097] Perform layout parsing on the certificate document, extract the qualification type identifier from the title area, extract key fields from the content area, and semantically group them according to the qualification type identifier;
[0098] Construct a field dependency tree for the key fields in each semantic group according to their hierarchical relationship, with the qualification type identifier as the root node and the key fields as child nodes;
[0099] The semantic information of the field dependency tree is aggregated layer by layer from the leaf node to the root node. Each node integrates the information passed from the lower layer with the semantics of the current field and then passes it upward. The root node outputs the text feature vector.
[0100] The environmental image is used to perform semantic segmentation to identify facility areas, detect facility objects and extract boundary coordinates, calculate the spatial proximity between facility objects based on the boundary coordinates, and combine facility objects with a proximity greater than the proximity threshold into functional units.
[0101] Local visual features are extracted from facility objects in functional units, and global visual features are extracted from the area covered by functional units. These features are then weighted and fused according to spatial proportion to generate a visual feature vector.
[0102] The text feature vector and the visual feature vector are projected into a shared semantic space, the projection parameters are adjusted to minimize the distance between them, and the resulting fusion generates a cross-modal semantic vector.
[0103] When parsing the layout of certificate documents, a document structure recognition method based on region classification is used to divide the document page into a title area, a content area, a seal area, and supplementary areas. The title area is usually located at the top of the document, with a large font and obvious formatting characteristics. Qualification type identifiers, such as category tags like "Medical Institution Practice License," "Health License," or "Skin Care Service Qualification Certificate," are extracted from it. The content area contains the core fields of the certificate, such as the institution name, validity period, scope of permission, responsible person information, and issuing authority. Key field extraction models are used to identify and locate each field. After extraction, based on the qualification type identifier, fields in different types of certificates are grouped into corresponding semantic groups, thus avoiding confusion of field information between different certificate types. For example, the field grouping for a health license includes fields related to health conditions, while the field grouping for a practice license focuses on the scope of treatment and information on practicing personnel.
[0104] The key fields within each semantic group are used to construct a field dependency tree based on their hierarchical relationships. The qualification type identifier is used as the root node, each first-level field (such as "Organization Information", "Scope of Licensing", and "Validity Period") as intermediate nodes, and specific field values (such as "Organization Name: [Name of Nursing Center]" and "Scope of Licensing: Acne Care") as leaf nodes. The construction of the field dependency tree relies on a predefined field hierarchy template, with different qualification types corresponding to different tree structure templates. For certificates with more complex structures, the tree depth can reach 3 to 4 levels, and the semantic hierarchy between nodes at each level is determined by the hierarchical labeling of the field tags.
[0105] After the field dependency tree is constructed, semantic information is aggregated layer by layer from the leaf nodes to the root node. The semantic representation of the leaf nodes is obtained by encoding the field value text. An initial semantic vector is generated from the field text using a pre-trained language model, denoted as . ,in Leaf nodes are numbered. After receiving the semantic vector from its lower-level child nodes, intermediate nodes fuse it with their own semantic vector. The fusion method combines weighted summation with nonlinear transformation; that is, the vector passed from the lower level is concatenated with the current node's vector, then subjected to linear transformation and activation function processing to generate the node's aggregated semantic vector, which continues to be passed upwards. The root node, after aggregating the semantic information of the entire tree, outputs the text feature vector. This vector comprehensively encodes the structural information and field semantic information of the certificate document, and has strong semantic expression capabilities.
[0106] Semantic segmentation is performed on environmental images, dividing pixels into different facility area categories, such as disinfection areas, operating table areas, storage areas, and treatment bed areas. Semantic segmentation employs an image segmentation network based on an encoder-decoder structure to generate pixel-by-pixel category prediction maps for the input image. Based on the semantic segmentation results, further object detection is performed on specific facility objects within each facility area, extracting the bounding box coordinates of each object, denoted as . ,in Number the facility objects, and These are the coordinates of the top left and bottom right corners of the bounding box, respectively.
[0107] Spatial proximity between facility objects is calculated based on their bounding box coordinates. The proximity is calculated using the normalized Euclidean distance between the center points of the bounding boxes of two facility objects; the smaller the distance, the greater the proximity. Specifically, facility objects... With facility objects Proximity between Defined as a monotonically decreasing function of distance, when Greater than the preset neighbor threshold At that time, if two objects are considered to be spatially adjacent, they are grouped into the same functional unit. The functional unit division adopts the connectivity clustering method, that is, establishing connection relationships for all pairs of facility objects that meet the proximity condition, and grouping mutually adjacent facility objects into the same functional unit through connectivity component analysis. For example, if the disinfection cabinet and the operating table are spatially adjacent, they are grouped into a "disinfection operation functional unit", which facilitates subsequent verification with the functional area requirements in the qualification specifications.
[0108] A two-layer visual feature extraction process is performed for each functional unit. Local visual features are extracted separately for the bounding box regions of each facility object within the functional unit. A convolutional neural network is used to encode the features of each clipped region, obtaining the local feature vector of each facility object. Then, the local feature vectors of all facility objects within the functional unit are averaged to obtain the local visual feature representation of that functional unit. ,in The functional units are numbered. Global visual features are extracted from the overall coverage area of the functional unit in the image. The bounding boxes of all member facilities of the functional unit are merged into a minimum bounding rectangle. Overall feature encoding is then performed on the image of this region to obtain the global visual feature vector. .
[0109] The visual feature vector of a functional unit is generated by weighting the bounding box area of each facility within the functional unit relative to the total coverage area of the functional unit, and then weighting and fusing local and global visual features. The visual feature vectors of all functional units are aggregated, and an attention mechanism is used to assign different importance weights to each functional unit, ultimately generating the visual feature vector of the entire environmental image. This vector comprehensively depicts the layout of facilities and the distribution of functional areas in the environmental imagery.
[0110] Text feature vectors With visual feature vectors The text is projected onto a shared semantic space to achieve cross-modal alignment. The projection employs two independent linear projection layers, mapping the text feature vectors and visual feature vectors to a shared space of the same dimension, respectively, to obtain the projected text representation. Visual representation The goal of adjusting the projection parameters is to minimize... and The cosine distance in the shared semantic space allows textual and visual information describing the qualification status of the same institution to be closer to each other in the semantic space, thus achieving cross-modal semantic alignment. After the projection parameters converge, and The vectors are concatenated and processed through a fusion layer to generate the final cross-modal semantic vector. This cross-modal semantic vector carries both the structured semantic information of the certificate document and the spatial visual information of the environmental image, providing a unified semantic expression basis for the subsequent construction of instance nodes and rule matching.
[0111] like Figure 3 As shown in the figure, this graph illustrates the numerical comparison of certificate document text features, environmental image visual features, and cross-modal semantic vectors in a unified semantic space. The horizontal axis represents the semantic feature dimensions (D1 to D8), and the vertical axis represents the feature values. The blue bars represent the components of the text feature vectors generated from the certificate document after layout parsing and field dependency tree aggregation in each dimension; the green bars represent the components of the visual feature vectors generated from the environmental image after semantic segmentation, facility object detection, and functional unit weighted fusion in each dimension; the orange curve represents the cross-modal semantic vector obtained by mapping the text feature vectors and visual feature vectors to the shared semantic space through linear projection and minimizing their cosine distance. The orange curve is higher than or between the two sets of bars in most dimensions, intuitively demonstrating the enhanced semantic expressiveness of cross-modal alignment and fusion. That is, the fused vector can simultaneously carry the structured semantic information of the certificate text and the spatial visual information of the environmental image, providing a unified semantic expression basis for subsequent instance node construction and rule matching.
[0112] The cross-modal semantic vectors are constructed as instance nodes and inserted into the dynamic instance layer. The semantic distance between the instance nodes and the rule nodes in the static rule layer is calculated. Rule nodes with a semantic distance less than a semantic threshold are selected as the applicable rule set, including:
[0113] The cross-modal semantic vector is converted into a graph node representation and node attribute fields are configured. Feature parameters are extracted from the vector dimension of the cross-modal semantic vector and mapped to the node attribute fields to generate instance nodes carrying structured attributes.
[0114] A node index structure is constructed in the dynamic instance layer. An index position is assigned to the instance node in the node index structure according to the node attribute field. The neighboring nodes in the dynamic instance layer are retrieved through the index position. A bidirectional link is established between the instance node and each of the neighboring nodes. The instance node carrying the bidirectional link is inserted into the dynamic instance layer.
[0115] Obtain the semantic vectors of the rule nodes in the static rule layer, measure the distance between the cross-modal semantic vectors and each of the semantic vectors to obtain the initial semantic distance, and obtain the historical distance data of the neighboring nodes based on the bidirectional links and perform correction to obtain the semantic distance;
[0116] Filter rule nodes whose semantic distance is less than the semantic threshold, and group and integrate the filtered rule nodes according to the node index structure to form the applicable rule set.
[0117] After multimodal feature fusion, cross-modal semantic vectors need to be converted into node representations recognizable by the graph database before they can be inserted into the dynamic instance layer and participate in subsequent rule matching processes. During this conversion, the dimensions of the cross-modal semantic vectors are grouped and mapped to predefined node attribute fields, including institution type identifier, certificate validity period, facility compliance feature, and comprehensive semantic embedding. The comprehensive semantic embedding field directly stores the complete cross-modal semantic vector for subsequent vector distance calculation; the institution type identifier and facility compliance feature fields are extracted from specific dimensional intervals of the semantic vectors through linear projection, forming discrete attribute values that can be directly parsed by the index structure. Through this mapping, the qualification materials of each institution to be reviewed are converted into an instance node carrying structured attributes. The node simultaneously stores both the original vector representation and the structured attributes, complementing each other and serving the needs of precise semantic calculation and fast index retrieval, respectively.
[0118] In the dynamic instance layer, the node index structure is constructed using a composite approach combining hierarchical hashing and vector nearest neighbor indexing. The hierarchical hash index uses the organization type identifier and facility compliance characteristics from the node attribute fields as hash keys to divide instance nodes into corresponding index buckets. Nodes within the same index bucket have high semantic similarity. The vector nearest neighbor index is constructed based on a comprehensive semantic embedding field, using a hierarchical navigable small-world graph (HNSW) structure to organize the semantic vectors of all nodes, supporting approximate nearest neighbor retrieval for any query vector. When a new instance node needs to be inserted, its hash key is calculated based on its node attribute fields and it is located in the corresponding index bucket. Simultaneously, a nearest neighbor search is performed in the vector nearest neighbor index to retrieve the nearest neighbor nodes with the closest semantic distance to the instance node. Bidirectional links are established between the instance node and each neighboring node, with the link attributes recording the estimated semantic similarity between the two nodes and the timestamp when the link was established. After the bidirectional links are established, the instance node is officially written into the dynamic instance layer, and the hierarchical hash index and vector nearest neighbor index are updated synchronously to ensure that subsequently inserted nodes can retrieve the node.
[0119] The calculation of semantic distance is divided into two stages: initial distance measurement and correction based on historical data. In the initial distance measurement stage, the semantic vectors of all rule nodes in the static rule layer are obtained. Let the semantic vector of the first node be... The semantic vector of each rule node is The cross-modal semantic vector of the current instance node is Then the initial semantic distance Calculated using cosine distance:
[0120]
[0121] in, This is the cross-modal semantic vector of the current instance node. For the static rule layer, the first The semantic vector of each rule node. This represents the initial semantic distance between the two. The initial distance only reflects the direct relative position of the current instance node and the rule node in the semantic space, without considering the historical review experience accumulated by existing nodes in the dynamic instance layer.
[0122] During the correction phase, a set of neighboring nodes connected to the current instance node is obtained through bidirectional links, and historical distance data of each neighboring node for the same rule node is extracted. Let the first... The neighboring node and the first The historical semantic distance between each rule node is The link similarity weight between the neighboring node and the current instance node is . Then the corrected semantic distance Calculated using a weighted fusion method:
[0123]
[0124] in, The initial distance fusion weights have a value range of [value range missing]. This is used to balance the contribution ratio of the initial distance and the historical correction term; For the first The neighboring node and the first Historical semantic distance of each rule node; For the first Link similarity weights corresponding to each neighboring node; This is the final semantic distance used after correction. When a rule node has no historical distance records among all its neighboring nodes, the correction term degenerates to zero. Directly equal to This ensures the stability of the calculation process during the layer initialization phase or the first review of a new category of institution.
[0125] After semantic distance correction is completed, the corrected semantic distance of all rule nodes is compared with the preset semantic threshold. Compare and retain those that meet the requirements. The rule nodes are used as the candidate set. The value is determined based on the 95th percentile of the distance distribution between correctly matched rule nodes and instance nodes in historical audit data. It is pre-set through statistical analysis before system deployment and is updated regularly as audit data accumulates. The candidate rule nodes obtained from the screening may come from different levels and categories of the static rule layer. If they are directly laid out as an unordered set, the triggering order of subsequent audit logic will lack a basis, which may lead to higher-level rules being activated prematurely when the preconditions are not met.
[0126] like Figure 4As shown in the figure, this diagram illustrates the semantic distance distribution between instance nodes and rule nodes in the static rule layer, as well as the process of selecting applicable rules based on semantic thresholds. The vertical axis lists 10 rule nodes (R01 to R10), covering categories such as health permits, personnel qualifications, facilities and equipment, and operating procedures; the horizontal axis represents the semantic distance value calculated using cosine distance. Each rule node corresponds to a horizontal bar, the length of which represents the semantic distance between the rule node and the current instance node. Green bars represent rule nodes with a semantic distance less than a preset semantic threshold (0.18), and these nodes are included in the applicable rule set; gray bars represent rule nodes with a semantic distance greater than the threshold, and these are excluded. The threshold positions are clearly marked with red dashed lines and accompanied by threshold labels. This diagram intuitively presents the basis and results of selecting rule nodes from the static rule layer that are semantically relevant to the current institution's qualification instance and require triggering review logic, reflecting the core selection logic of "rule nodes with a semantic distance less than the semantic threshold are included in the applicable rule set."
[0127] To address the aforementioned issues, the selected rule nodes are grouped and integrated based on the node index structure. Specifically, using the category information of the rule nodes recorded in the hierarchical hash index, candidate rule nodes are grouped into corresponding rule groups according to regulatory categories (such as health permits, personnel qualifications, facilities and equipment, and operating procedures). Nodes within the same rule group are arranged from shallow to deep according to their hierarchy depth in the static rule layer, forming a verification priority order within the group. The execution order between different rule groups is determined based on the dependencies between categories in the agency's audit business process. For example, the health permit rule group must complete verification before the personnel qualification rule group because the compliance determination of personnel qualifications depends on the confirmation of the agency's overall licensing status. After grouping and integration, a hierarchical set of applicable rules is formed. This rule set is organized in a tree structure, with the root node corresponding to the regulatory category group and the leaf nodes corresponding to specific rule nodes. Each path from the root to the leaf represents a complete rule triggering chain. The tree structure of the applicable rule set and the bidirectional links in the dynamic instance layer together provide a structural foundation for the establishment of subsequent cross-layer anchors and the orderly execution of audit logic, ensuring that the compliance verification process proceeds in accordance with the inherent constraints of the regulatory logic and avoiding the disruption of inter-rule dependencies.
[0128] For the applicable rule set, cross-layer anchor points are established between the instance nodes and each rule node. These anchor points trigger corresponding audit logic to perform compliance verification and generate an audit report. The audit report and its verification path within the two-layer graph structure are then output, including:
[0129] Traverse each rule node in the applicable rule set and construct a cross-layer anchor point between the instance node and the corresponding rule node. Configure a bidirectional transmission channel for the cross-layer anchor point and transmit the semantic features of the instance node to the rule node through the bidirectional transmission channel to obtain the encapsulated audit logic.
[0130] Based on the bidirectional transmission channel, a data mapping relationship is established between the audit logic and the instance node. The audit logic parses the semantic features to extract compliance verification elements and performs rule matching to generate verification tags. The verification tags are then sent back to the instance node through the bidirectional transmission channel.
[0131] At the instance node, the returned verification tags are aggregated to construct a verification result set, and the verification result set is integrated with the connection information of the cross-layer anchor point to form an audit report.
[0132] Tracing back from the instance node along the bidirectional transmission channel to the node index position of the dynamic instance layer, and tracing to the storage position of the rule node in the static rule layer through cross-layer anchor points, the connection sequence of the node index position, the storage position, and the bidirectional transmission channel is concatenated to generate a verification path and output with the audit report.
[0133] After the applicable rule set is determined, for each rule node, cross-layer anchors are constructed sequentially between instance nodes in the dynamic instance layer and rule nodes in the static rule layer. The construction of cross-layer anchors is not a simple pointer association, but rather the configuration of a bidirectional data flow channel between each pair of instance nodes and rule nodes. This bidirectional channel physically consists of two directed edges in opposite directions: one from the instance node to the rule node, used for forward transmission of semantic features; the other from the rule node to the instance node, used for reverse feedback of the audit logic execution results. During channel establishment, the channel identifier, the connected instance node identifier, the rule node identifier, and the channel creation timestamp are recorded simultaneously. This metadata is referenced as part of the connection sequence when subsequently generating the verification path.
[0134] During the forward transmission phase, the cross-modal semantic vector carried by the instance node, along with its associated semantic features such as certificate field parsing results and facility object spatial relationship descriptions, is encapsulated into a feature payload packet and sent to the corresponding rule node via the forward edge of the bidirectional transmission channel. Upon receiving the feature payload packet, the rule node extracts the matching logic unit from its stored audit logic and establishes a data mapping relationship between the audit logic and each semantic field in the feature payload packet. The establishment of this data mapping relationship relies on a predefined field binding table within the rule node, which describes the names of the input fields required for each audit logic and their corresponding positions in the semantic features. When a field in the feature payload packet matches the expected field name in the binding table, the field value is injected into the input slot of the corresponding audit logic, completing the data mapping. If the feature payload packet contains a field not covered by the binding table, this field is marked as a redundant field and temporarily stored for later use; if the binding table contains a necessary field that the feature payload packet fails to provide, a field missing anomaly is triggered, and the corresponding missing item information is recorded in the verification flag.
[0135] After completing data mapping, the audit logic enters the execution phase. The execution process is divided into different processing branches based on the rule type to which the rule node belongs. For format compliance rules, the audit logic performs regular expression matching or enumeration value comparison on the injected text fields to determine whether the certificate number format, the date format of the validity period field, and the consistency between the organization name and registration information conform to the preset format specifications in the static rule layer. For qualification integrity rules, the audit logic traverses the list of required qualifications associated with the rule node, checking item by item whether there are corresponding certificate instances and their validity markers in the instance nodes. For environmental compliance rules, the audit logic reads the facility spatial relationship description in the feature payload package, compares the layout information of functional units with the facility configuration requirements stored in the rule node, and verifies whether functional units such as disinfection areas, operating areas, and waiting areas are independently set up as required and meet minimum area or proximity constraints. After each audit logic is executed, a verification marker is generated. The verification marker includes the rule identifier, the execution conclusion (pass, fail, or pending verification), the key evidence fields that trigger the conclusion and their values, and the non-compliance description text in the case of failure.
[0136] All verification tokens are transmitted back to the instance nodes via the reverse edge of the bidirectional transmission channel. Each instance node maintains a verification token summary buffer; upon receiving a verification token, it appends it to the buffer and updates the verification status of the corresponding rule node to "completed." Once all verification tokens for all rule nodes in the applicable rule set have been transmitted back, all verification tokens in the summary buffer constitute the verification result set. The verification result set is grouped by rule type, with verification tokens for the same type of rule aggregating into the same group, facilitating the structured output of subsequent audit reports. The verification result set is integrated with the connection information of each cross-layer anchor point. This connection information includes the instance node identifier, rule node identifier, and channel identifier of the bidirectional transmission channel connected to each cross-layer anchor point. The integrated result set forms the audit report. The main body of the audit report includes a summary of verification conclusions for each rule type, a detailed description of non-compliant items and corresponding evidence fields, and a comprehensive judgment of the overall compliance status. The overall judgment logic is as follows: if there is no verification mark with the conclusion "failed" in the verification result set, the overall compliance status is judged as compliant; if there is one or more "failed" marks, the overall compliance status is judged as non-compliant, and all non-compliant items are listed in the report; if there is a "pending verification" mark but no "failed" mark, the overall compliance status is judged as pending manual review.
[0137] The verification path is generated starting from the node index position of the instance node in the dynamic instance layer. The node index position records the unique address identifier of the instance node in the storage structure of the dynamic instance layer, including a combination of the layer identifier and the node's internal number. Following the forward edge of the bidirectional transmission channel, the path is traced sequentially to the storage position of each rule node in the static rule layer. The storage position is also represented by a combination of the layer identifier and the rule node's internal number. The node index positions of the instance nodes, the bidirectional transmission channel connection sequence of each cross-layer anchor point, and the storage positions of each rule node are sequentially connected according to the establishment time of the anchor points, forming an ordered path record sequence. This path record sequence is the verification path, which fully describes the entire topology of this compliance verification process, starting from the instance data, passing through each cross-layer anchor point, and reaching each rule node. The verification path is bundled with the audit report, providing a complete traceability basis for the audit conclusions. Auditors or regulatory departments can use the node index positions and storage positions in the verification path to accurately locate all nodes involved in this audit and their relationships in the two-layer graph structure, thereby progressively reproducing and verifying the audit process.
[0138] In practical applications, when a skin care clinic submits certificate documents and environmental images, after multimodal feature extraction and instance node construction, the applicable rule set may simultaneously include multiple rule nodes such as health permit format verification rules, practitioner qualification verification rules, and treatment environment configuration rules. Through the aforementioned cross-layer anchor point construction and bidirectional transmission channel mechanism, the review logic of various rules can receive instance semantic features in parallel and perform verification independently, ultimately converging their respective verification tags to form a complete verification result set at the instance node. The review report includes both the specific verification conclusions of each rule and the verification path for subsequent traceability, effectively improving the transparency and traceability of the institution's qualification review process.
[0139] A second aspect of this invention provides an institutional qualification verification system based on knowledge graphs and multimodal collaboration, comprising:
[0140] The material receiving unit is used to receive qualification materials uploaded by acne care institutions to be reviewed, including certificate documents and environmental images;
[0141] The graph construction unit is used to construct a two-layer graph structure for acne skin care qualifications, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to the regulatory requirements, and the dynamic instance layer stores the qualification instances submitted by the institution. The association between the static rule layer and the dynamic instance layer is established through cross-layer anchor points.
[0142] The feature extraction unit is used to extract multimodal features from the qualification materials, map the text features of the certificate document and the visual features of the environmental image to a unified semantic space, and generate cross-modal semantic vectors.
[0143] The rule matching unit is used to construct the cross-modal semantic vector into instance nodes and insert them into the dynamic instance layer, calculate the semantic distance between the instance nodes and the rule nodes in the static rule layer, and filter the rule nodes with semantic distance less than the semantic threshold as the applicable rule set.
[0144] The audit generation unit is used to establish cross-layer anchor points between the instance node and each rule node for the applicable rule set, trigger the corresponding audit logic through the cross-layer anchor points to perform compliance verification and generate an audit report, and output the audit report and its verification path in the two-layer graph structure.
[0145] A third aspect of the present invention provides an electronic device, comprising:
[0146] processor;
[0147] Memory used to store processor-executable instructions;
[0148] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.
[0149] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.
[0150] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.
[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for institutional qualification verification based on knowledge graphs and multimodal collaboration, characterized in that, include: We receive qualification materials uploaded by acne care institutions awaiting review, including certificate documents and environmental images; A two-layer map structure for acne-prone skin care qualifications is constructed, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to the regulatory requirements, and the dynamic instance layer stores the qualification instances submitted by the institution. The relationship between the static rule layer and the dynamic instance layer is established through cross-layer anchor points. Multimodal feature extraction is performed on the qualification materials, and the textual features of the certificate document and the visual features of the environmental image are mapped to a unified semantic space to generate cross-modal semantic vectors. The cross-modal semantic vector is constructed as an instance node and inserted into the dynamic instance layer. The semantic distance between the instance node and the rule node in the static rule layer is calculated, and rule nodes with a semantic distance less than the semantic threshold are selected as the applicable rule set. For the applicable rule set, cross-layer anchor points are established between the instance node and each rule node. The corresponding audit logic is triggered through the cross-layer anchor points to perform compliance verification and generate an audit report. The audit report and its verification path in the two-layer graph structure are then output.
2. The method according to claim 1, characterized in that, A two-layer map structure for acne-prone skin care qualifications is constructed, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to regulatory requirements, and the dynamic instance layer stores qualification instances submitted by institutions. The association between the static rule layer and the dynamic instance layer is established through cross-layer anchor points, including: Compliance elements are extracted from regulatory documents and an ontology model is constructed. Based on the ontology model, a hierarchical representation is organized in the static rule layer. The hierarchical representation includes upper-level concept nodes and lower-level detailed rule nodes. Inheritance constraints are established for the upper-level concept nodes and the lower-level detailed rule nodes. The inheritance constraints stipulate that the lower-level detailed rule nodes satisfy the preconditions of the upper-level concept nodes. The qualification instances are structured and organized into an instance diagram, and the supporting material nodes of the instance diagram are represented in the dynamic instance layer; Semantic encoding is performed on the proof material nodes, and the semantic distance between the encoding results and the nodes in the static rule layer is calculated. The node combination with the smallest semantic distance is selected as the candidate anchor set. Constraint compatibility checks are performed on the candidate anchor set to verify whether the proof material nodes meet the inheritance constraints. The candidate anchors that pass the verification are established as cross-layer anchors, and the static rule layer and the dynamic instance layer are connected into a unified two-layer graph structure. Hierarchical path information is attached to the cross-layer anchor points. When the dynamic instance layer changes, the path information is used to trace back to the static rule layer and trigger the corresponding level of review rules.
3. The method according to claim 2, characterized in that, Perform constraint compatibility checks on the candidate anchor point set to verify whether the proof material nodes conform to inheritance constraints. Establish the verified candidate anchor points as cross-layer anchor points. Connect the static rule layer and the dynamic instance layer into a unified two-layer atlas structure, including: Extract the corresponding static rule layer nodes from the candidate anchor point set, traverse upwards along the static rule layer nodes to the upper-level concept node in the hierarchical representation, and aggregate the inherited constraints on the traversal path in hierarchical order to form a constraint propagation chain. The preconditions are extracted from the constraint propagation chain and arranged in hierarchical order to form a verification sequence; The semantic encoding result of the proof material node is matched and verified with the preconditions in the verification sequence in turn. When the semantic encoding result satisfies all the preconditions in the verification sequence, the corresponding candidate anchor point is marked as compliant. When the semantic encoding result does not satisfy any precondition, the verification is terminated and the corresponding candidate anchor point is marked as non-compliant. Candidate anchor points marked as compliant are selected and established as the cross-layer anchor points; In the dual-layer graph structure, a connection relationship is established from the proof material node to the cross-layer anchor point. The constraint propagation chain is attached to the connection relationship as a verification basis, thus completing the connection between the static rule layer and the dynamic instance layer.
4. The method according to claim 1, characterized in that, Multimodal feature extraction is performed on the qualification materials, mapping the textual features of the certificate document and the visual features of the environmental image to a unified semantic space to generate cross-modal semantic vectors, including: Perform layout parsing on the certificate document, extract the qualification type identifier from the title area, extract key fields from the content area, and semantically group them according to the qualification type identifier; Construct a field dependency tree for the key fields in each semantic group according to their hierarchical relationship, with the qualification type identifier as the root node and the key fields as child nodes; The semantic information of the field dependency tree is aggregated layer by layer from the leaf node to the root node. Each node integrates the information passed from the lower layer with the semantics of the current field and then passes it upward. The root node outputs the text feature vector. The environmental image is used to perform semantic segmentation to identify facility areas, detect facility objects and extract boundary coordinates, calculate the spatial proximity between facility objects based on the boundary coordinates, and combine facility objects with a proximity greater than the proximity threshold into functional units. Local visual features are extracted from facility objects in functional units, and global visual features are extracted from the area covered by functional units. These features are then weighted and fused according to spatial proportion to generate a visual feature vector. The text feature vector and the visual feature vector are projected into a shared semantic space, the projection parameters are adjusted to minimize the distance between them, and the resulting fusion generates a cross-modal semantic vector.
5. The method according to claim 1, characterized in that, The cross-modal semantic vectors are constructed as instance nodes and inserted into the dynamic instance layer. The semantic distance between the instance nodes and the rule nodes in the static rule layer is calculated. Rule nodes with a semantic distance less than a semantic threshold are selected as the applicable rule set, including: The cross-modal semantic vector is converted into a graph node representation and node attribute fields are configured. Feature parameters are extracted from the vector dimension of the cross-modal semantic vector and mapped to the node attribute fields to generate instance nodes carrying structured attributes. A node index structure is constructed in the dynamic instance layer. An index position is assigned to the instance node in the node index structure according to the node attribute field. The neighboring nodes in the dynamic instance layer are retrieved through the index position. A bidirectional link is established between the instance node and each of the neighboring nodes. The instance node carrying the bidirectional link is inserted into the dynamic instance layer. Obtain the semantic vectors of the rule nodes in the static rule layer, measure the distance between the cross-modal semantic vectors and each of the semantic vectors to obtain the initial semantic distance, and obtain the historical distance data of the neighboring nodes based on the bidirectional links and perform correction to obtain the semantic distance; Filter rule nodes whose semantic distance is less than the semantic threshold, and group and integrate the filtered rule nodes according to the node index structure to form the applicable rule set.
6. The method according to claim 1, characterized in that, For the applicable rule set, cross-layer anchor points are established between the instance nodes and each rule node. These anchor points trigger corresponding audit logic to perform compliance verification and generate an audit report. The audit report and its verification path within the two-layer graph structure are then output, including: Traverse each rule node in the applicable rule set and construct a cross-layer anchor point between the instance node and the corresponding rule node. Configure a bidirectional transmission channel for the cross-layer anchor point and transmit the semantic features of the instance node to the rule node through the bidirectional transmission channel to obtain the encapsulated audit logic. Based on the bidirectional transmission channel, a data mapping relationship is established between the audit logic and the instance node. The audit logic parses the semantic features to extract compliance verification elements and performs rule matching to generate verification tags. The verification tags are then sent back to the instance node through the bidirectional transmission channel. At the instance node, the returned verification tags are aggregated to construct a verification result set, and the verification result set is integrated with the connection information of the cross-layer anchor point to form an audit report. Tracing back from the instance node along the bidirectional transmission channel to the node index position of the dynamic instance layer, and tracing to the storage position of the rule node in the static rule layer through cross-layer anchor points, the connection sequence of the node index position, the storage position, and the bidirectional transmission channel is concatenated to generate a verification path and output with the audit report.
7. An institutional qualification review system based on knowledge graphs and multimodal collaboration, used to implement the method as described in any one of claims 1-6, characterized in that, include: The material receiving unit is used to receive qualification materials uploaded by acne care institutions to be reviewed, including certificate documents and environmental images; The graph construction unit is used to construct a two-layer graph structure for acne skin care qualifications, including a static rule layer and a dynamic instance layer. The static rule layer stores the review logic corresponding to the regulatory requirements, and the dynamic instance layer stores the qualification instances submitted by the institution. The association between the static rule layer and the dynamic instance layer is established through cross-layer anchor points. The feature extraction unit is used to extract multimodal features from the qualification materials, map the text features of the certificate document and the visual features of the environmental image to a unified semantic space, and generate cross-modal semantic vectors. The rule matching unit is used to construct the cross-modal semantic vector into instance nodes and insert them into the dynamic instance layer, calculate the semantic distance between the instance nodes and the rule nodes in the static rule layer, and filter the rule nodes with semantic distance less than the semantic threshold as the applicable rule set. The audit generation unit is used to establish cross-layer anchor points between the instance node and each rule node for the applicable rule set, trigger the corresponding audit logic through the cross-layer anchor points to perform compliance verification and generate an audit report, and output the audit report and its verification path in the two-layer graph structure.
8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.
