Data desensitization and sensitive information management and control method based on multi-modal recognition and related device

Abstract

The application provides a data desensitization and sensitive information management and control method based on multi-modal recognition and related devices, and relates to the technical field of information security. Multi-modal feature extraction and cross-modal sensitive entity association are performed on supply chain multi-modal original data, and a structured recognition result is output. A desensitization strategy is constructed based on supply chain compliance requirements and business scenarios, and an execution instruction is generated. In the data display and sharing link, layered dynamic rendering desensitization is adopted, and a traceability association is established. Feedback data is collected to iteratively optimize the model and rules. Through the above technical means, the problems of insufficient recognition accuracy, static desensitization strategy, data integrity destruction, lack of traceability and optimization in the prior art are solved, the sensitive information recognition accuracy is improved, dynamic desensitization is realized, the integrity of the original data is protected, and the compliance traceability and adaptive optimization management and control effect are supported.

Description

Technical Field

[0001] This application relates to the field of information security technology, and in particular to a data desensitization and sensitive information management method and related apparatus based on multimodal recognition. Background Technology

[0002] With the accelerated digitalization of the supply chain, massive amounts of multimodal data are continuously generated across all business segments. This data, serving as a core carrier for business collaboration and compliance management, encompasses various types, including text, images, and audio / video. Text data includes purchase contracts, order documents, customs declarations, supplier qualification documents, warehouse ledgers, customer receipt information, and settlement invoices. Image data includes photos of cargo labels, scanned copies of customs declarations, photos of transport vehicle license plates, warehouse monitoring screenshots, scanned copies of supplier identification documents, and images of product packaging labels. Audio / video data includes recordings of logistics communications, warehouse inspection videos, cross-border loading and unloading monitoring audio / video, and recordings of supply chain collaboration meetings. This type of data contains a large amount of sensitive information, such as customer personal privacy information (name, ID number, delivery address, and contact information), corporate trade secrets (purchase prices, supplier cooperation agreements, core customer lists, and warehouse inventory data), and cross-border regulatory sensitive information (customs data, cross-border logistics tracks, and import / export qualification information).

[0003] Existing technical solutions are generally designed for general scenarios and are difficult to adapt to the specific needs of supply chain scenarios and the complex characteristics of multimodal data. Specifically, existing methods lack cross-modal sensitive entity association mechanisms, and cannot uniformly identify the corresponding information in the text of the supplier's unified social credit code and the scanned ID card, or the voice content in the logistics recording, leading to missed detection of sensitive information. At the same time, the recognition accuracy of supply chain professional terms such as customs declaration documents and warehouse ledgers is insufficient, which easily leads to false detections. In addition, de-identification strategies mostly adopt static rule configuration and fail to achieve dynamic generation by combining supply chain compliance requirements, business scenario tags, and data permission matrices. For example, they cannot intelligently resolve rule conflicts and generate appropriate de-identification instructions based on different links such as procurement management, warehousing and logistics, cross-border customs clearance, and the different access permissions of administrators, operators, and partners, resulting in over-identification affecting business efficiency or under-identification causing compliance risks. At the anonymization execution level, existing technologies often directly tamper with the original data, compromising the integrity and business logic of supply chain documents, contracts, and monitoring data. They fail to provide on-demand processing for data display, API calls, and external sharing, and lack a traceability mechanism between the original data and the anonymized copy, making it difficult to meet the needs of customs audits and accountability. More critically, existing systems lack closed-loop iterative capabilities based on anonymization effect feedback, compliance audit results, and business-adaptive data. They cannot adaptively optimize for updates in supply chain data types, changes in cross-border compliance, and expansion of business scenarios, leading to decreased accuracy of the identification model and invalidation of anonymization rules. This makes it difficult to continuously ensure the effectiveness of sensitive information control in the long term. These problems result in significant deficiencies in the accurate identification of multimodal supply chain data, dynamic strategy execution, business value protection, and compliance traceability, failing to balance privacy protection and data usability.

[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0005] This application provides a data desensitization and sensitive information management method and related apparatus based on multimodal recognition, which can effectively solve the problems of inaccurate identification of sensitive information in multimodal data of the supply chain, low business efficiency or compliance risks caused by static configuration of desensitization strategy, destruction of integrity of original data by tampering, and lack of traceability mechanism and closed-loop optimization, thereby achieving high-precision identification, dynamic adaptation of desensitization, data integrity protection, compliance traceability support and adaptive iterative optimization.

[0006] Firstly, the data desensitization and sensitive information management method based on multimodal recognition provided in this application adopts the following technical solution: A method for data desensitization and sensitive information control based on multimodal recognition includes: Multimodal feature extraction and cross-modal sensitive entity association are performed on multimodal raw data of text, images, audio and video in supply chain scenarios. Sensitive information identification is completed through dynamic confidence calibration, and structured identification results containing global ID of sensitive information, cross-modal location mapping and risk level are output. Based on supply chain compliance requirements, business scenario tags, and data permission matrix, a de-identification strategy is constructed through intelligent rule generation and conflict resolution. The de-identification rules are matched according to the structured identification results to generate differentiated de-identification execution instructions. In the process of supply chain data display, interface call and external sharing, layered dynamic rendering is used to perform de-identification processing, without tampering with the original data and generating a de-identified data copy, while establishing a traceability relationship between the original data and the de-identified copy; By collecting feedback on the effectiveness of data masking, compliance audit results, and supply chain business adaptation data, we can adaptively iterate and optimize the identification model, data masking rules, and rendering granularity to form a closed-loop management system.

[0007] Optionally, the multimodal raw data of the supply chain includes: supply chain text data, specifically purchase contracts, order documents, customs declarations, supplier qualification documents, warehouse ledgers, customer receiving information, and settlement invoice texts; Supply chain image data includes: photos of goods labels, scanned copies of customs declarations, photos of transport vehicle license plates, screenshots of warehouse surveillance footage, scanned copies of supplier ID documents, and images of goods packaging labels; Supply chain audio and video data includes: logistics communication recordings, warehouse inspection videos, cross-border loading and unloading monitoring audio and video, and supply chain collaboration meeting recordings.

[0008] Optionally, the unified recognition engine performs the following processing: The system preprocesses multimodal data from the supply chain, performs supply chain terminology normalization and named entity recognition on text data, performs target detection and key area localization on image data, and performs speech transcription and inter-frame parsing on audio and video data. Construct a cross-modal association graph, assign a global ID to the same sensitive entity, and perform location association and weight binding on customer ID numbers in text, supplier unified social credit codes, corresponding document information in images, and voice-sensitive content in audio and video. A confidence-weighted fusion and anomaly verification mechanism is adopted to perform secondary calibration on the cross-modal recognition results, resulting in a structured recognition result that includes sensitive type, location coordinates, and risk level.

[0009] Optionally, the dynamic strategy engine builds a strategy system based on supply chain scenarios: Establish a supply chain compliance knowledge base, integrating customs supervision regulations, cross-border data compliance requirements, personal information protection regulations, and supply chain trade secret protection rules; Based on five business scenarios—procurement management, warehousing and logistics, cross-border customs clearance, supplier collaboration, and order fulfillment—and combined with the data permission matrix of administrators, operators, partners, and third-party regulators, the system automatically generates de-identification rules. Through a rule conflict resolution mechanism, rule conflicts are resolved based on sensitivity level, compliance priority, and business availability, generating de-identification execution instructions that include de-identification algorithms, de-identification granularity, and execution time.

[0010] Optionally, layered dynamic rendering specifically includes: Based on the de-identification execution instructions and cross-modal location mapping, sensitive areas in the multimodal data of the supply chain are located; For text-based supply chain data, mask replacement, field generalization, and partial retention are used; for image-based data, partial pixelation and key area occlusion are used; and for audio-visual data, voice desensitization and image blurring are used. Each set of de-identified copies is assigned a unique traceability identifier, which is associated with the original data storage address, de-identification rule parameters, access subject, and operation time, and is used for supply chain compliance auditing and data traceability.

[0011] Optionally, dynamic rendering supports granular adaptive adjustment based on the supply chain access subject and scenario: Grant internal management personnel access to preview low-sensitization or even raw data; Medium-granularity anonymized data is made available to warehouse and logistics personnel; We provide high-level, strictly anonymized data to overseas partners and third-party service providers, achieving minimal anonymization without disrupting the supply chain's business logic.

[0012] Optionally, closed-loop iterative optimization includes: Collect false detections, missed detections, business availability scores, and compliance audit results in supply chain scenarios, and identify and locate rule defects through error analysis; Incremental training and transfer learning were used to optimize the multimodal recognition model, the supply chain compliance rule base and desensitization strategy were updated, and the layered rendering granularity parameters were adjusted. The optimized model and rules will be iteratively updated to adapt to changes in supply chain data types, cross-border compliance changes, and business scenario expansion needs.

[0013] Secondly, this application provides a data desensitization and sensitive information management system based on multimodal recognition, comprising: The feature extraction module is used to extract multimodal features and associate cross-modal sensitive entities from multimodal raw data such as text, images, and audio / video in supply chain scenarios. It completes the identification of sensitive information through dynamic confidence calibration and outputs structured identification results containing global ID of sensitive information, cross-modal location mapping, and risk level. The rule matching module is used to construct a de-identification strategy based on supply chain compliance requirements, business scenario tags, and data permission matrix through intelligent rule generation and conflict resolution. It matches the de-identification rules according to the structured identification results and generates differentiated de-identification execution instructions. The association establishment module is used to perform de-identification processing through layered dynamic rendering in the process of supply chain data display, interface call and external sharing. It does not tamper with the original data and generates a de-identified data copy, while establishing a traceability relationship between the original data and the de-identified copy. The iterative optimization module is used to collect feedback on the de-identification effect, compliance audit results, and supply chain business adaptation data, and to adaptively iteratively optimize the identification model, de-identification rules, and rendering granularity to form a closed-loop management system.

[0014] Thirdly, this application provides a computer device, the device comprising: a memory and a processor, wherein the processor, when executing computer instructions stored in the memory, performs the method described above.

[0015] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described above.

[0016] In summary, this application extracts multimodal features and associates sensitive entities across modalities in raw multimodal supply chain data to output structured recognition results. Based on supply chain compliance requirements and business scenarios, it constructs desensitization strategies and generates execution instructions. In the data display and sharing phase, it employs layered dynamic rendering to perform desensitization processing and establish traceability links. Furthermore, it collects feedback data to adaptively iteratively optimize the model and rules. Through multimodal feature extraction and cross-modal association, dynamic strategy construction, layered rendering without tampering with raw data, establishment of traceability mechanisms, and closed-loop optimization, this application solves the problems of insufficient recognition accuracy, static desensitization strategies, data integrity corruption, lack of traceability, and optimization in existing technologies. It can effectively improve the recognition accuracy of sensitive information in multimodal supply chain data, achieve dynamically adaptable desensitization processing, protect the integrity of raw data, support compliant traceability, and adaptively optimize control effects. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application; Figure 2 This is a flowchart illustrating the first embodiment of the data desensitization and sensitive information management method based on multimodal recognition in this application; Figure 3 This is a structural block diagram of the first embodiment of the data desensitization and sensitive information management system based on multimodal recognition in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] Reference Figure 1 , Figure 1 This is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiments of this application.

[0020] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0021] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0022] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a data desensitization and sensitive information management program based on multimodal recognition.

[0023] exist Figure 1In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this application can be set in the computer device. The computer device calls the data desensitization and sensitive information management program based on multimodal recognition stored in the memory 1005 through the processor 1001, and executes the data desensitization and sensitive information management method based on multimodal recognition provided in the embodiments of this application.

[0024] Traditional data anonymization and sensitive information management technologies are generally designed for general scenarios and cannot adapt to the specific characteristics of supply chain scenarios and multimodal data. Their multimodal recognition capabilities are insufficient, resulting in high rates of missed and false detections of sensitive information; their anonymization strategies are rigid and cannot adapt to the diverse scenarios and access requirements of the supply chain; their anonymization methods are unreasonable, damaging data business value and lacking traceability; and they lack a closed-loop iterative optimization mechanism, leading to poor system adaptability.

[0025] To address this, this application provides a data desensitization and sensitive information management method based on multimodal recognition, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the data desensitization and sensitive information management method based on multimodal recognition in this application.

[0026] In this embodiment, the data desensitization and sensitive information management method based on multimodal recognition includes the following steps: Step S10: Extract multimodal features and associate cross-modal sensitive entities from the raw text, image, audio and video data in the supply chain scenario. Complete the identification of sensitive information through dynamic confidence calibration and output a structured identification result containing the global ID of sensitive information, cross-modal location mapping, and risk level.

[0027] Step S20: Based on supply chain compliance requirements, business scenario tags, and data permission matrix, construct a de-identification strategy through intelligent rule generation and conflict resolution, match the de-identification rules according to the structured identification results, and generate differentiated de-identification execution instructions.

[0028] Step S30: In the process of supply chain data display, interface call and external sharing, layered dynamic rendering is used to perform de-identification processing, without tampering with the original data and generating a de-identified data copy, while establishing a traceability relationship between the original data and the de-identified copy.

[0029] Step S40: Collect feedback on the de-identification effect, compliance audit results, and supply chain business adaptation data; adaptively iteratively optimize the identification model, de-identification rules, and rendering granularity to form a closed-loop management system.

[0030] For ease of understanding, the following explains some key terms in this embodiment: Multimodal recognition refers to the system's ability to comprehensively process and understand information from different data modalities (such as text, images, audio, and video). In supply chain scenarios, it is used to identify sensitive content in different forms of data.

[0031] Data anonymization refers to processing sensitive information in data without affecting the data business value, so that it cannot be directly identified or associated with a specific entity, thereby protecting privacy and trade secrets.

[0032] Sensitive information management refers to the management and control of the entire lifecycle of sensitive information, including its identification, desensitization, use, storage, and transmission, in order to ensure data compliance and security.

[0033] Multimodal feature extraction refers to extracting feature information that can represent the content and semantics of raw data in different modalities such as text, images, audio and video, so as to provide a foundation for subsequent recognition and analysis.

[0034] Cross-modal sensitive entity association refers to the association and integration of scattered information pointing to the same sensitive entity in different modal data. For example, it is possible to associate an ID number in text with an ID card image in a picture in order to form a comprehensive understanding of the sensitive entity.

[0035] Dynamic confidence calibration refers to adjusting and optimizing the confidence level in real time during the identification of sensitive information based on the reliability or accuracy of the identification results, in order to improve the accuracy of identification and reduce false alarms or missed alarms.

[0036] Structured identification results refer to representing the identified sensitive information in a predefined data structure with clear fields and formats, such as the type, location, and risk level of the sensitive information.

[0037] An anonymization strategy refers to a set of rules and methods developed based on specific compliance requirements, business scenarios, and data permissions to guide the anonymization of sensitive information.

[0038] Intelligent rule generation refers to the system's ability to automatically create or update de-identification rules based on preset conditions and a knowledge base, in order to adapt to ever-changing business needs and compliance standards.

[0039] Conflict resolution refers to the ability of a system to automatically resolve conflicts when there are contradictions or conflicts among multiple desensitization rules, based on preset priorities or strategies, thereby ensuring the consistency and effectiveness of desensitization processing.

[0040] Differentiated desensitization execution instructions refer to specific operation instructions that generate different desensitization algorithms, granularity, and timeliness requirements based on the specific type of sensitive information, risk level, access subject, and business scenario.

[0041] Layered dynamic rendering refers to the process of desensitizing sensitive information to different degrees in real time during data display or access, based on factors such as the access subject's permissions and business scenarios, and presenting the data in a way that does not destroy the original data.

[0042] Source tracing refers to establishing a link between the original data and the data copy after anonymization, so that the executor, time, rules, and source of the original data can be traced when needed.

[0043] Adaptive iterative optimization refers to the system's ability to continuously learn and adjust the identification model, desensitization rules, and rendering parameters based on feedback collected during actual operation (such as desensitization effect and compliance audit results) in order to continuously improve the system's performance and adaptability.

[0044] A closed-loop management system refers to a continuous improvement process that involves identification, strategy, execution, feedback, and optimization to ensure the long-term effective operation of a sensitive information management system.

[0045] In practical implementation, the first step is to process the multimodal raw data (text, images, audio, and video) within the supply chain scenario. Specifically, keyword extraction can be performed on text data, such as identifying specific words in contracts; image content classification can be performed on image data, such as distinguishing between photos of goods and scanned documents; and speech recognition can be performed on audio and video data, such as converting speech content into text. Based on this, preliminary associations can be made between potentially related sensitive information from different modalities, for example, matching text records with image files by filename or timestamp. Subsequently, a preset recognition threshold can be set to judge the initially identified sensitive information to complete the sensitive information identification process. This generates a record containing a sensitive information identifier, its approximate location in the raw data, and a preset risk level, serving as the structured identification result.

[0046] Secondly, based on supply chain compliance requirements, business scenario tags, and data access control matrices, a data masking strategy is constructed. For example, a series of data masking rules can be configured according to the company's internal data usage standards, the responsibilities of different business departments (such as the purchasing department and the warehousing department), and the data access permissions of different user roles (such as ordinary employees and department managers). These rules can specify which masking method should be adopted when a certain type of sensitive information is identified. When multiple rules may apply to the same sensitive information, they can be selected according to a preset priority order. Based on the structured identification results, the system can find and apply the corresponding masking rules, thereby generating specific masking execution instructions, such as instructing to replace a certain field or to mask a certain area.

[0047] Furthermore, data anonymization is performed during the display, API calls, and external sharing of supply chain data. Specifically, when data is requested for access, it can be processed in real time based on the visitor's identity and the level of anonymization they are authorized to use. For example, internal users can be shown data with a lower level of anonymization, while external partners can be shown data with a higher level of anonymization. This process does not directly modify the original data file but generates a temporary, anonymized copy of the data for the visitor to use. Simultaneously, the path to the original data file for each generated anonymized data copy can be recorded to trace its origin when needed.

[0048] Finally, feedback on the effectiveness of data masking, compliance audit results, and supply chain business adaptation data are collected to adaptively and iteratively optimize the identification model, masking rules, and rendering granularity. For example, user evaluations of the usability of masked data and audit reports from compliance departments on whether the masking process complies with regulations can be collected periodically. Based on this feedback, manual analysis is conducted to determine if the identification model has any missed or false alarms, whether the masking rules meet actual needs, and whether the rendering granularity meets the requirements of different scenarios. Based on the analysis results, the parameters of the identification model can be manually adjusted, the definitions of masking rules can be modified, or the rendering granularity corresponding to different access subjects can be adjusted to improve system performance and adaptability, thereby forming a continuous improvement control loop.

[0049] The method provided in this embodiment improves the accuracy of sensitive information identification and reduces false negatives and missed detections by performing cross-modal sensitive entity association and confidence calibration on multimodal data of the supply chain. Simultaneously, it dynamically generates desensitization strategies based on supply chain compliance requirements and business scenarios, achieving flexibility and scenario adaptability. Furthermore, it employs layered dynamic rendering to ensure data business value and compliance traceability capabilities without tampering with the original data. Through a closed-loop iterative optimization mechanism, it ensures the system's continuous adaptability to dynamic changes in the supply chain, meeting the needs of supply chain enterprises for multimodal data sensitive information management, compliant operation, and efficient utilization.

[0050] To address this, this embodiment proposes a data desensitization and sensitive information control method based on multimodal recognition. This method extracts features and associates sensitive entities with raw multimodal data (text, images, audio, and video) within a supply chain scenario to identify sensitive information. However, the data types in a supply chain scenario are extremely complex and diverse. Failure to clearly define and comprehensively cover these specific raw data sources may result in insufficient coverage or decreased accuracy in sensitive information identification, thereby affecting the overall effectiveness of desensitization and control.

[0051] To address the aforementioned issues, this embodiment further clarifies the specific composition of the multimodal raw data of the supply chain, which includes supply chain text data, supply chain image data, and supply chain audio and video data.

[0052] Specifically, the supply chain text data encompasses various document types commonly found in supply chain business processes, such as purchase contracts, order forms, customs declarations, supplier qualification documents, warehouse ledgers, customer receipt information, and settlement invoices. This text data is the core carrier of supply chain information flow, containing a wealth of trade secrets, personal privacy, financial data, and compliance information. For example, purchase contracts may contain commercially sensitive content such as price terms and supplier information; customer receipt information directly involves private data such as personal names, addresses, and contact information; and customs declarations involve detailed information on import and export goods and corporate compliance data. Identifying these specific types of text data ensures the comprehensiveness and accuracy of sensitive information identification.

[0053] The supply chain image data includes photos of goods labels, scanned copies of customs declarations, photos of transport vehicle license plates, screenshots of warehouse monitoring, scanned copies of supplier identification documents, and images of product packaging labels. This image data carries visual information within the supply chain. For example, photos of goods labels may contain sensitive logistical information such as batch number and destination; scanned copies of customs declarations are electronic versions of paper customs declarations, containing the same content as the text documents, but requiring image recognition technology for information extraction; and scanned copies of supplier identification documents directly contain personal identification information. By analyzing these specific types of image data, sensitive information can be obtained from non-textual dimensions, such as recognizing text information in images using Optical Character Recognition (OCR) technology, or identifying specific sensitive objects using image recognition technology, thus compensating for the limitations of single text recognition.

[0054] The supply chain audio and video data includes recordings of logistics communications, warehouse inspection videos, cross-border loading and unloading monitoring audio and video, and recordings of supply chain collaborative meetings. This audio and video data is a crucial form of dynamic interaction and on-site recording in supply chain operations. For example, logistics communication recordings may contain voice content such as customer complaints and dispatch instructions, involving customer privacy or operational details; warehouse inspection videos record the internal operations and goods placement within the warehouse, potentially involving trade secrets; and cross-border loading and unloading monitoring audio and video record the handover process of goods at critical stages, potentially involving customs inspection information. By using Automatic Speech Recognition (ASR) technology to convert the voice content into text and combining it with video content analysis, sensitive information at the text and visual levels can be extracted, supplementing the deficiencies of text and image data and achieving more comprehensive sensitive information identification.

[0055] By explicitly defining the original multimodal data of the supply chain as specific supply chain text data, supply chain image data, and supply chain audio and video data, this embodiment provides clear and specific input sources for multimodal feature extraction and cross-modal sensitive entity association. This explicit data type definition allows the identification model to be trained and optimized specifically, for example, targeting the text features of procurement contracts, the image features of goods label photos, and the voice features of logistics communication recordings, thereby significantly improving the accuracy and coverage of sensitive information identification. Given the complexity of the supply chain scenario, this specific data type division helps the system more comprehensively capture sensitive information scattered across different modalities and business carriers, avoiding missed identification problems caused by unclear data sources. Simultaneously, the identification of these specific data sources also provides a solid foundation for subsequent de-identification strategy construction and hierarchical dynamic rendering, ensuring the accuracy and effectiveness of de-identification processing, thereby improving the practicality and reliability of the entire data de-identification and sensitive information management method.

[0056] In supply chain scenarios, raw data presents multiple modalities such as text, images, audio, and video, and sensitive information may be scattered across different modalities. Simply extracting features from a single modality is insufficient to effectively capture the cross-modal relationships between sensitive entities, potentially leading to incomplete and inaccurate identification of sensitive information, thus affecting the formulation and implementation of subsequent de-identification strategies. To address this, this embodiment proposes a unified identification engine that performs the following processing: First, the multimodal data of the supply chain are preprocessed separately. Specifically, for supply chain text data, such as purchase contracts, order documents, and customs declarations, supply chain terminology normalization and named entity recognition are performed. Supply chain terminology normalization aims to map common industry abbreviations, aliases, or synonyms to standardized terminology. This is typically achieved by building and maintaining a dictionary and rule base containing supply chain domain knowledge to ensure consistency in recognition. Named entity recognition utilizes natural language processing technology to identify entities with specific meanings in the text, such as names of people, organizations, geographical locations, identification numbers, and contract numbers. This can be achieved through deep learning-based models (such as the Transformer architecture) or a combination of rules and dictionaries. For supply chain image data, such as photos of goods labels, scanned copies of customs declarations, and scanned copies of supplier identification documents, object detection and key area localization are performed. Object detection technologies (such as YOLO and Faster R-CNN) are used to identify specific objects within the images, such as document regions, license plates, and faces. Key area localization further refines the identification of sensitive information areas within these objects, such as names and ID numbers in scanned ID cards, or customer information fields in customs declarations. This typically involves combining optical character recognition (OCR) technology to extract text information from images. For supply chain audio and video data, such as logistics communication recordings, warehouse inspection videos, and cross-border loading and unloading monitoring audio and video, speech-to-text transcription and inter-frame analysis are performed. Automatic speech transcription (ASR) technology converts speech content into text for text-level sensitive word recognition and entity extraction. Inter-frame analysis analyzes the video stream frame-by-frame or keyframe, using image processing and computer vision techniques to identify sensitive information in the footage, such as ID cards, license plates, and faces appearing in the video, and can be combined with behavior recognition to determine the presence of abnormal behavior.

[0057] Building upon this foundation, a cross-modal association graph is constructed, assigning a global ID to the same sensitive entity. The system then performs location-based association and weight binding on customer ID numbers in text, supplier unified social credit codes, corresponding document information in images, and sensitive audio content in audio and video. After preprocessing each modality, the system identifies potential sensitive entities. For example, it identifies the ID number of "Zhang San" in text, identifies the name and ID number of "Zhang San" on the same ID card in images using OCR, and identifies dialogues mentioning "Zhang San" in audio and video through speech-to-text transcription. Constructing a cross-modal association graph involves using graph databases or knowledge graph technologies to treat these sensitive information points identified in different modalities that point to the same physical entity as nodes, establishing association edges through entity linking technology. The assignment of global IDs ensures that regardless of which modality the sensitive information appears in, as long as it points to the same entity, it has a unique identifier. Location association records the specific location of sensitive information in the original data (such as character offsets in text, pixel coordinates in images, timestamps and regions in audio and video), while weight binding assigns different weights to these associations based on the confidence level of recognition, the importance of the information, or the reliability of the modality. For example, identification information in an image may have a higher weight than mentions in speech transcription.

[0058] Furthermore, a confidence-weighted fusion and anomaly detection mechanism is employed to perform secondary calibration on the cross-modal recognition results, yielding a structured recognition result that includes sensitive type, location coordinates, and risk level. When the same sensitive entity is recognized in multiple modalities, each modal recognition result carries a confidence score. The confidence-weighted fusion mechanism comprehensively calculates these confidence scores based on preset weights (e.g., image OCR has a higher weight than speech transcription) or dynamically learned weights, resulting in a more reliable final confidence score. For example, if both text and image recognize the same ID number with high confidence, the final confidence score will be even higher. The anomaly detection mechanism performs logical consistency checks and anomaly pattern detection on the fused recognition results. For example, it checks whether the recognized ID number conforms to the national standard format, whether there are contradictions in the information recognized for the same entity in different modalities (e.g., inconsistent names), or whether there are discrepancies between the recognition results and known sensitive word databases or blacklists. For detected anomalies, the system can trigger a manual review process or reduce the confidence score of the recognition result. Through the above mechanism, the final structured recognition result not only includes the type of sensitive information (such as ID card number, bank card number, trade secrets, etc.) and its location coordinates in the original data (accurate to characters, pixels or time frames), but also includes a calibrated risk level, which can be comprehensively evaluated based on factors such as the type of sensitive information, confidence level, and importance of related entities.

[0059] Through the above technical solution, this embodiment can effectively address the complexity of multimodal data in supply chain scenarios. First, specialized preprocessing of different modalities transforms raw, heterogeneous data into a unified, analyzable format, laying the foundation for subsequent identification. Second, a cross-modal association graph is constructed and global IDs are assigned, achieving consistent identification and association of the same sensitive entity across different modalities, avoiding information fragmentation and identification omissions. Finally, a confidence-weighted fusion and anomaly verification mechanism performs secondary calibration on the identification results, significantly improving the accuracy and robustness of sensitive information identification and effectively reducing false positives and false negatives. This allows subsequent de-identification strategies to be based on more accurate and comprehensive sensitive information identification results, thereby ensuring the effectiveness and compliance of de-identification processing and providing a solid foundation for traceability between the original data and the de-identified copy.

[0060] In some of the above implementations, a data anonymization and sensitive information management method based on multimodal recognition is proposed. This method identifies sensitive information in the raw multimodal data of the supply chain and constructs anonymization strategies based on compliance requirements, business scenarios, and data permissions, thereby performing layered dynamic rendering anonymization processing. However, in practical applications, supply chain scenarios are complex and varied, involving multiple compliance requirements and business roles. How to efficiently and accurately construct anonymization strategies adapted to different scenarios and effectively resolve potential conflicts between strategies to ensure the compliance, effectiveness, and business availability of anonymization is a problem that urgently needs to be solved.

[0061] To address this, this embodiment further proposes a dynamic strategy engine, which constructs a strategy system based on supply chain scenarios. This dynamic strategy engine aims to provide an intelligent strategy management core. Its core function is to dynamically generate, adjust, and optimize data anonymization strategies according to the specific needs of supply chain operations and the constantly changing external environment, ensuring the accuracy and adaptability of the anonymization process.

[0062] Specifically, this dynamic strategy engine first establishes a supply chain compliance knowledge base, integrating various compliance requirements such as customs regulations, cross-border data compliance requirements, personal information protection regulations, and supply chain trade secret protection rules. This knowledge base is a structured collection of information, containing various laws, regulations, industry standards, and internal company rules related to supply chain data processing. It provides authoritative and comprehensive evidence for formulating de-identification strategies, ensuring that all de-identification operations comply with legal and regulatory requirements. For example, this knowledge base can be constructed and managed using technologies such as ontology, knowledge graphs, or rule engines to facilitate efficient querying and reasoning by the strategy engine. Based on this, the engine can automatically generate de-identification rules according to five business scenarios: procurement management, warehousing and logistics, cross-border customs clearance, supplier collaboration, and order fulfillment, combined with data permission matrices for administrators, operators, partners, and third-party regulators. This mechanism divides complex supply chain operations into specific scenarios and considers the data access permissions of different roles. The data permission matrix defines the access and operation permissions of different roles for various types of data in specific business scenarios. By combining business scenarios, role permissions, and requirements in the compliance knowledge base, the policy engine can intelligently derive de-identification rules for specific data, specific scenarios, and specific roles, significantly improving the efficiency and accuracy of rule formulation and reducing the complexity and potential errors of manual configuration.

[0063] Furthermore, to address potential conflicts between multiple rules, this embodiment employs a rule conflict resolution mechanism. This mechanism resolves rule conflicts based on sensitivity level, compliance priority, and business availability, ultimately generating a de-identification execution instruction that includes the de-identification algorithm, de-identification granularity, and execution timeliness. In complex supply chain environments, different compliance requirements, business scenarios, and permission settings may lead to multiple de-identification rules simultaneously applying to the same data, and these rules may contradict each other. The rule conflict resolution mechanism resolves these conflicts through preset priorities (e.g., compliance priority is higher than business availability, and rules with higher sensitivity levels take precedence), ensuring that the final generated de-identification execution instruction is unique, explicit, and optimal. For example, one rule might require complete de-identification of a field to protect personal privacy, while another rule might require retaining some information to meet business analysis needs. The conflict resolution mechanism determines the final de-identification method based on the preset priorities. The final generated de-identification execution instruction includes not only the specific de-identification algorithm (e.g., masking, generalization, encryption), de-identification granularity (e.g., field-level, character-level, pixel-level), but also execution timeliness (e.g., real-time de-identification, timed de-identification), thereby guiding the subsequent layered dynamic rendering process.

[0064] Through the above technical solution, this embodiment effectively addresses the challenge of efficiently and accurately constructing de-identification strategies adapted to different scenarios in complex supply chain environments, and effectively resolves potential conflicts between strategies. The dynamic strategy engine, by establishing a comprehensive supply chain compliance knowledge base and combining it with specific business scenarios and data permission matrices, achieves automated generation of de-identification rules, significantly improving the efficiency and accuracy of strategy formulation. Simultaneously, a rule conflict resolution mechanism is introduced, based on sensitivity level, compliance priority, and business availability, ensuring that a unique, clear, and optimal de-identification execution instruction is generated under the influence of multiple rules, avoiding improper de-identification or business interruption due to rule conflicts. This allows de-identification processing to better adapt to the dynamic changes and diverse needs of the supply chain, maximizing business continuity and data availability while ensuring data compliance and security, thereby enhancing the intelligence and robustness of the entire data de-identification and sensitive information management system.

[0065] In the stages of supply chain data display, interface calls, and external sharing, although layered dynamic rendering is used to perform de-identification processing and generate de-identified data copies to prevent tampering with the original data, the lack of refined de-identification methods for different data modalities and an effective traceability mechanism for the de-identification process may lead to inconsistent de-identification effects, increased risk of sensitive information leakage, and difficulty in meeting strict compliance audit requirements. To address this, this embodiment further proposes a specific implementation method for layered dynamic rendering, which includes: locating sensitive areas in multimodal supply chain data based on de-identification execution instructions and cross-modal location mapping; employing mask replacement, field generalization, and partial retention processing for text-based supply chain data; employing partial pixelation and key area occlusion processing for image-based data; and employing voice de-identification and image blurring processing for audio and video-based data; assigning a unique traceability identifier to each set of de-identified copies, which is associated with the original data storage address, de-identification rule parameters, access subject, and operation time, for use in supply chain compliance auditing and data traceability.

[0066] Specifically, layered dynamic rendering refers to applying different levels of data anonymization based on different access subjects, business scenarios, and data sensitivity levels, and rendering the data in real time when it is accessed or displayed, rather than permanently modifying the original data. This approach ensures the integrity of the original data while providing flexible data protection capabilities. When performing anonymization, sensitive areas in the multimodal data of the supply chain are first located based on the anonymization execution instructions and cross-modal location mapping. The anonymization execution instructions are specific operational commands generated by matching the anonymization rules with the sensitive information identification results after constructing an anonymization strategy based on supply chain compliance requirements, business scenario tags, and data permission matrices through intelligent rule generation and conflict resolution. These instructions specify the anonymization algorithm to be used, the granularity of anonymization, and the timeliness of execution, serving as the core basis for guiding subsequent anonymization operations. Cross-modal location mapping is one of the outputs of the sensitive information identification stage. It accurately records the specific location information of each sensitive entity in the original multimodal data, such as text, images, audio, and video. For example, for text data, it may include character start and end indices; for image data, it may include pixel coordinates or bounding boxes; and for audio and video data, it may include timestamps and screen regions. Locating sensitive areas refers to the system accurately identifying and delineating specific data segments or regions that need to be de-identified in the original multimodal data, based on the de-identification range and method specified in the de-identification execution instruction and combined with the precise coordinate information provided by cross-modal location mapping. This process ensures the accuracy of the de-identification operation, avoiding accidental damage to non-sensitive information or omission of sensitive information.

[0067] After locating sensitive areas, differentiated desensitization methods are employed for data of different modalities. For text-based supply chain data, mask replacement, field generalization, or partial retention can be used. Mask replacement is a common text desensitization technique that replaces some or all characters in sensitive information with preset placeholders (such as asterisks "*" or "X"). Field generalization refers to converting sensitive information from specific values ​​into broader categories or ranges to reduce its recognizability. Partial retention refers to retaining only some non-critical information within the sensitive data while hiding the rest. For image-based data, partial pixelation or key area occlusion can be used. Partial pixelation is an image desensitization technique that blurs pixels in specific sensitive areas of an image, making them indistinct and thus unrecognizable. Key area occlusion involves completely covering sensitive areas in an image with solid color blocks (such as black rectangles), making them completely invisible. For audio and video data, voice desensitization and image blurring can be used. Voice desensitization refers to processing sensitive speech content in audio and video data to make it unrecognizable. Examples include muting, pitch shifting, obfuscating sensitive speech segments, or replacing sensitive words with synthesized speech. Image blurring refers to blurring specific frames or time periods within audio and video data to hide sensitive visual information appearing in the video.

[0068] To ensure the traceability and compliance of the de-identification process, this embodiment assigns a unique traceability identifier to each set of de-identified copies. The unique traceability identifier is a unique identification code assigned to each generated de-identified data copy, serving as the basis for subsequent traceability and auditing. This traceability identifier is associated with the original data storage address, de-identification rule parameters, access subject, and operation time. Specifically, the original data storage address points to the location of the un-de-identified original data in the storage system; the de-identification rule parameters record the specific de-identification strategy and algorithm configuration used to generate the copy; the access subject indicates the user or system that requested and obtained the de-identified copy; and the operation time records the specific time point when the de-identified copy was generated or accessed. These associated information collectively constitute a complete audit chain for supply chain compliance auditing and data traceability. Supply chain compliance auditing refers to the review of whether the data processing process complies with relevant laws, regulations, industry standards, and internal corporate policies. Data traceability means that in the event of a data breach or compliance issue, the traceability identifier can be used to trace back the generation process of the de-identified copy, the original data, the rules used, and the accessor, thereby clarifying responsibility, analyzing the causes, and taking corrective measures.

[0069] Through the above technical solution, this embodiment enables refined and differentiated de-identification processing of multimodal data in the supply chain. Specifically, through the synergistic effect of de-identification execution instructions and cross-modal location mapping, the system can accurately locate sensitive areas in text, images, and audio / video data, and adopt the most suitable de-identification technology according to their modal characteristics, ensuring effective protection of sensitive information while maximizing the usability of non-sensitive information. Furthermore, by assigning a unique traceability identifier to each de-identified copy and associating it with the original data storage address, de-identification rule parameters, access subject, and operation time, a complete audit chain is constructed, greatly improving the transparency and traceability of data processing. This not only effectively solves the challenges of multimodal data de-identification in complex supply chain scenarios but also provides solid technical support for supply chain compliance auditing, ensuring the unity of data security and business compliance.

[0070] In some of the above implementations of this embodiment, a layered dynamic rendering approach is proposed to perform data anonymization processing and generate anonymized data copies during the supply chain data display, interface call, and external sharing stages. However, in actual supply chain business scenarios, different access subjects (such as internal managers, operators, and external partners) have significantly different needs for data sensitivity and reliance on business operations. If a uniform anonymization granularity is adopted, it may lead to insufficient information for internal personnel, affecting decision-making efficiency, or excessive information for external personnel, increasing the risk of data leakage. It is difficult to achieve the best balance between data security and business efficiency.

[0071] To address this, this embodiment further proposes dynamic rendering support for granular adaptive adjustment based on the access subject and scenario of the supply chain: granting low-level anonymization or even raw data preview permissions to internal management personnel; granting medium-level anonymized data to warehousing and logistics operators; and granting high-level strictly anonymized data to overseas partners and third-party service providers, achieving minimum necessary anonymization without disrupting the supply chain business logic.

[0072] Specifically, the dynamic rendering supports granular adaptive adjustment based on the access subject and scenario within the supply chain. This aims to flexibly adjust the level of data anonymization according to the data accessor's identity and their specific business context. This means the system can identify the user's role, department, and the business operation the data will be used for, and then dynamically select the most appropriate anonymization level. For example, management requiring data analysis and decision-making may need more detailed data; while external partners only needing to perform specific operations require only minimal data to meet their business needs. This adaptive adjustment mechanism is key to achieving refined data control, ensuring data security and availability in different contexts.

[0073] Granting internal management personnel access to preview low-level anonymized or even raw data is based on the consideration that they typically require comprehensive and accurate data for decision-making, monitoring, and analysis. Therefore, for these users, the system can be configured to provide a low level of anonymization, such as lightly processing only a very small amount of core sensitive information, or allowing them to preview raw data under strict access control and auditing mechanisms. This access configuration aims to maximize internal business efficiency and decision-making accuracy, while ensuring that data is not misused through internal management systems and technical means.

[0074] Meanwhile, providing medium-granularity anonymized data to warehousing and logistics operators is based on their need for sufficient data to perform specific operations such as receiving, storing, picking, shipping, and tracking goods in their daily work. For these users, the system provides medium-granularity anonymized data, retaining only necessary information directly related to the operation while anonymizing unnecessary or highly sensitive information. For example, it might display a customer's delivery address but hide their contact number, or display the quantity of goods but hide their specific value. This granularity aims to ensure operators can smoothly execute business processes while avoiding unnecessary data exposure.

[0075] Furthermore, the provision of highly anonymized data to overseas partners and third-party service providers is based on the fact that these entities typically engage in cross-organizational and cross-regional data interactions, posing relatively higher data security risks and potentially subject to data compliance regulations in different countries and regions. Therefore, the system implements high-level, rigorous anonymization for these access entities. This means that only the minimal data necessary to fulfill their business functions is provided, and this data undergoes deep anonymization processing, employing techniques such as generalization, encryption, and masking to maximize the protection of sensitive information. This stringent anonymization strategy aims to meet the compliance requirements of cross-border data transfer and third-party collaborations, and effectively reduce the risk of data leakage.

[0076] All the above-mentioned data masking processes adhere to the principle of achieving minimum necessary masking without disrupting the supply chain's business logic. This principle emphasizes that the impact on supply chain business processes and logic must be assessed before implementing any masking process. The purpose of masking is to protect sensitive information, but not at the expense of normal business operations. Therefore, when determining the granularity of masking, the system ensures that the provided masked data is sufficient to support the accessing entity in completing its authorized business functions, avoiding business interruptions, efficiency reductions, or decision-making errors due to excessive masking. This requires the system to understand the supply chain's business processes and data dependencies, thereby achieving the optimal balance between data security and business availability.

[0077] Through the above technical solution, this embodiment further achieves adaptive adjustment of data anonymization granularity based on layered dynamic rendering. Given the diverse access subjects and business scenarios within the supply chain, a uniform anonymization strategy would struggle to balance data security and business efficiency. This embodiment identifies different supply chain access subjects (such as internal management personnel, warehousing and logistics operators, overseas partners, and third-party service providers) and dynamically adjusts the granularity of anonymization based on their respective business scenarios. For example, internal management personnel can obtain low-level anonymization or even raw data preview permissions to support their comprehensive decision-making; warehousing and logistics operators receive medium-granularity anonymized data to meet their daily operational needs; while overseas partners and third-party service providers are assigned high-level, strictly anonymized data to maximize data security and meet compliance requirements. This refined, on-demand anonymization strategy ensures that minimum necessary anonymization is achieved without disrupting the supply chain business logic, effectively balancing data security and business efficiency, reducing the risk of data leakage, and improving the flexibility and compliance of supply chain data management.

[0078] In some of the embodiments described above, a data desensitization and sensitive information management method based on multimodal recognition is proposed. This method effectively manages sensitive information by extracting features from raw multimodal data of the supply chain, identifying sensitive information, constructing desensitization strategies, and performing hierarchical dynamic rendering. However, in practical applications, supply chain scenarios are complex and ever-changing, data types are constantly updated, and compliance requirements are continuously evolving. If the identification model, desensitization rules, and rendering granularity cannot adapt to these changes in a timely manner, it may lead to a decrease in the accuracy of sensitive information identification, poor desensitization effects, or damage to business availability, making it difficult to form a continuous and efficient sensitive information management capability.

[0079] To address this, this embodiment further proposes a closed-loop iterative optimization mechanism, which includes: collecting false detections, missed detections, business availability scores, and compliance audit results in the supply chain scenario; identifying and addressing rule defects through error analysis; optimizing the multimodal recognition model using incremental training and transfer learning; updating the supply chain compliance rule base and desensitization strategy; and adjusting the layered rendering granularity parameters; and iteratively updating the optimized model and rules to adapt to changes in supply chain data types, cross-border compliance changes, and business scenario expansion needs.

[0080] Specifically, in the closed-loop iterative optimization process, the first step is to collect false positives, false negatives, business availability scores, and compliance audit results within the supply chain scenario. False positives refer to situations where non-sensitive information is incorrectly identified as sensitive information, while false negatives refer to situations where sensitive information is not identified. Business availability scores can be quantitatively assessed using indicators such as user experience with the anonymized data and the smoothness of business processes. Compliance audit results refer to compliance assessment reports from external or internal audit institutions regarding the data anonymization and control processes. This data can be collected through system logs, user feedback interfaces, importing manual audit reports, or automated compliance scanning tools.

[0081] Subsequently, error analysis is used to pinpoint identification and rule deficiencies. Based on the collected false positives, false negatives, business availability scores, and compliance audit results, the system conducts in-depth analysis of this data to identify problems in the multimodal identification model and desensitization rules. Error analysis can employ statistical methods, such as classifying and clustering false positives and false negatives to discover common characteristics; and performing root cause analysis on scenarios with low business availability scores. Identification deficiencies may involve insufficient model recognition capabilities for specific data types or newly emerging sensitive entities; rule deficiencies may involve desensitization rules that are too strict or too lenient, or that fail to cover new compliance requirements.

[0082] Building upon this foundation, incremental training and transfer learning are employed to optimize the multimodal recognition model. Addressing the shortcomings of the existing recognition model, this embodiment utilizes techniques such as incremental training and transfer learning for optimization. Incremental training refers to using newly collected, manually labeled, or confirmed false positives and false negatives as incremental data to continuously train the existing recognition model in small batches. This updates the model's weights and parameters, enabling it to better identify sensitive information that was previously mishandled. Transfer learning, on the other hand, utilizes a model already trained in other related fields as a base model when significant changes occur in the supply chain scenario or when entirely new data types are introduced. This model is then fine-tuned using a small amount of data from the supply chain scenario, allowing for rapid adaptation to the new scenario and avoiding the enormous cost and time of retraining from scratch.

[0083] Simultaneously, update the supply chain compliance rule base and data masking strategies. Based on error analysis results and new compliance audit requirements, manually or semi-automatically modify, add, or delete rules in the supply chain compliance rule base. For example, if it is found that the data masking granularity of a certain field is too coarse, causing business disruptions, the data masking algorithm or granularity for that field can be adjusted; if new cross-border data compliance requirements arise, corresponding data masking strategies need to be added. Updates to the data masking strategies should be synchronized with the supply chain compliance knowledge base to ensure that they always meet the latest compliance requirements and business needs.

[0084] In addition, the granularity parameters for layered rendering were adjusted. Based on business availability scores and user feedback, the granularity of data display was finely adjusted for different access subjects (such as internal management personnel, warehouse operators, and overseas partners) and different business scenarios. For example, if it was found that the anonymization granularity for warehouse operators was too high, preventing them from completing picking tasks, the anonymization granularity could be appropriately reduced, allowing them to view some key information. These parameter adjustments can be made manually or assisted by an intelligent recommendation system based on business availability indicators to balance data security and business availability.

[0085] Ultimately, the optimized model and rules will be iteratively updated to adapt to changes in supply chain data types, cross-border compliance, and business scenario expansion needs. The optimized identification model, compliance rule base, and data masking strategy need to be deployed to the production environment, replacing the old version. The system should have flexible configuration management and version control capabilities to ensure a smooth transition during the update process. This iterative update mechanism enables the entire data masking and sensitive information management method to continuously adapt to new data formats, new compliance requirements, and business process expansions that constantly emerge in the supply chain.

[0086] Through the above technical solution, this embodiment constructs a continuously optimized closed-loop mechanism. By collecting actual operational data in the supply chain scenario, including false positives, false negatives, business availability scores, and compliance audit results, and conducting in-depth error analysis, defects in the multimodal recognition model and desensitization rules can be accurately located. Based on this, advanced technologies such as incremental training and transfer learning are used to optimize the recognition model, enabling it to quickly adapt to new data patterns and sensitive entities. Simultaneously, the supply chain compliance rule base and desensitization strategies are updated in a timely manner to ensure that desensitization processing always meets the latest compliance requirements and business needs. Furthermore, by adjusting the granularity parameters of layered rendering, an optimal balance between data security and business availability is achieved. This iterative update mechanism allows the entire data desensitization and sensitive information management method to continuously adapt to updates in supply chain data types, changes in cross-border compliance, and the expansion needs of business scenarios. This significantly improves the accuracy of sensitive information identification, the effectiveness of desensitization strategies, and the overall robustness and adaptability of the system, thereby ensuring the long-term efficiency and compliance of supply chain sensitive information management and avoiding system performance degradation due to environmental changes.

[0087] This method extracts multimodal features and associates sensitive entities across modalities from raw supply chain data to output structured recognition results. Based on supply chain compliance requirements and business scenarios, it constructs desensitization strategies and generates execution instructions. In the data display and sharing phase, it employs layered dynamic rendering to perform desensitization processing and establish traceability links. It also collects feedback data for adaptive iterative optimization of the model and rules. Through multimodal feature extraction and cross-modal association, dynamic strategy construction, layered rendering without tampering with raw data, traceability mechanism establishment, and closed-loop optimization, this method solves the problems of insufficient recognition accuracy, static desensitization strategies, data integrity corruption, lack of traceability, and optimization in existing technologies. It effectively improves the recognition accuracy of sensitive information in multimodal supply chain data, achieves dynamically adaptable desensitization processing, protects the integrity of raw data, supports compliance traceability, and adaptively optimizes control effects. Furthermore, embodiments of this application also propose a computer-readable storage medium storing a program for data desensitization and sensitive information management based on multimodal recognition. When the program for data desensitization and sensitive information management based on multimodal recognition is executed by a processor, it implements the steps of the method for data desensitization and sensitive information management based on multimodal recognition as described above.

[0088] Reference Figure 3 , Figure 3 This is a structural block diagram of the first embodiment of the data desensitization and sensitive information management system based on multimodal recognition in this application.

[0089] like Figure 3 As shown in the embodiments of this application, the data desensitization and sensitive information management system based on multimodal recognition includes: The feature extraction module 10 is used to perform multimodal feature extraction and cross-modal sensitive entity association on multimodal raw data of text, images, audio and video in the supply chain scenario. It completes the identification of sensitive information through dynamic confidence calibration and outputs a structured identification result containing the global ID of sensitive information, cross-modal location mapping, and risk level. The rule matching module 20 is used to construct a desensitization strategy based on supply chain compliance requirements, business scenario tags and data permission matrix through intelligent rule generation and conflict resolution, match desensitization rules according to the structured recognition results, and generate differentiated desensitization execution instructions. The association establishment module 30 is used to perform de-identification processing by using layered dynamic rendering in the process of supply chain data display, interface call and external sharing. It does not tamper with the original data and generates a de-identified data copy, while establishing a traceability relationship between the original data and the de-identified copy. The iterative optimization module 40 is used to collect feedback on the de-identification effect, compliance audit results and supply chain business adaptation data, and to adaptively iteratively optimize the identification model, de-identification rules and rendering granularity to form a control closed loop.

[0090] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of this application. In specific applications, those skilled in the art can make settings as needed, and this application does not impose any restrictions on this.

[0091] This embodiment extracts multimodal features and associates cross-modal sensitive entities with raw multimodal data from the supply chain to output structured recognition results. Based on supply chain compliance requirements and business scenarios, it constructs a desensitization strategy and generates execution instructions. In the data display and sharing stage, it employs layered dynamic rendering to perform desensitization processing and establish traceability associations. It also collects feedback data to adaptively iteratively optimize the model and rules. Through multimodal feature extraction and cross-modal association, dynamic strategy construction, layered rendering without tampering with raw data, establishment of a traceability mechanism, and closed-loop optimization, it solves the problems of insufficient recognition accuracy, static desensitization strategies, data integrity corruption, lack of traceability, and optimization in existing technologies. It can effectively improve the recognition accuracy of sensitive information in multimodal supply chain data, achieve dynamically adaptable desensitization processing, protect the integrity of raw data, support compliant traceability, and adaptively optimize control effects.

[0092] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0093] In addition, for technical details not described in detail in this embodiment, please refer to the data desensitization and sensitive information control method based on multimodal recognition provided in any embodiment of this application, which will not be repeated here.

[0094] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0095] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application. The above are only preferred embodiments of this application and do not limit the patent scope of this application. All equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for data desensitization and sensitive information control based on multimodal recognition, characterized in that, include: Multimodal feature extraction and cross-modal sensitive entity association are performed on multimodal raw data of text, images, audio and video in supply chain scenarios. Sensitive information identification is completed through dynamic confidence calibration, and structured identification results containing global ID of sensitive information, cross-modal location mapping and risk level are output. Based on supply chain compliance requirements, business scenario tags, and data permission matrix, a de-identification strategy is constructed through intelligent rule generation and conflict resolution. The de-identification rules are matched according to the structured identification results to generate differentiated de-identification execution instructions. In the process of supply chain data display, interface call and external sharing, layered dynamic rendering is used to perform de-identification processing, without tampering with the original data and generating a de-identified data copy, while establishing a traceability relationship between the original data and the de-identified copy; By collecting feedback on the effectiveness of data masking, compliance audit results, and supply chain business adaptation data, we can adaptively iterate and optimize the identification model, masking rules, and rendering granularity to form a closed-loop management system.

2. The method according to claim 1, characterized in that, The multimodal raw data of the supply chain includes: supply chain text data, specifically purchase contracts, order documents, customs declarations, supplier qualification documents, warehouse ledgers, customer receiving information, and settlement invoice texts; Supply chain image data includes: photos of goods labels, scanned copies of customs declarations, photos of transport vehicle license plates, screenshots of warehouse surveillance footage, scanned copies of supplier ID documents, and images of goods packaging labels; Supply chain audio and video data includes: logistics communication recordings, warehouse inspection videos, cross-border loading and unloading monitoring audio and video, and supply chain collaboration meeting recordings.

3. The method according to claim 1, characterized in that, The unified recognition engine performs the following processing: The system preprocesses multimodal data from the supply chain, performs supply chain terminology normalization and named entity recognition on text data, performs target detection and key area localization on image data, and performs speech transcription and inter-frame parsing on audio and video data. Construct a cross-modal association graph, assign a global ID to the same sensitive entity, and perform location association and weight binding on customer ID numbers in text, supplier unified social credit codes, corresponding document information in images, and voice-sensitive content in audio and video. A confidence-weighted fusion and anomaly verification mechanism is adopted to perform secondary calibration on the cross-modal recognition results, resulting in a structured recognition result that includes sensitive type, location coordinates, and risk level.

4. The method according to claim 1, characterized in that, The dynamic strategy engine builds a strategy system based on supply chain scenarios: Establish a supply chain compliance knowledge base, integrating customs supervision regulations, cross-border data compliance requirements, personal information protection regulations, and supply chain trade secret protection rules; Based on five business scenarios—procurement management, warehousing and logistics, cross-border customs clearance, supplier collaboration, and order fulfillment—and combined with the data permission matrix of administrators, operators, partners, and third-party regulators, the system automatically generates de-identification rules. Through a rule conflict resolution mechanism, rule conflicts are resolved based on sensitivity level, compliance priority, and business availability, generating de-identification execution instructions that include de-identification algorithms, de-identification granularity, and execution time.

5. The method according to claim 1, characterized in that, Layered dynamic rendering specifically includes: Based on the de-identification execution instructions and cross-modal location mapping, sensitive areas in the multimodal data of the supply chain are located; For text-based supply chain data, mask replacement, field generalization, and partial retention are used; for image-based data, partial pixelation and key area occlusion are used; and for audio-visual data, voice desensitization and image blurring are used. Each set of de-identified copies is assigned a unique traceability identifier, which is associated with the original data storage address, de-identification rule parameters, access subject, and operation time, and is used for supply chain compliance auditing and data traceability.

6. The method according to claim 5, characterized in that, Dynamic rendering supports granular adaptive adjustment based on supply chain access subjects and scenarios: Grant internal management personnel access to preview low-sensitization or even raw data; Medium-granularity anonymized data is made available to warehouse and logistics personnel; We provide high-level, strictly anonymized data to overseas partners and third-party service providers, achieving minimal anonymization without disrupting the supply chain's business logic.

7. The method according to claim 1, characterized in that, Closed-loop iterative optimization includes: Collect false detections, missed detections, business availability scores, and compliance audit results in supply chain scenarios, and identify and locate rule defects through error analysis; Incremental training and transfer learning were used to optimize the multimodal recognition model, the supply chain compliance rule base and desensitization strategy were updated, and the layered rendering granularity parameters were adjusted. The optimized model and rules will be iteratively updated to adapt to changes in supply chain data types, cross-border compliance changes, and business scenario expansion needs.

8. A data desensitization and sensitive information management system based on multimodal recognition, characterized in that, include: The feature extraction module is used to extract multimodal features and associate cross-modal sensitive entities from multimodal raw data such as text, images, and audio / video in supply chain scenarios. It completes the identification of sensitive information through dynamic confidence calibration and outputs structured identification results containing global ID of sensitive information, cross-modal location mapping, and risk level. The rule matching module is used to construct a de-identification strategy based on supply chain compliance requirements, business scenario tags, and data permission matrix through intelligent rule generation and conflict resolution. It matches the de-identification rules according to the structured identification results and generates differentiated de-identification execution instructions. The association establishment module is used to perform de-identification processing through layered dynamic rendering in the process of supply chain data display, interface call and external sharing. It does not tamper with the original data and generates a de-identified data copy, while establishing a traceability relationship between the original data and the de-identified copy. The iterative optimization module is used to collect feedback on the de-identification effect, compliance audit results, and supply chain business adaptation data, and to adaptively iteratively optimize the identification model, de-identification rules, and rendering granularity to form a closed-loop management system.

9. A computer device, characterized in that, The device includes a memory and a processor, wherein the processor, when executing computer instructions stored in the memory, performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images