Data de-sensitization method and device, electronic device and storage medium

Abstract

The application discloses a data desensitization method and device, an electronic device and a storage medium, and relates to the field of data processing or other related fields, wherein the method comprises the following steps: sensitive object extraction is performed on a business data set to obtain a candidate sensitive object set; cross-modal recognition is performed on the candidate sensitive object set to obtain a global sensitive object, and a fusion confidence of the global sensitive object is calculated; the business intention and business audience of a target business are recognized, the authorized access situation of multi-modal business data is determined, a data desensitization strategy is generated, and the multi-modal business data corresponding to the target business is desensitized based on the data desensitization strategy; and plaintext residual checking is performed on the desensitized data, and desensitized data is obtained after the checking is passed. The application solves the technical problem that in the related art, a one-size-fits-all method based on static rules is used for data desensitization, and it is difficult to realize dual protection of data security and information availability.

Description

Technical Field

[0001] This invention relates to the field of data processing or other related technical fields, and more specifically, to a data desensitization method and apparatus, electronic device and storage medium. Background Technology

[0002] With the rapid development of digital systems such as intelligent customer service, intelligent Q&A, complaint handling, work order processing, and intelligent review, the data uploaded by users during business interactions has gradually exhibited significant multimodal characteristics, including various data formats such as text, images, and structured data. This type of multimodal business data typically contains a large amount of sensitive personal and business-sensitive information, such as names, mobile phone numbers, ID numbers, QR codes, barcodes, and other key information that can identify individuals or business credentials. If plaintext is retained during subsequent large-scale model inference, log recording, data storage, work order processing, manual review, and API calls, it can easily lead to data leakage risks.

[0003] In related technologies, data anonymization processing focuses on anonymizing text data and adopts a one-size-fits-all approach based on static rules. This approach is prone to blind spots or missing information, which can affect subsequent business operations. Furthermore, it is difficult to ensure data security while also considering the availability of information in different business scenarios.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a data anonymization method, apparatus, electronic device, and storage medium to at least solve the technical problem in related technologies where a one-size-fits-all approach based on static rules is used for data anonymization, making it difficult to achieve dual protection of data security and information availability.

[0006] According to one aspect of the present invention, a data anonymization method is provided, comprising: receiving multimodal business data corresponding to a target business, and preprocessing the multimodal business data to obtain a business data set; extracting sensitive objects from the business data set to obtain a candidate sensitive object set; performing cross-modal recognition on the candidate sensitive object set, merging the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain a global sensitive object, and calculating the fusion confidence of the global sensitive object; identifying the business intent and business audience of the target business, and determining the authorized access status of the multimodal business data, generating a data anonymization strategy based on the business intent, the business audience, the authorized access status, and the fusion confidence, anonymizing the multimodal business data corresponding to the target business based on the data anonymization strategy to obtain anonymized multimodal business data; performing residual verification on the anonymized multimodal business data to obtain a verification result, and obtaining the final anonymized data based on the verification result, wherein, if the verification result indicates that the verification passed, the final anonymized data is obtained, or if the verification result indicates that the verification failed, iterative rework processing is performed.

[0007] According to another aspect of the present invention, a data desensitization apparatus is also provided, comprising: a receiving unit, configured to receive multimodal service data corresponding to a target service, and preprocess the multimodal service data to obtain a service data set; an extraction unit, configured to extract sensitive objects from the service data set to obtain a candidate sensitive object set; a merging unit, configured to perform cross-modal recognition on the candidate sensitive object set, merge the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain a global sensitive object, and calculate the fusion confidence of the global sensitive object; and a desensitization unit, configured to identify the service intent and service recipients of the target service. The system identifies the authorized access status of the multimodal business data, generates a data anonymization strategy based on the business intent, the business audience, the authorized access status, and the fusion confidence level, and anonymizes the multimodal business data corresponding to the target business based on the data anonymization strategy to obtain anonymized multimodal business data; a verification unit performs residual verification on the anonymized multimodal business data to obtain a verification result, and obtains the final anonymized data based on the verification result, wherein, if the verification result indicates that the verification passed, the final anonymized data is obtained, or if the verification result indicates that the verification failed, iterative rework processing is performed.

[0008] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described data desensitization methods.

[0009] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described data desensitization methods.

[0010] In this application, the following steps are performed: receiving multimodal business data corresponding to the target business, preprocessing the multimodal business data to obtain a business data set, extracting sensitive objects from the business data set to obtain a candidate sensitive object set, performing cross-modal recognition on the candidate sensitive object set, merging the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain global sensitive objects, calculating the fusion confidence of global sensitive objects, identifying the business intent and business audience of the target business, and determining the authorized access status of the multimodal business data, generating a data desensitization strategy based on the business intent, business audience, authorized access status, and fusion confidence, and desensitizing the multimodal business data corresponding to the target business based on the data desensitization strategy; performing residual verification on the desensitized multimodal business data to obtain verification results, and obtaining the final desensitized data based on the verification results. Specifically, if the verification result indicates that the verification passed, the final desensitized data is obtained; otherwise, if the verification result indicates that the verification failed, iterative rework processing is performed.

[0011] In this application, sensitive objects are extracted from different modalities, and cross-modal recognition is used to merge candidate sensitive objects scattered across different modalities to generate global sensitive objects. This ensures that the same sensitive entity is treated as a whole in all modalities, achieving synchronous desensitization of the same object across different modalities. Furthermore, business intent, business audience, and authorization status are introduced as key dimensions for generating the desensitization strategy. This allows for the dynamic retention of the minimum necessary information based on task intent, satisfying the data requirements of subsequent business processing while minimizing the exposure of sensitive information. This balances data security and information availability. Iterative residual verification is performed on the desensitized multimodal business data to ensure the security of the output results and improve the accuracy of the desensitization results. This solves the technical problem in related technologies where a one-size-fits-all approach based on static rules for data desensitization is insufficient to achieve dual protection of data security and information availability. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0013] Figure 1 This is a flowchart of an optional data desensitization method according to an embodiment of the present invention;

[0014] Figure 2 This is a schematic diagram of an optional data desensitization process according to an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of an optional data desensitization device according to an embodiment of the present invention;

[0016] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) that performs a data desensitization method according to an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0020] Passenger Name Record (PNR) is a core database record used to store passenger reservation information.

[0021] It should be noted that the data desensitization method and apparatus in this application can be used in the field of data processing for desensitizing multimodal data, and can also be used in any field other than data processing for desensitizing multimodal data. This application does not limit the application field of the data desensitization method and apparatus.

[0022] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of such data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0023] It should be noted that in this application, when collecting and analyzing customer information, users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision-making results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0024] The following embodiments of the present invention can be applied to various data anonymization systems / applications / devices. The present invention ensures that the same sensitive information is simultaneously anonymized across all carriers, including images, text, and structured data, through global sensitive object modeling, minimizing anonymization blind spots. By employing dynamic strategies based on business intent and audience, the minimum necessary information is retained, satisfying the data requirements of subsequent business processing while minimizing the exposure of sensitive information, thus balancing data security and information availability.

[0025] In addition, the present invention introduces a fusion confidence level and residual verification mechanism, which can intelligently identify uncertainties and ensure the accuracy of the desensitization results through automatic rework or manual review, thereby reducing the security risks or business interruption probability caused by automated processing errors.

[0026] The present invention will now be described in detail with reference to various embodiments.

[0027] Example 1

[0028] According to an embodiment of the present invention, an embodiment of a data desensitization method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] Figure 1 This is a flowchart of an optional data anonymization method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0030] The implementation subject of this invention is a data anonymization system, which can be applied to scenarios with high requirements for data security, business availability and audit traceability, such as air passenger services, intelligent customer service, complaint handling and work order circulation.

[0031] Step S101: Receive the multimodal service data corresponding to the target service, and preprocess the multimodal service data to obtain a service data set.

[0032] In step S101 above, a service request initiated from a user terminal or upstream business system is received, and multimodal service data corresponding to the target service is obtained. This multimodal service data may include various forms of data such as natural language text, various image files, optical character recognition results, QR code or barcode area information, and structured business fields. Subsequently, the received multimodal service data undergoes standardized preprocessing to eliminate noise, format differences, and unstructured interference in the original data, thereby obtaining a unified format set of business data, laying the data foundation for subsequent sensitive information extraction.

[0033] Through the above steps, heterogeneous raw data is transformed into a standardized data format that is machine-readable and easy to process, ensuring that subsequent processing can efficiently and accurately perform unified analysis on various data sources.

[0034] An optional embodiment includes the following steps for receiving multimodal service data corresponding to a target service and preprocessing the multimodal service data: determining the target service to be desensitized and receiving the multimodal service data corresponding to the target service, wherein the type of multimodal service data includes at least one of the following: text data, image data, optical character recognition results, QR code region detection results, and structured service fields; performing a cleaning operation on the text data to remove irrelevant characters, duplicate spaces, and abnormal line breaks to generate cleaned text modal data; performing orientation correction, sharpness detection, and noise reduction processing on the image data to obtain a preprocessed image, and performing layout region recognition on the preprocessed image to obtain image modal data containing layout coordinates; and integrating the data from the optical character recognition results, QR code region detection results, text modal data, and image modal data to form a service data set.

[0035] Furthermore, the preprocessing steps for multimodal business data include: first, identifying the target business that requires sensitive information processing and needs to be de-identified. This target business could be intelligent customer service Q&A, complaint handling, or work order processing, etc.; then, receiving the multimodal business data associated with this target business. This multimodal business data comes from diverse sources and includes at least one of the following types: text data, image data, optical character recognition results, QR code area detection results, and structured business fields, aiming to comprehensively cover all types of information carriers that may appear in user interactions.

[0036] The system performs data cleaning on the received text data to remove noise interference such as irrelevant characters, repeated spaces, and abnormal line breaks, thereby generating cleaned text modal data with standardized format, providing accurate basic text for subsequent natural language processing.

[0037] For image data, orientation correction is first performed to correct image tilt, followed by sharpness detection to assess image quality and denoising to obtain a clear and clean preprocessed image. On this basis, the system further performs layout region recognition on the preprocessed image. By locating key information regions in the image, image modal data containing precise layout coordinates is generated, thereby realizing the correspondence between image content and spatial location.

[0038] High-risk areas such as QR code area detection results and optical character recognition results are detected, and code area coordinates, area type and recognition confidence are generated. For structured fields, field name normalization, field value standardization, field path marking and source marking are performed.

[0039] After independently processing various modal data, the data from optical character recognition, QR code region detection, cleaned text modal data, and processed image modal data are deeply integrated to construct a unified, standardized, and structured business data set. This transforms the heterogeneous raw multimodal data into a standardized data format common to the system, eliminating format differences and noise interference between data sources. This ensures that subsequent steps can accurately extract sensitive objects and perform cross-modal associations based on a unified data model, laying a solid data foundation for subsequent consistent desensitization.

[0040] Step S102: Extract sensitive objects from the business data set to obtain a candidate sensitive object set.

[0041] In step S102 above, based on a pre-configured rule matching library, named entity recognition model, business dictionary, layout analysis algorithm, and image target detection technology, a deep scan and analysis is performed on each data unit in the business data set to extract sensitive objects, such as names, ID numbers, contact information, and ticket numbers, to the maximum extent possible. This results in a candidate sensitive object set.

[0042] Through the above steps, we can comprehensively and accurately discover potentially sensitive information in business data, providing a complete candidate object foundation for subsequent cross-modal association and unified modeling, and ensuring that no sensitive information is missed.

[0043] An optional embodiment includes the following steps for extracting sensitive objects from a business data set to obtain a candidate sensitive object set: Based on a preset business dictionary and entity recognition model, performing entity extraction on text modal data and optical character recognition results in the business data set to obtain text-level candidate sensitive objects; using an image detection model to perform visual feature extraction on image modal data in the business data set to obtain image-level candidate sensitive objects; performing normalization processing on structured business fields in the business data set using a preset standard field table to obtain structured-level candidate sensitive objects; merging text-level candidate sensitive objects, image-level candidate sensitive objects, structured-level candidate sensitive objects, and QR code region detection results to obtain a candidate sensitive object set; and configuring field type, original field value, field normalized value, source modality, location information, and confidence level for each candidate sensitive object in the candidate sensitive object set.

[0044] Furthermore, the system extracts sensitive entities from data of different modalities. First, for the text modal data and optical character recognition results contained in the business data set, it calls a preset business dictionary and combines it with an entity recognition model for in-depth analysis. The business dictionary contains a sensitive word library in a specific domain, such as common sensitive entities like names, ID numbers, and ticket numbers. The entity recognition model uses natural language processing technology to understand the contextual semantics. The two work together to accurately locate sensitive segments in the text, thereby generating text-level candidate sensitive objects.

[0045] Meanwhile, the system applies an image detection model to the image modal data in the business data set. Through visual feature extraction techniques such as convolutional neural networks, it identifies key visual elements in the images, such as face regions, document layouts, names, and document numbers, thereby obtaining image-level candidate sensitive objects and ensuring that sensitive information at the visual level is not overlooked.

[0046] In addition, the system maps and normalizes the structured business fields in the business data set based on the preset standard field table, and converts structured data from different sources or formats into a standard format to generate structured candidate sensitive objects, thereby eliminating the processing obstacles caused by data format differences.

[0047] Subsequently, the system merges the generated text-level candidate sensitive objects, image-level candidate sensitive objects, structured-level candidate sensitive objects, and QR code region detection results to form a unified set of candidate sensitive objects.

[0048] In addition, the system configures detailed attribute information for each candidate sensitive object in the sensitive object set, including the field type that identifies its sensitivity type, the original value of the field that records the original identification content, the standardized value of the field after format unification, the source modality that indicates the source of the data, the location information that locates its specific coordinates or path in the original data, and the confidence level that reflects the reliability of the identification.

[0049] By employing multi-dimensional extraction methods, we comprehensively cover sensitive information in text, images, and structured data, and establish standardized attribute descriptions for each sensitive object, providing a structured and comparable data foundation for subsequent cross-modal recognition and object classification.

[0050] Step S103: Perform cross-modal recognition on the candidate sensitive object set, merge the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain the global sensitive objects, and calculate the fusion confidence of the global sensitive objects.

[0051] In step S103 above, cross-modal recognition is initiated for candidate sensitive objects in the candidate sensitive object set to identify the same sensitive entity existing in different modalities, thereby addressing the problem of the same sensitive entity being expressed in scattered and isolated forms across different data carriers. The system performs pairwise or multi-directional comparative analysis on each object in the candidate sensitive object set based on multiple dimensions such as field type matching, numerical similarity comparison, spatial proximity, and business context relevance, thereby identifying the same sensitive object in different modalities and merging them to obtain the global sensitive objects.

[0052] Based on this globally sensitive object, the system further calculates the fusion confidence of each globally sensitive object. This fusion confidence comprehensively considers the recognition confidence of each source modality and the credibility weight of each modality's evidence, and obtains the overall reliability score through a weighted fusion algorithm.

[0053] By breaking down the data barriers between modalities through the above steps, unified modeling and association of cross-modal sensitive objects can be achieved. At the same time, by integrating confidence metrics to quantify the reliability of identification results, an objective basis is provided for the subsequent generation of confidence-based differentiated desensitization strategies, thereby improving processing efficiency while ensuring security.

[0054] An optional embodiment involves performing cross-modal recognition on a set of candidate sensitive objects, merging the expressions of candidate sensitive objects in different modalities into a unified expression to obtain global sensitive objects. The steps include: traversing the candidate sensitive objects in the set; filtering candidate sensitive objects with the same field type or existing business mapping relationships based on preset business mapping rules to construct initial sensitive object groups; calculating the similarity between the field normative values ​​of each candidate sensitive object within the same initial sensitive object group, and merging candidate sensitive objects with similarity higher than a preset similarity threshold into the same association cluster; checking the location information and source metadata of each candidate sensitive object in the association cluster, and retaining the association cluster as a valid association cluster if the location information and source metadata of the candidate sensitive objects meet preset conditions; performing spatial association verification and business association verification on the candidate sensitive objects in the valid association cluster; determining that the candidate sensitive objects in the valid association cluster point to the same entity if the spatial association verification and / or business association verification are passed, and merging the expressions of candidate sensitive objects pointing to the same entity in different modalities into a unified expression to obtain global sensitive objects.

[0055] Furthermore, in the process of merging the same candidate sensitive objects, the set of candidate sensitive objects is first traversed and analyzed. Based on the preset business mapping rules, candidate sensitive objects with the same field type or with a clear business mapping relationship are selected. For example, the ticket number in the airline ticket order is associated with the ticket number text in the boarding pass image, thereby constructing an initial sensitive object group containing the potential same entity.

[0056] Subsequently, the system performs in-depth calculations on the similarity between the standardized field values ​​of each candidate sensitive object within the same initial sensitive object group. By comparing the numerical content after format standardization, candidate sensitive objects with similarity higher than a preset similarity threshold are aggregated together to form preliminary association clusters, thereby eliminating identification fragmentation caused by format differences. Based on this, the system further examines the location information and source metadata of each candidate sensitive object in the association cluster. By verifying whether these objects are located on the same credential page, in the same session context, or in the same business request chain, if the location information and source metadata meet preset spatiotemporal consistency conditions, the association cluster is retained as a valid association cluster, thus filtering out erroneous associations caused by coincidence.

[0057] Based on the effective association clusters, the system performs spatial association verification and business association verification on candidate sensitive objects. Spatial association verification aims to confirm the proximity or inclusion relationship of objects of different modalities in physical space or on the data page. For example, it verifies whether the text recognized by optical character recognition is indeed located near the corresponding sensitive area in the image. Business association verification is used to confirm the logical coherence of these objects in the business process, such as verifying whether boarding pass information and check-in status information belong to the same itinerary. When a candidate sensitive object in the effective association cluster successfully passes the spatial association verification and / or business association verification, the system determines that these objects distributed in different modalities point to the same business entity. Then, it deeply integrates and merges the expression forms of candidate sensitive objects pointing to the same entity in different modalities to generate a global sensitive object with a unified identifier and a complete chain of evidence.

[0058] The above steps, through a multi-level and multi-dimensional verification mechanism, ensure the accuracy and robustness of cross-modal object merging, avoid erroneous merging, and thus perform highly reliable global sensitive object modeling.

[0059] In one optional embodiment, the step of calculating the fusion confidence of a global sensitive object includes: obtaining the confidence of each candidate sensitive object pointing to the same entity; obtaining the confidence weight value of the source modality of each candidate sensitive object pointing to the same entity; multiplying the confidence of each candidate sensitive object pointing to the same entity by the confidence weight value to obtain a weighted confidence; and calculating the complement of the product of all weighted confidences to obtain the fusion confidence of the global sensitive object.

[0060] Furthermore, the fusion confidence score is the core basis for generating subsequent desensitization strategies. After identifying the global sensitive objects, it is necessary to calculate their fusion confidence scores. Specifically, for each candidate sensitive object identified as pointing to the same entity, the confidence score in its original modality is obtained one by one. This confidence score reflects the accuracy of sensitive information extraction in a single modality. Subsequently, the system obtains the corresponding confidence weight value based on the source modality of each candidate sensitive object. The confidence weight value of the source modality is used to characterize the credibility of different data sources (such as high-definition images, low-resolution OCR results, etc.) in the overall evaluation. Typically, structured and high-quality images have higher weights. The system multiplies the original identification confidence score of each candidate sensitive object with the confidence weight value of its respective source modality to calculate the weighted confidence score reflecting the reliability of the specific evidence in the current source context. Based on this, the system calculates the complement of all weighted confidence score products to obtain the fusion confidence score of the global sensitive objects.

[0061] Furthermore, the formula for calculating the fusion confidence level is expressed as: Where C(O) represents the object-level fusion confidence of global sensitive object O; ci represents the confidence of the i-th candidate evidence (i.e., candidate sensitive object); wi represents the credibility weight of the source of the i-th candidate evidence; Tobj represents the object-level fusion confidence threshold, used to determine whether multiple candidate objects can be reliably merged into the same sensitive object.

[0062] By quantifying the overall reliability of multimodal evidence, an objective basis is provided for the generation of subsequent de-identification strategies. This ensures that high-confidence globally sensitive objects can trigger an efficient automated de-identification process, while low-confidence objects may trigger a more stringent review mechanism or a higher de-identification intensity, thereby optimizing processing efficiency while ensuring data security.

[0063] Step S104: Identify the business intent and audience of the target business, and determine the authorized access status of the multimodal business data. Generate a data anonymization strategy based on the business intent, audience, authorized access status, and fusion confidence level. Anonymize the multimodal business data corresponding to the target business based on the data anonymization strategy.

[0064] In step S104 above, the context of the current processing task is analyzed in depth to accurately identify the business intent of the target business, such as determining whether the user is currently inquiring about check-in status, handling baggage complaints, or filling in work orders. Simultaneously, the target audience for the anonymized data is identified, distinguishing whether the data is displayed to end users, used for large-scale model inference prompts, internal authorized work order processing, or audit log entries. Based on this, the system further verifies and determines the authorized access status of multimodal business data, assesses whether the current business request object has the permission to access specific sensitive information, and comprehensively judges the compliance of the access based on the sensitivity level and source attributes of the business data. Finally, a data anonymization strategy is generated based on the business intent, target audience, authorized access status, and fusion confidence level, and the multimodal data is anonymized accordingly.

[0065] Through the above steps, taking into account business intent, audience permissions, authorization status, and identification confidence, the system can generate a desensitization strategy that is highly adapted to the current business scenario. This not only ensures the security of sensitive information during its flow but also maximizes the retention of the minimum necessary information required for business processing, significantly improving the intelligence level and compliance of multimodal data processing.

[0066] In one optional embodiment, the steps of identifying the business intent and audience of the target service and determining the authorized access status of multimodal business data include: identifying the business intent of the target service based on the context information of the target service, wherein the business intent includes at least one of the following: check-in inquiry, baggage complaint, work order backfilling, audit log; determining the target audience of the multimodal business data output based on the business intent to obtain the business audience of the target service, wherein the target audience includes at least one of the following: user, internal business system, manual review interface; querying the authorization status of the current session based on the target audience to obtain the authorized access status of the multimodal business data.

[0067] Furthermore, the contextual information of the target business is first extracted and analyzed, including the natural language text input by the user, historical interaction records, the business scenario identifier of the current request, and related metadata. Through semantic analysis and rule matching technologies, the specific business intent of the current operation is accurately identified. Business intents are subdivided into several types, including but not limited to check-in inquiries, baggage complaints, work order backfilling, and audit log generation. Check-in inquiries aim to obtain flight status or boarding instructions, baggage complaints are used to handle claims for damaged or lost baggage, work order backfilling focuses on the structured entry of processing results into the system, and audit logs are used to record key operations to support subsequent compliance reviews. Clearly defining business intents provides contextual basis for subsequent data processing, and different business intents have significantly different requirements and sensitivity requirements for data fields.

[0068] Subsequently, based on the identified business intent and pre-defined business flow rules, the system determines the target audience for multimodal business data output, thus obtaining the target business audience. The target audience can encompass users, internal business systems, or manual review interfaces. For example, in a check-in consultation scenario, users require intuitive responses, large models may need anonymized text for reasoning, internal business systems require structured data updates, and manual review interfaces require complete evidence chains for verification. Determining the target audience helps differentiate data usage scenarios and risk exposure areas, as different audiences have different requirements for data integrity and security. For instance, input to large models must strictly avoid the leakage of sensitive plaintext, while internal authorization systems may retain some controlled information under specific conditions.

[0069] Finally, based on the identified target audience, the system queries the authorization status of the current session to obtain the authorized access information for multimodal business data. This process involves verifying whether the current operating entity (such as a user account, system service process, or auditor) has the authority to access specific sensitive data. It dynamically assesses the legality and granularity of data access by combining session identifiers, user roles, business scenario sensitivity, and compliance policies. Determining the authorized access status is a crucial step in ensuring data security and compliance, aiming to prevent unauthorized or excessive data access, thereby minimizing the risk of sensitive information leakage while ensuring business continuity.

[0070] By following the steps above, a dynamic access control mechanism based on scenario, audience, and permissions is constructed, providing an accurate decision-making basis for generating refined de-identification strategies in the future.

[0071] An optional embodiment of the data anonymization strategy generated based on business intent, business audience, authorized access status, and fusion confidence level includes: determining the business necessity of each global sensitive object based on business intent; determining the sensitivity level of each global sensitive object based on business audience and authorized access status; and generating a data anonymization strategy that includes at least processing actions and mask templates by combining the fusion confidence level, business necessity, and sensitivity level of each global sensitive object.

[0072] Furthermore, when generating data anonymization strategies, the business necessity of each globally sensitive object is first assessed based on the identified business intent of the target business. This determines whether a specific sensitive field is indispensable for completing the current business task. For example, in a check-in scenario, flight number and gate information have high business necessity, while name and PNR may have only low business necessity or even no necessity, because users do not need to know this private information to obtain their check-in status. The purpose of determining business necessity is to achieve the principle of minimum necessity, avoiding the retention of too much irrelevant sensitive information in business processing, thereby reducing the risk of leakage at the source.

[0073] Subsequently, the system, combining the identified business audience and current authorized access status, assigns sensitivity levels to each globally sensitive object. The sensitivity level reflects the degree of privacy, security, or compliance risk that may result from the disclosure of that sensitive object, and is also constrained by the audience's access permissions. For example, if the target audience is unauthorized external users or large models, even if a field has high business necessity, its actual exposure risk is extremely high, and therefore it will be assigned a high sensitivity level; if the target audience is authorized internal business systems with high business necessity, the sensitivity level may be lowered accordingly. The purpose of determining sensitivity levels is to quantify the risk exposure of data under different flow paths, providing a basis for differentiated protection.

[0074] Based on this, the system uses the fusion confidence level, business necessity level, and sensitivity level of each globally sensitive object as key input parameters to comprehensively generate a data anonymization strategy. The fusion confidence level is used to assess the reliability of the identification results; objects with high confidence levels can undergo more deterministic anonymization actions, while objects with low confidence levels tend to use more conservative masking or verification strategies. Business necessity level and sensitivity level together determine the type of processing action, such as retention, masking, deletion, or fingerprinting, as well as the specific mask template. Examples include retaining the first three and last four digits, or complete masking.

[0075] The above steps generate a refined and adaptive data anonymization strategy through collaborative decision-making based on multi-dimensional parameters, ensuring accurate protection of sensitive information while meeting business availability requirements.

[0076] In another optional embodiment, the step of desensitizing multimodal business data corresponding to the target business based on the data desensitization strategy includes: performing desensitization operations such as deletion, replacement, or masking on globally sensitive objects in the text modal data according to the processing actions in the data desensitization strategy to obtain desensitized text; performing image masking operations on sensitive areas in the image modal data according to the processing actions in the data desensitization strategy to obtain desensitized images, wherein the masking operations include solid color coverage, mosaic, blurring, region replacement, or security placeholder map coverage; for QR code and barcode areas, whole-block masking is preferred. This approach avoids relying on parsing code values ​​before local processing, thus preventing downstream components from re-parseing and writing ticket numbers, passenger reservation records, membership numbers, itinerary information, or business voucher information carried in QR codes or barcodes into logs, model contexts, or structured fields. Based on the processing actions in the data anonymization strategy, it performs synchronous update operations on structured business fields to obtain anonymized structured data, avoiding inconsistencies such as the original image being anonymized but structured fields still retaining plaintext, QR code or barcode areas being anonymized but logs still retaining plaintext, and user responses being anonymized but model contexts still retaining plaintext.

[0077] Step S106: Perform residual verification on the de-identified multimodal business data to obtain the verification result. Based on the verification result, obtain the final de-identified data. If the verification result indicates that the verification passed, obtain the final de-identified data; otherwise, if the verification result indicates that the verification failed, perform iterative rework.

[0078] In one optional embodiment, the step of performing residual verification on the de-identified multimodal service data includes: traversing the de-identified multimodal service data, checking whether there is residual plaintext data in the de-identified multimodal service data, and obtaining the check result; and calculating the residual risk value of the de-identified multimodal service data based on the check result.

[0079] Specifically, residual verification includes: detecting residual sensitive data in the de-identified text data and counting the number of times residual fields are hit; performing secondary optical character recognition on the de-identified images to check whether sensitive text such as names, ticket numbers, mobile phone numbers, membership numbers, and luggage tag numbers still exist; re-detecting QR codes and barcodes in the de-identified images to check whether there are still identifiable or decryptable code areas; scanning for sensitive fields in the minimum necessary structured fields, model context, response drafts, work order summaries, and log summaries; performing consistency verification on the results of the de-identification strategy execution to check whether the same object violates its strategy constraints in different output links; and performing output boundary verification on manually reviewed objects to check whether they are mistakenly output as deterministic business conclusions or plaintext sensitive fields.

[0080] The formula for calculating the residual risk value can be expressed as: Risk = F(hit_count, risk_level, output_scope, code_detected, policy_deviation), where Risk represents the residual risk value after de-identification; F represents the residual risk calculation function; hit_count represents the number of times the residual sensitive field is hit; risk_level represents the risk level of the residual field; output_scope represents the output audience or output link where the residual content is located; code_detected represents whether the QR code or barcode area is still detected after de-identification; and policy_deviation represents the degree of deviation of the actual output result from the object-level de-identification strategy.

[0081] In one optional embodiment, if the verification result indicates that the verification passed, the final de-identified data is obtained; or if the verification result indicates that the verification failed, the step of performing iterative rework includes: if the verification result indicates that the residual risk value of the de-identified multimodal business data is greater than the residual risk threshold, triggering iterative rework until the residual risk value of the de-identified multimodal business data is less than or equal to the residual risk threshold, thus obtaining the final de-identified data. Specifically, when iterative rework is triggered, the current number of reworks is counted; if the current number of reworks exceeds the rework count threshold, manual review is triggered to obtain the final de-identified data. If the residual risk value of the de-identified multimodal business data is less than or equal to the residual risk threshold, the de-identified multimodal business data is used as the final de-identified data.

[0082] Furthermore, after desensitizing the multimodal business data corresponding to the target business based on the data desensitization strategy, the process also includes: performing residual verification on the desensitized business data to obtain residual verification results, wherein the residual verification results include at least: the number of times sensitive fields are hit, the results of secondary detection, and the degree of policy deviation; calculating the residual risk value based on the residual verification results; when the residual risk value exceeds a preset threshold, triggering rework processing, adjusting the masking area or mask strength, and re-performing the desensitization; when the residual risk value is lower than the preset threshold, generating a security fingerprint and outputting the security result.

[0083] The system calculates the residual risk value based on the number of residual hits, field risk level, output audience risk, code area identifiability, policy deviation, and manual review markings. Specifically, the number of residual hits indicates the number or frequency of sensitive fields still being identified after de-identification; the field risk level indicates the sensitivity of the residual field itself; the output audience risk indicates the exposure risk of the output link containing the residual content; the code area identifiability indicates whether the QR code or barcode can still be detected or decoded; and the policy deviation indicates the degree of inconsistency between the actual output result and the object-level de-identification policy.

[0084] In one optional embodiment, after obtaining the final de-identified data based on the verification results, the method further includes: generating a security fingerprint, a chain of evidence description, and an audit log for the final de-identified data; and integrating and outputting the de-identified data, security fingerprint, chain of evidence description, and audit log.

[0085] Specifically, after the residual verification passes, the system generates a security fingerprint, a chain of evidence description, and an audit log. The security fingerprint is used to support subsequent consistent queries and cross-process associations without retaining plaintext sensitive values. The system can generate an irreversible security fingerprint based on field type, normalized value, and business scope identifier. In one optional implementation, the security fingerprint can be generated as follows: fingerprint = HMAC_SHA256(salt, type || value_norm || case_scope), where fingerprint represents an irreversible security fingerprint; HMAC_SHA256 represents a key-based hash operation; salt represents the key or salt value maintained by the security management component; type represents the field type; value_norm represents the field normalized value; case_scope represents the session, work order, case, tenant, or business processing scope identifier; and the symbol || represents string concatenation or equivalent data combination.

[0086] Using the above method, different system modules can determine whether two desensitized objects belong to the same sensitive object based on the security fingerprint, but cannot deduce the original sensitive value from the security fingerprint.

[0087] The evidence chain description is used to record the source, location, processing actions, and verification results of sensitive objects. For scenarios such as baggage complaints, the evidence chain may include nodes such as pictures of damaged baggage, pictures of baggage tags, boarding passes or check-in screenshots, complaint texts, and check-in vouchers, and records the relationships between each node, field consistency, evidence completeness, and suggestions for manual review.

[0088] Audit logs should include at least the following: input source identifier, object identifier, field type, processing strategy, strategy version number, execution time, output audience, residual validation results, number of rework attempts, manual review flags, and exception descriptions. Audit logs do not directly store plaintext sensitive values; instead, they prioritize storing irreversible fingerprints, anonymized summaries, and strategy execution records.

[0089] Through the above steps, multimodal business data corresponding to the target business is received and preprocessed to obtain a business data set. Sensitive objects are extracted from the business data set to obtain a candidate sensitive object set. Cross-modal recognition is performed on the candidate sensitive object set to merge the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain global sensitive objects. The fusion confidence of global sensitive objects is calculated to identify the business intent and business audience of the target business and determine the authorized access status of the multimodal business data. A data desensitization strategy is generated based on the business intent, business audience, authorized access status, and fusion confidence. The multimodal business data corresponding to the target business is desensitized based on the data desensitization strategy.

[0090] In this embodiment, sensitive objects are extracted from different modalities, and cross-modal recognition is used to merge candidate sensitive objects scattered across different modalities to generate global sensitive objects. This ensures that the same sensitive entity is treated as a whole in all modalities, achieving synchronous desensitization of the same object across different modalities. Furthermore, business intent, business audience, and authorization status are introduced as key dimensions for generating the desensitization strategy. This allows for the dynamic retention of the minimum necessary information based on task intent, satisfying the data requirements of subsequent business processing while minimizing the exposure of sensitive information. This balances data security and information availability. Iterative residual verification is performed on the desensitized multimodal business data to ensure the security of the output results and improve the accuracy of the desensitization results. This solves the technical problem in related technologies where a one-size-fits-all approach based on static rules for data desensitization is insufficient to achieve dual protection of data security and information availability.

[0091] The following describes in detail another optional implementation method.

[0092] Figure 2 This is a schematic diagram of an optional data desensitization process according to an embodiment of the present invention, such as... Figure 2 As shown, the data anonymization process includes:

[0093] Step 1: Accept business requests and obtain multimodal business data;

[0094] Receive multimodal business data from users or business systems. The multimodal business data includes at least one or more of the following: text data, image data, QR code area detection results, barcode area detection results, structured business fields, form fields, chat history screenshots, boarding pass images, check-in screenshots, baggage tag images, baggage claim document images, complaint description text, etc.

[0095] Step two, data processing;

[0096] Standardization processing of multimodal business data includes: cleaning text data to remove irrelevant characters, duplicate spaces, abnormal line breaks, system prompts, and irrelevant formatting information; performing orientation correction, sharpness detection, noise reduction, cropping, layout area recognition, and voucher type recognition on image data; performing optical character recognition (OCR) on text areas in images to generate OCR results such as text content, character span, text box coordinates, page number, and source identifier; detecting high-risk areas such as QR code areas, barcode areas, RFID tags, and invoice code areas to generate code area coordinates, area type, and recognition confidence; and normalizing field names, standardizing field values, marking field paths, and marking sources for structured fields.

[0097] After the above processing, the system forms a unified standard business data set: Input={text,image,ocr,code_region,structured_field,metadata}, where text represents the text content obtained from user input or image recognition, image represents the original image and its preprocessing results, ocr represents the optical character recognition results, code_region represents the QR code or barcode area, structured_field represents the business system field, and metadata represents metadata such as source file, page number, session number, and business scenario.

[0098] Step 3: Extraction of candidate sensitive objects;

[0099] The system performs sensitive candidate extraction on the standard input set based on rule matching, named entity recognition, field templates, business dictionary, layout analysis, and image object detection.

[0100] Sensitive candidate data includes, but is not limited to: name, mobile phone number, ID number, ticket number, membership number, baggage tag number, order number, QR code, barcode, flight number, flight date, departure airport, arrival airport, terminal, boarding gate, seat number, check-in status, complaint type, contact information, and business voucher number.

[0101] Each sensitive candidate object is represented as: c={type,value_raw,value_norm,modality,source_id,location,confidence,context}, where type represents the field type; value_raw represents the original value of the field; value_norm represents the standardized value of the field after formatting, case unification, whitespace removal, date format conversion, or checksum extraction; modality represents the source modality, including text, image region, QR code / barcode region, or structured field; source_id represents the source image, page, session, or business form number; location represents the location of the candidate object, including character span, image coordinate frame, or structured field path; confidence represents the recognition confidence level; and context represents the semantic context, field label, or business description around the candidate object.

[0102] Through the above candidate extraction method, the system can simultaneously obtain the text value, image location, field source, and business context of sensitive objects, providing a foundation for subsequent cross-modal object merging and consistent desensitization.

[0103] Step 4: Merge cross-modal objects to generate globally sensitive objects;

[0104] The system normalizes, compares, and merges sensitive candidate objects from different modalities to form a cross-modal unified sensitive object, i.e., a global sensitive object. The unified sensitive object is represented as: O = {object_id, type, value_norm_hash, evidence_set, source_index, object_confidence, policy_status}, where object_id represents a unique identifier generated by the system for the same sensitive object; type represents the sensitive object type; value_norm_hash represents the irreversible summary of the normalized value; evidence_set represents the evidence set of the object in different modalities; source_index represents the text location, OCR location, image location, and structured field path corresponding to the object; object_confidence represents the object-level fusion confidence; and policy_status represents the subsequent processing status of the object.

[0105] The system determines whether multiple candidate sensitive objects belong to the same entity based on the following conditions: the field types are the same or there is a preset business mapping relationship; the normalization values ​​are the same, partially the same, or meet the similarity threshold; the candidate objects are located near the same voucher, the same area, the same field label, or in the same business context; the candidate objects originate from the same session, the same work order, the same complaint case, or the same business request; the target area of ​​the QR code, barcode, or image has a spatial proximity relationship or business association relationship with the optical character recognition text, visible field, or structured field.

[0106] Step 5: Calculate the fusion confidence score;

[0107] After obtaining the globally sensitive objects, the system calculates the object-level fusion confidence score in the following manner: , where C(O) represents the object-level fusion confidence of the global sensitive object O; ci represents the confidence of the i-th candidate evidence; and wi represents the credibility weight of the source of the i-th candidate evidence.

[0108] Step 6: Determine whether the fusion confidence level is lower than the preset threshold. If yes, mark it as pending review and restrict its plaintext output. If no, proceed to step 7.

[0109] Tobj represents the object-level fusion confidence threshold, which is used to determine whether multiple candidate objects can be reliably merged into the same sensitive object. When C(O) is less than Tobj, it indicates that the evidence supporting this unified sensitive object is insufficient, and the system marks the object as a to-be-reviewed object; when there are field value conflicts, source conflicts or positional relationship conflicts between different candidate evidences, even if C(O) is not lower than Tobj, the system may also mark the object as a to-be-reviewed object and restrict its automatic plaintext output. The to-be-reviewed object is not directly discarded, but enters the subsequent policy generation link in a desensitized form. The system can retain its object ID, source index, risk description and review prompt in the secure output, so as to avoid sensitive information residue caused by ignoring low-confidence objects.

[0110] Step 7: Identify business intent and output audience;

[0111] Before generating the desensitization policy, the system identifies the current business task. Business tasks include but are not limited to: boarding pass question and answer, check-in status consultation, terminal query, baggage damage complaint, baggage loss complaint, work order backfilling, manual review, evidence chain analysis, model inference, audit trace retention, etc.

[0112] The system simultaneously identifies the output audience and the output link. The output audience (corresponding to the aforementioned business audience) at least includes: user-oriented natural language reply; large model-oriented context input; internal business system-oriented work order field; review interface for human customer service or auditors; logs or trace records for audit systems.

[0113] For the same sensitive object, the system may adopt different processing policies according to different output audiences. For example, fields such as complete ticket number, mobile phone number, membership number, baggage tag number generally shall not enter the large model-oriented context or user-oriented natural language reply; in the scenario of internal authorized work orders, they may circulate in the form of mask values, fingerprint values or controlled fields according to business necessity; in the audit trace retention scenario, irreversible fingerprints, policy version numbers and processing logs are preferentially adopted. Meanwhile, it is also necessary to identify the authorization status and risk level. The authorization status includes unlicensed, user-authorized, business-authorized, manual review-authorized, etc.; the risk level may be comprehensively determined based on the field type, scenario sensitivity, output audience, object confidence, data source and residual risk.

[0114] Step 8: Generate a desensitization policy;

[0115] The system generates an object-level desensitization policy according to the sensitive object type, object confidence, task intent, field necessity, field sensitivity level, output audience, authorization status and risk level.

[0116] The field to be processed (i.e., the globally sensitive object) is represented as f, and the intent of the current business task is represented as intent. The intent can include task types such as check-in consultation, boarding pass inquiry, baggage complaint, work order backfilling, manual review, evidence chain analysis, model reasoning, or audit record keeping.

[0117] The business necessity of field f under the task intent is represented as N(f, intent), and the sensitivity level of field f is represented as S(f). Here, N(f, intent) measures the necessity of field f in completing the current business task, and S(f) measures the potential privacy, security, business, or compliance risks that field f might pose if leaked. Based on the combination of N(f, intent) and S(f), the system determines the processing strategy for the corresponding unified sensitive object of this field.

[0118] The system determines the processing strategy based on the combination relationship between N(f, intent) and S(f) as follows: When a field is highly necessary for the current task and has a low sensitivity level, the original value or normalized value can be retained; when a field is highly necessary for the current task but has a high sensitivity level, partial masking, generalization, fingerprinting, or controlled retention only in the authorization link can be used; when a field is less necessary for the current task but has a high sensitivity level, deletion, complete masking, image masking, or irreversible fingerprinting should be used; when a field has low confidence, conflicting evidence, insufficient authorization status, or involves high-risk business conclusions such as liability judgment, compensation judgment, and security check release, a manual review strategy should be triggered.

[0119] The object-level de-identification strategy is represented as: Policy(O)={action,mask_template,output_scope,review_flag,audit_level}, where action represents the processing action, including retention, deletion, generalization, masking, image masking, block masking, fingerprinting, or manual review; mask_template represents the field mask template; output_scope represents the output audience to which this strategy applies; review_flag indicates whether manual review is required; and audit_level represents the audit logging level.

[0120] Step nine: Perform the desensitization operation;

[0121] The system performs consistent desensitization on globally sensitive objects based on object-level desensitization strategies.

[0122] The objects of consistent desensitization include: sensitive segments in the original text; sensitive text and its character span in optical character recognition results; sensitive text boxes, QR code areas, barcode areas, portrait areas or other sensitive areas in the original image; sensitive field values ​​in structured fields; model-oriented context, retrieval fragments and intermediate inference inputs; business response drafts, evidence chain descriptions, work order summaries and log summaries.

[0123] For text fields, the system can use deletion, replacement, partial masking, generalized description, or fingerprint substitution. For example, mobile phone numbers can retain the first three and last four digits, ticket numbers can retain part of the prefix and suffix, passenger reservation records can retain only one digit or be completely masked, and names can retain the last name or be completely masked.

[0124] For image fields, the system performs image occlusion based on the target detection box or code area detection box. The occlusion methods can include solid color coverage, mosaic, blurring, region replacement, or safe placeholder map coverage.

[0125] For QR code and barcode areas, the system prioritizes full-area masking rather than relying on parsing the code value before performing local processing. This is to prevent downstream components from re-parsing and writing the ticket number, passenger reservation record, membership number, itinerary information, or business voucher information carried in the QR code or barcode into logs, context, or structured fields.

[0126] For structured fields, the system updates field values ​​synchronously based on object identifiers and policy results to avoid inconsistencies such as the original image being masked but the structured fields still retaining plaintext, the optical character recognition results being de-identified but the logs still retaining plaintext, and the user's response being de-identified but the model context still retaining plaintext.

[0127] Step 10: Residue verification, calculate residual risk value;

[0128] To ensure the reliability of the de-identification results, this invention includes a residual verification step after consistent de-identification execution. Residual verification includes: detecting residual sensitive data in the de-identified text data and counting the number of times residual fields are hit; performing secondary optical character recognition on the de-identified images to check for the presence of sensitive text such as names, ticket numbers, mobile phone numbers, membership numbers, and luggage tags; re-detecting QR codes and barcodes in the de-identified images to check for the presence of identifiable or decodeable code areas; scanning for sensitive fields in the minimum necessary structured fields, model context, response drafts, work order summaries, and log summaries; performing consistency verification on the de-identification strategy execution results to check whether the same object violates its policy constraints in different output links; and performing output boundary verification on manually reviewed objects to check whether they have been mistakenly output as deterministic business conclusions or plaintext sensitive fields.

[0129] The system calculates the residual risk value based on the number of residual hits, field risk level, output audience risk, code area identifiability, policy deviation, and manual review markings. Specifically, the number of residual hits indicates the number or frequency of sensitive fields still being identified after de-identification; the field risk level indicates the sensitivity of the residual field itself; the output audience risk indicates the exposure risk of the output link containing the residual content; the code area identifiability indicates whether the QR code or barcode can still be detected or decoded; and the policy deviation indicates the degree of inconsistency between the actual output result and the object-level de-identification policy.

[0130] Subsequently, the residual risk value is calculated based on the residual verification results. The formula for calculating the residual risk value is: Risk = F(hit_count, risk_level, output_scope, code_detected, policy_deviation), where Risk represents the residual risk value after desensitization; F represents the residual risk calculation function; hit_count represents the number of times the residual sensitive field is hit; risk_level represents the risk level of the residual field; output_scope represents the output audience or output link where the residual content is located; code_detected represents whether the QR code or barcode area is still detected after desensitization; and policy_deviation represents the degree of deviation of the actual output result from the object-level desensitization strategy.

[0131] Tres represents the residual risk threshold. When Risk > Tres, the system determines that the desensitization result does not meet the safe output conditions, and it must not enter the normal output process, triggering rework or manual review.

[0132] Step 11: Determine whether the residual risk value is greater than the residual risk threshold. If not, generate a security fingerprint, audit log, evidence chain description and risk warning, and output the anonymized data.

[0133] After the residual verification passes, the system generates a security fingerprint, a chain of evidence description, and an audit log.

[0134] Security fingerprints are used to support subsequent consistent queries and cross-process associations without retaining plaintext sensitive values. The system can generate irreversible security fingerprints based on field type, normalized value, and business scope identifier. Security fingerprints can be generated as follows: fingerprint = HMAC_SHA256(salt, type||value_norm||case_scope), where fingerprint represents an irreversible security fingerprint; HMAC_SHA256 represents a key-based hash operation; salt represents the key or salt value maintained by the security management component; type represents the field type; value_norm represents the field normalized value; case_scope represents the session, work order, case, tenant, or business processing scope identifier; and the symbol || represents string concatenation or equivalent data combination.

[0135] Using the above method, different system modules can determine whether two desensitized objects belong to the same entity based on the security fingerprint, but cannot deduce the original sensitivity value from the security fingerprint.

[0136] The evidence chain description is used to record the source, location, processing actions, and verification results of sensitive objects. For scenarios such as baggage complaints, the evidence chain may include nodes such as pictures of damaged baggage, pictures of baggage tags, boarding passes or check-in screenshots, complaint texts, and check-in vouchers, and records the relationships between each node, field consistency, evidence completeness, and suggestions for manual review.

[0137] Audit logs should include at least the following: input source identifier, object ID, field type, processing strategy, strategy version number, execution time, output audience, residual validation results, number of rework attempts, manual review flags, and exception descriptions. Audit logs do not directly store plaintext sensitive values; instead, they prioritize storing irreversible fingerprints, anonymized summaries, and strategy execution records.

[0138] Step 12: Determine if the number of rework attempts exceeds the limit. If yes, proceed to manual review. If no, return to step 8 and repeat the desensitization operation.

[0139] To prevent the system from looping indefinitely after a residual verification failure, a maximum number of rework attempts, Rmax, is set. When the current number of rework attempts is less than Rmax, the system adjusts the masking area, mask strength, or output strategy and re-executes consistent desensitization and residual verification. When the current number of rework attempts reaches or exceeds Rmax, the system stops automatic rework, transfers the task to manual review, and prohibits the automatic output of sensitive plaintext or deterministic business conclusions.

[0140] The embodiments of the present invention will be described below with reference to specific implementation scenarios.

[0141] First, taking the extraction and desensitization of multimodal sensitive information in the context of boarding pass image Q&A as an example, a user uploads a screenshot of an electronic boarding pass through the intelligent customer service system and enters the question: "Have I successfully checked in? Can I go directly to security?"

[0142] The system needs to generate auxiliary answers based on user-uploaded images and text questions, while avoiding sensitive information such as names, ticket numbers, passenger reservation information, membership numbers, and QR codes from entering the model, user answers, logs, or structured output.

[0143] Input data includes user questions, boarding pass images, and optical character recognition results obtained by the system. Example input is shown in the table below.

[0144] Table 1. Exemplary Multimodal Service Data

[0145]

[0146] Based on this, the specific desensitization steps include: Step 1, the system receives the electronic boarding pass image and user text question uploaded by the user, and generates a session identifier case_scope for this request. The system performs orientation correction, sharpness detection, layout analysis, and voucher type identification on the image to confirm that the image is an electronic boarding pass.

[0147] Step two: The system performs OCR recognition on the image, obtaining the OCR text, character span, OCR bounding box, page number, and source identifier. Simultaneously, the system performs QR code region detection on the image, obtaining the QR code bounding box. The system then combines these results into a standard input set.

[0148] Input={

[0149] Text: "Have you successfully checked in? Can I proceed directly to security?"

[0150] image:"boarding_pass_image",

[0151] ocr:[

[0152] {"text":"Zhang Mou","box":[30,40,90,60]},

[0153] {"text":"CA1234","box":[120,80,190,100]},

[0154] {"text":"2026-05-20","box":[120,110,230,130]},

[0155] {"text":"AB12CD","box":[120,140,190,160]},

[0156] {"text":"12A","box":[120,170,160,190]},

[0157] {"text":"C12","box":[120,200,160,220]}

[0158] ],

[0159] code_region:[

[0160] {"type":"QR","box":[300,40,420,160]}

[0161] ],

[0162] metadata:{

[0163] "scene":"boarding_pass_qa",

[0164] "case_scope":"session_001"

[0165] }

[0166] }

[0167] Step 3: The system extracts candidate sensitive objects from the standard input set based on rule matching, named entity recognition, field templates, and layout position relationships.

[0168] Step four involves merging the candidate sensitive objects across modalities. If the same field appears repeatedly in OCR text, image regions, or structured fields, the system merges them into global sensitive objects.

[0169] Meanwhile, for the merged global sensitive objects, the fusion confidence score is calculated using the fusion confidence score calculation formula. The fusion confidence scores of PNR objects, name objects, and QR code objects are all not lower than the threshold Tobj, and they can enter the strategy generation stage. If a certain field has insufficient fusion confidence score due to image blurring, the system will still treat it as a suspected sensitive object and adopt a stronger masking strategy.

[0170] Step five: The system identifies the user's question as a check-in status inquiry and security check-related Q&A, and outputs to user-facing responses, large-scale models, and audit logs. The system generates object-level de-identification strategies based on field necessity N(f, intent) and field sensitivity level S(f).

[0171] Table 2. Exemplary object-level desensitization strategies

[0172]

[0173] Step Six: The system performs consistent data masking. For text and OCR results, the system deletes, masks, or generalizes fields such as name, PNR, and seat number according to a strategy; for images, the system masks corresponding areas based on OCR bounding boxes and QR code detection boxes; for models, the system retains only the minimum necessary information to complete the task; for logs, the system does not save plaintext sensitive values, but only the object ID, strategy version, and irreversible fingerprint.

[0174] For example, if a user uploads a screenshot of a boarding pass, the image can be anonymized from its original content to the following secure information during the model's data masking process: Visible fields in the image include flight number CA1234, flight date 2026-05-20, gate C12, and check-in status. Name, PNR, QR code, and seat number have been anonymized. Please generate a cautious supplementary response based on the visible information, without making definitive judgments regarding security clearance or boarding eligibility.

[0175] Step 7: The system performs residual verification after de-identification. Verification includes: performing secondary OCR recognition on the de-identified image to check for the presence of sensitive fields such as name, PNR, ticket number, and member number; performing QR code detection on the de-identified image to check if the QR code area is still recognizable; scanning the model Prompt, user response draft, and log summary for sensitive fields; and checking the policy consistency of each output link. The residual risk value (Risk) is calculated. If Risk > Tres, the system expands the image masking area, increases the mask strength, or regenerates the Prompt, and re-executes residual verification. If the number of rework attempts reaches the maximum number of rework attempts (Rmax), the system switches to manual review and prohibits the automatic output of sensitive plaintext or definitive business conclusions.

[0176] Step 8: After the residual verification passes, the system generates a security fingerprint, audit log, and security output results. The security fingerprint can be generated as follows: fingerprint=HMAC_SHA256(salt,type||value_norm||case_scope), where salt represents the key or salt value maintained by the security management component, type represents the field type, value_norm represents the normalized value, and case_scope represents the session, ticket, case, or tenant scope identifier.

[0177] It can be seen that the data anonymization method proposed in this embodiment of the invention can achieve consistent anonymization of images, OCR, prompts, user responses and logs in the boarding pass question and answer scenario, while retaining the minimum necessary business information required to complete the question and answer.

[0178] Secondly, taking the extraction and desensitization of multimodal sensitive information in the scenario of evidence chain for baggage damage complaints as an example, users upload pictures of damaged baggage, baggage tag, and boarding pass screenshots, and enter the complaint text: "The baggage was damaged upon arrival, please process and compensate."

[0179] The system needs to extract complaint facts, flight information, baggage information, and evidence from multiple images and texts. It must also consistently anonymize sensitive information such as names, contact information, PNR, ticket numbers, complete baggage tag numbers, QR codes, and barcodes, and generate an evidence chain description, evidence sufficiency score, and manual review suggestions. Specific execution steps include:

[0180] Step 1: The system receives photos of damaged luggage, luggage tags, boarding passes, and complaint text uploaded by users. It then categorizes the input data, classifying the images into damaged luggage photos, luggage tag photos, and boarding pass photos based on image content and layout characteristics.

[0181] Step two involves processing different images separately: Damage area detection is performed on baggage damage images to identify candidate damage types such as tears, dents, cracks, and stains; OCR recognition and barcode detection are performed on baggage tag images to identify the baggage tag number, carrier code, flight information, and barcode area; OCR recognition and QR code detection are performed on boarding pass screenshots to identify fields such as name, flight number, date, PNR, ticket number, and QR code area.

[0182] Step 3: The system extracts sensitive candidate objects and business candidate objects.

[0183] Step four: The system performs cross-modal object merging. For flight number and flight date, the system merges relevant content from boarding pass screenshots, baggage tag photos, and complaint texts into the same flight evidence object; for baggage tag number, the system merges the OCR text and barcode area in the baggage tag image into the same baggage claim object; for name, PNR, ticket number, and QR code areas, the system generates highly sensitive object IDs and records their source location.

[0184] Step 5: The system constructs a chain of evidence, and the nodes of the chain of evidence are shown in the table below.

[0185] Table 3 Nodes of the Evidence Chain

[0186]

[0187] Step six: The system calculates the sufficiency score of the evidence. The sufficiency score of the evidence can be calculated based on the completeness of the evidence, the consistency of the fields, the clarity of the image, the confidence level of damage identification, and the degree of correlation with the flight.

[0188] In one optional implementation, the evidence sufficiency score is expressed as: E_score = α × E_complete + β × E_consistency + γ × E_quality + δ × E_damage + λ × E_flight, where E_score represents the evidence sufficiency score; E_complete represents the completeness of evidence nodes; E_consistency represents the consistency of fields between different pieces of evidence; E_quality represents the image quality; E_damage represents the confidence level of damage identification; E_flight represents the degree of correlation between baggage information and flight information; α, β, γ, δ, and λ are weight parameters and can be configured according to business rules.

[0189] When E_score is lower than the preset evidence threshold, the system outputs a message indicating insufficient evidence and a suggestion for manual review, and does not automatically generate a conclusion on liability for compensation.

[0190] Step 7: The system identifies the current task intent as evidence chain analysis for baggage damage complaints, and outputs to audiences including internal work orders, models, manual review interfaces, and audit logs. Based on this, the system generates an object-level desensitization strategy.

[0191] Step 8: The system performs consistent desensitization, including masking the PNR, ticket number, and QR code areas in the boarding pass image, masking the barcode area in the baggage tag image, partially masking the baggage tag number, masking the contact information in the complaint text, and synchronously updating the OCR results, structured document fields, model context, evidence chain summary, and log summary.

[0192] Step nine: The system performs residual verification. If the de-identified baggage tag image still detects a barcode area, the system expands the barcode masking area. If the evidence chain digest still contains the complete baggage tag number, the system re-performs masking or fingerprinting. If the model context contains PNR or ticket number plaintext, the system deletes the relevant fields and regenerates the model context. If the number of rework attempts exceeds Rmax and the verification still fails, the system transfers the case to manual review.

[0193] It can be seen that the embodiments of the present invention can not only process sensitive fields in a single image, but also form a chain of evidence for multiple images, text descriptions and structured fields, and support complaint processing and manual review while ensuring the security of sensitive information.

[0194] The following is a detailed description with reference to another embodiment.

[0195] Example 2

[0196] The data desensitization device provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0197] Figure 3 This is a schematic diagram of an optional data desensitization device according to an embodiment of the present invention, such as... Figure 3 As shown, the data anonymization device may include: a receiving unit 31, an extraction unit 32, a merging unit 33, an anonymization unit 34, and a verification unit 35, wherein,

[0198] The receiving unit 31 is used to receive multimodal service data corresponding to the target service and preprocess the multimodal service data to obtain a service data set.

[0199] Extraction unit 32 is used to extract sensitive objects from the business data set to obtain a candidate sensitive object set;

[0200] The merging unit 33 is used to perform cross-modal recognition on the candidate sensitive object set, merge the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain the global sensitive objects, and calculate the fusion confidence of the global sensitive objects;

[0201] The desensitization unit 34 is used to identify the business intent and business audience of the target business, and determine the authorized access status of the multimodal business data. Based on the business intent, business audience, authorized access status and fusion confidence, a data desensitization strategy is generated. Based on the data desensitization strategy, the multimodal business data corresponding to the target business is desensitized to obtain the desensitized multimodal business data.

[0202] The verification unit 35 is used to perform residual verification on the de-identified multimodal business data, obtain the verification result, and obtain the final de-identified data based on the verification result. Specifically, if the verification result indicates that the verification passed, the final de-identified data is obtained; otherwise, if the verification result indicates that the verification failed, iterative rework processing is performed.

[0203] The aforementioned data anonymization device receives multimodal service data corresponding to the target service through receiving unit 31, and preprocesses the multimodal service data to obtain a service data set; extracts sensitive objects from the service data set through extraction unit 32 to obtain a candidate sensitive object set; performs cross-modal recognition on the candidate sensitive object set through merging unit 33, merging the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain global sensitive objects, and calculates the fusion confidence of global sensitive objects; and identifies the business intent and business audience of the target service through de-anonymization unit 34, and confirms... The system determines the authorized access status of multimodal business data, generates a data anonymization strategy based on business intent, business audience, authorized access status, and fusion confidence, and anonymizes the multimodal business data corresponding to the target business based on the data anonymization strategy to obtain anonymized multimodal business data; the anonymized multimodal business data is then subjected to residual verification by verification unit 35 to obtain verification results, and the final anonymized data is obtained based on the verification results. Wherein, if the verification result indicates that the verification passed, the final anonymized data is obtained, or if the verification result indicates that the verification failed, iterative rework processing is performed.

[0204] In this embodiment, sensitive objects are extracted from different modalities, and cross-modal recognition is used to merge candidate sensitive objects scattered across different modalities to generate global sensitive objects. This ensures that the same sensitive entity is treated as a whole in all modalities, achieving synchronous desensitization of the same object across different modalities. Furthermore, business intent, business audience, and authorization status are introduced as key dimensions for generating the desensitization strategy. This allows for the dynamic retention of the minimum necessary information based on task intent, satisfying the data requirements of subsequent business processing while minimizing the exposure of sensitive information. This balances data security and information availability. Iterative residual verification is performed on the desensitized multimodal business data to ensure the security of the output results and improve the accuracy of the desensitization results. This solves the technical problem in related technologies where a one-size-fits-all approach based on static rules for data desensitization is insufficient to achieve dual protection of data security and information availability.

[0205] Further, the receiving unit includes: a first receiving module, used to determine the target service to be desensitized, and receive multimodal service data corresponding to the target service, wherein the type of multimodal service data includes at least one of the following: text data, image data, optical character recognition results, QR code region detection results, and structured service fields; a first cleaning module, used to perform cleaning operations on the text data, removing irrelevant characters, duplicate spaces, and abnormal line breaks, and generating cleaned text modal data; a first processing module, used to perform orientation correction, sharpness detection, and noise reduction processing on the image data to obtain a preprocessed image, and perform layout region recognition on the preprocessed image to obtain image modal data containing layout coordinates; and a first integration module, used to integrate the data from the optical character recognition results, the QR code region detection results, the text modal data, and the image modal data to form a service data set.

[0206] Furthermore, the extraction unit includes: a first extraction module, used to perform entity extraction on text modal data and optical character recognition results in the business data set based on a preset business dictionary and entity recognition model, to obtain text-level candidate sensitive objects; a first extraction module, used to perform visual feature extraction on image modal data in the business data set using an image detection model, to obtain image-level candidate sensitive objects; a second processing module, used to perform normalization processing on structured business fields in the business data set using a preset standard field table, to obtain structured-level candidate sensitive objects; and a first configuration module, used to merge text-level candidate sensitive objects, image-level candidate sensitive objects, structured-level candidate sensitive objects, and QR code region detection results to obtain a candidate sensitive object set, and to configure field type, original field value, field normalized value, source modality, location information, and confidence level for each candidate sensitive object in the candidate sensitive object set.

[0207] Furthermore, the merging unit includes: a first filtering module, used to traverse the candidate sensitive objects in the candidate sensitive object set, and based on preset business mapping rules, filter candidate sensitive objects with the same field type or with business mapping relationships to construct an initial sensitive object group; a first calculation module, used to calculate the similarity between the field normative values ​​of each candidate sensitive object within the same initial sensitive object group, and merge candidate sensitive objects with similarity higher than a preset similarity threshold into the same association cluster; a first checking module, used to check the location information and source metadata of each candidate sensitive object in the association cluster, and retain the association cluster as a valid association cluster if the location information and source metadata of the candidate sensitive object meet preset conditions; and a first verification module, used to perform spatial association verification and business association verification on the candidate sensitive objects in the valid association cluster, and determine that the candidate sensitive objects in the valid association cluster point to the same entity if the spatial association verification and / or business association verification are passed, and merge the expression forms of the candidate sensitive objects pointing to the same entity in different modalities into a unified expression form to obtain global sensitive objects.

[0208] Furthermore, the merging unit also includes: a first acquisition module, used to acquire the confidence scores of each candidate sensitive object pointing to the same entity; a second acquisition module, used to acquire the confidence weight values ​​of the source modalities of each candidate sensitive object pointing to the same entity; a second calculation module, used to multiply the confidence scores and confidence weight values ​​of each candidate sensitive object pointing to the same entity to obtain a weighted confidence score; and a third calculation module, used to calculate the complement of the product of all weighted confidence scores to obtain the fusion confidence score of the global sensitive object.

[0209] Furthermore, the desensitization unit includes: a first identification module, used to identify the business intent of the target business based on the context information of the target business, wherein the business intent includes at least one of the following: check-in inquiry, baggage complaint, work order backfilling, audit log; a first determination module, used to determine the target audience of the multimodal business data output based on the business intent, thereby obtaining the business audience of the target business, wherein the target audience includes at least one of the following: user, internal business system, manual review interface; and a first query module, used to query the authorization status of the current session based on the target audience, thereby obtaining the authorized access status of the multimodal business data.

[0210] Furthermore, the desensitization unit also includes: a second determination module, used to determine the business necessity of each global sensitive object based on business intent; a third determination module, used to determine the sensitivity level of each global sensitive object based on business audience and authorized access status; and a first generation module, used to generate a data desensitization strategy that includes at least processing actions and mask templates by combining the fusion confidence, business necessity, and sensitivity level of each global sensitive object.

[0211] Furthermore, the verification unit includes: a first inspection module, used to traverse the de-identified multimodal business data, check whether there is residual plaintext data in the de-identified multimodal business data, and obtain the inspection result; and a first calculation module, used to calculate the residual risk value of the de-identified multimodal business data based on the inspection result.

[0212] Furthermore, the verification unit also includes: a first triggering module, used to trigger iterative rework processing when the verification result indicates that the residual risk value of the desensitized multimodal business data is greater than the residual risk threshold, until the residual risk value of the desensitized multimodal business data is less than or equal to the residual risk threshold, thus obtaining the final desensitized data. Specifically, when iterative rework processing is triggered, the current number of reworks is counted, and when the current number of reworks exceeds the rework count threshold, manual review is triggered to obtain the final desensitized data; and a first using module, used to use the desensitized multimodal business data as the final desensitized data when the residual risk value of the desensitized multimodal business data is less than or equal to the residual risk threshold.

[0213] Furthermore, the data anonymization device also includes: a second generation module for generating security fingerprints, evidence chain descriptions, and audit logs for the final anonymized data; and a first output module for integrating and outputting the anonymized data, security fingerprints, evidence chain descriptions, and audit logs.

[0214] The aforementioned data desensitization device may also include a processor and a memory. The receiving unit 31, extraction unit 32, merging unit 33, desensitization unit 34, verification unit 35, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0215] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and multimodal data can be anonymized by adjusting kernel parameters.

[0216] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0217] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute any of the above-described data desensitization methods.

[0218] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-described data desensitization methods.

[0219] According to another aspect of the present invention, a computer program product is also provided, the computer program product including a computer program, wherein the computer program, when executed by a processor, implements any of the above-described data desensitization methods.

[0220] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: receiving multimodal business data corresponding to a target business, and preprocessing the multimodal business data to obtain a business data set; extracting sensitive objects from the business data set to obtain a candidate sensitive object set; performing cross-modal recognition on the candidate sensitive object set, merging the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain global sensitive objects, and calculating the fusion confidence of the global sensitive objects; identifying the business intent and business audience of the target business, and determining the authorized access status of the multimodal business data, generating a data desensitization strategy based on the business intent, business audience, authorized access status, and fusion confidence, desensitizing the multimodal business data corresponding to the target business based on the data desensitization strategy to obtain desensitized multimodal business data; performing residual verification on the desensitized multimodal business data to obtain a verification result, and obtaining the final desensitized data based on the verification result, wherein, if the verification result indicates that the verification passed, the final desensitized data is obtained, or if the verification result indicates that the verification failed, iterative rework processing is performed.

[0221] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) that performs a data desensitization method according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more processors ( Figure 4 The processor, denoted by 402a, 402b, ..., 402n, can include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA), and a memory 404 for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports in the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 4The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0222] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0223] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0224] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0225] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0227] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0228] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data anonymization method, characterized in that, include: Receive multimodal service data corresponding to the target service, and preprocess the multimodal service data to obtain a service data set; Sensitive objects are extracted from the business data set to obtain a candidate sensitive object set; Cross-modal recognition is performed on the candidate sensitive object set to merge the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain global sensitive objects, and the fusion confidence of the global sensitive objects is calculated. Identify the business intent and audience of the target business, and determine the authorized access status of the multimodal business data. Generate a data anonymization strategy based on the business intent, the business audience, the authorized access status, and the fusion confidence level. Anonymize the multimodal business data corresponding to the target business based on the data anonymization strategy to obtain anonymized multimodal business data. The de-identified multimodal business data is subjected to residual verification to obtain the verification result. The final de-identified data is obtained based on the verification result. If the verification result indicates that the verification passed, the final de-identified data is obtained. If the verification result indicates that the verification failed, iterative rework is performed.

2. The method according to claim 1, characterized in that, The steps of receiving multimodal service data corresponding to the target service and preprocessing the multimodal service data include: Identify the target service to be de-identified and receive the multimodal service data corresponding to the target service, wherein the type of the multimodal service data includes at least one of the following: text data, image data, optical character recognition results, QR code area detection results, and structured service fields; The text data is cleaned to remove irrelevant characters, duplicate spaces, and abnormal line breaks, generating cleaned text modal data. The image data is subjected to orientation correction, sharpness detection and noise reduction to obtain a preprocessed image, and the preprocessed image is subjected to page area recognition to obtain image modal data containing page coordinates; The data from the optical character recognition results, the QR code area detection results, the text modal data, and the image modal data are integrated to form the business data set.

3. The method according to claim 2, characterized in that, The steps of extracting sensitive objects from the business data set to obtain a candidate sensitive object set include: Based on a preset business dictionary and entity recognition model, entity extraction is performed on the text modal data and optical character recognition results in the business data set to obtain text-level candidate sensitive objects; Visual feature extraction is performed on the image modal data in the business data set using an image detection model to obtain image-level candidate sensitive objects; The structured business fields in the business data set are normalized using a preset standard field table to obtain structured candidate sensitive objects; The text-level candidate sensitive objects, the image-level candidate sensitive objects, the structure-level candidate sensitive objects, and the QR code region detection results are merged to obtain a candidate sensitive object set. Each candidate sensitive object in the candidate sensitive object set is configured with a field type, field original value, field normalized value, source modality, location information, and confidence level.

4. The method according to claim 1, characterized in that, The steps of performing cross-modal recognition on the candidate sensitive object set, merging the representations of the candidate sensitive objects in different modalities into a unified representation, and obtaining the global sensitive objects include: Traverse the candidate sensitive objects in the candidate sensitive object set, and based on the preset business mapping rules, filter candidate sensitive objects with the same field type or with a business mapping relationship to construct the initial sensitive object group; Calculate the similarity between the field normed values ​​of each candidate sensitive object within the same initial sensitive object group, and merge candidate sensitive objects with similarity higher than a preset similarity threshold into the same associated cluster; Check the location information and source metadata of each candidate sensitive object in the association cluster. If the location information and source metadata of the candidate sensitive object meet the preset conditions, retain the association cluster as a valid association cluster. Spatial association verification and business association verification are performed on the candidate sensitive objects in the effective association cluster. If the spatial association verification and / or the business association verification are passed, it is determined that the candidate sensitive objects in the effective association cluster point to the same entity. The expression forms of the candidate sensitive objects pointing to the same entity in different modalities are merged into a unified expression form to obtain the global sensitive object.

5. The method according to claim 4, characterized in that, The steps for calculating the fusion confidence of the globally sensitive object include: Obtain the confidence level of each of the candidate sensitive objects pointing to the same entity; Obtain the confidence weight values ​​of the source modalities of each of the candidate sensitive objects pointing to the same entity; The weighted confidence score is obtained by multiplying the confidence scores and confidence weight values ​​of each of the candidate sensitive objects that point to the same entity. Calculate the complement of all weighted confidence products to obtain the fused confidence of the global sensitive object.

6. The method according to claim 1, characterized in that, The steps of identifying the business intent and audience of the target service, and determining the authorized access status of the multimodal service data, include: Based on the contextual information of the target service, the business intent of the target service is identified, wherein the business intent includes at least one of the following: check-in inquiry, baggage complaint, work order feedback, and audit log. Based on the business intent, the target audience of the multimodal business data output is determined, and the business audience of the target business is obtained, wherein the target audience includes at least one of the following: users, internal business systems, and manual review interfaces; Based on the authorization status of the current session query by the target audience, the authorized access status of the multimodal service data can be obtained.

7. The method according to claim 6, characterized in that, The steps for generating a data anonymization strategy based on the business intent, the business audience, the authorized access status, and the fusion confidence level include: The business necessity of each of the globally sensitive objects is determined based on the stated business intent; The sensitivity level of each of the globally sensitive objects is determined based on the business audience and the authorized access status; The data anonymization strategy is generated by combining the fusion confidence, business necessity, and sensitivity level of each of the global sensitive objects, and includes at least processing actions and mask templates.

8. The method according to claim 1, characterized in that, The steps for performing residual verification on the anonymized multimodal service data include: Traverse the anonymized multimodal service data and check whether there is any residual plaintext data in the anonymized multimodal service data to obtain the check results; The residual risk value of the anonymized multimodal business data is calculated based on the inspection results.

9. The method according to claim 8, characterized in that, If the verification result indicates that the verification passed, the final de-identified data is obtained; or, if the verification result indicates that the verification failed, the iterative rework process includes the following steps: If the verification result indicates that the residual risk value of the desensitized multimodal business data is greater than the residual risk threshold, iterative rework is triggered until the residual risk value of the desensitized multimodal business data is less than or equal to the residual risk threshold, and the final desensitized data is obtained. In the case of triggering iterative rework, the current number of reworks is counted. If the current number of reworks is greater than the rework count threshold, manual review is triggered to obtain the final desensitized data. If the residual risk value of the de-identified multimodal service data is less than or equal to the residual risk threshold, the de-identified multimodal service data will be used as the final de-identified data.

10. The method according to claim 9, characterized in that, After obtaining the final de-identified data based on the verification results, the process also includes: Generate a security fingerprint, a chain of evidence description, and an audit log for the final de-identified data; The anonymized data, the security fingerprint, the evidence chain description, and the audit log are integrated and output.

11. A data anonymization device, characterized in that, include: The receiving unit is used to receive multimodal service data corresponding to the target service, and to preprocess the multimodal service data to obtain a service data set. An extraction unit is used to extract sensitive objects from the business data set to obtain a candidate sensitive object set. The merging unit is used to perform cross-modal recognition on the candidate sensitive object set, merge the expression forms of the candidate sensitive objects in different modalities into a unified expression form to obtain global sensitive objects, and calculate the fusion confidence of the global sensitive objects; The desensitization unit is used to identify the business intent and business audience of the target business, and determine the authorized access status of the multimodal business data. Based on the business intent, the business audience, the authorized access status and the fusion confidence, a data desensitization strategy is generated. Based on the data desensitization strategy, the multimodal business data corresponding to the target business is desensitized to obtain the desensitized multimodal business data. The verification unit is used to perform residual verification on the de-identified multimodal service data, obtain the verification result, and obtain the final de-identified data based on the verification result. If the verification result indicates that the verification passed, the final de-identified data is obtained; otherwise, if the verification result indicates that the verification failed, iterative rework processing is performed.

Images