Private data intelligent identification method and system based on deep multi-modal fusion

Abstract

The invention discloses a privacy data intelligent identification method and system based on deep multi-modal fusion, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly obtaining a multi-modal to-be-analyzed privacy parameter of a medical data main body, screening a key privacy parameter, and optimizing a current adaptive multi-modal integrated identification model to obtain a target model; based on the convergence state, the total parameter amount, the AUC value and the privacy parameter importance evaluation iterative optimization model, selecting a privacy parameter contained in the target model corresponding to the AUC peak value as a reference privacy parameter; and matching the reference features and loading the reference features to the target model to obtain a privacy risk confidence coefficient, and determining a privacy protection decision according to the privacy risk confidence coefficient. According to the method, through multi-modal fusion and dynamic model optimization, the privacy identification accuracy is improved, and a quantitative basis is provided for protection decision making.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for intelligent identification of privacy data based on deep multimodal fusion. Background Technology

[0002] With the development of medical informatization, medical data is exhibiting multimodal characteristics such as text, numerical values, and images, making the need for privacy protection increasingly urgent. Existing privacy data identification methods mostly rely on single-modal features, which are prone to missed or false detections of privacy risks due to insufficient feature dimensions; the model optimization process lacks a comprehensive evaluation of convergence state, total number of parameters, and recognition performance, making it difficult to balance model accuracy and complexity; privacy protection decisions are mostly based on subjective judgment and lack quantitative risk assessment basis. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for intelligent identification of privacy data based on deep multimodal fusion.

[0004] In a first aspect, embodiments of the present invention provide a method for intelligent identification of privacy data based on deep multimodal fusion, comprising: Multiple privacy parameters to be analyzed are obtained from the medical data subject. These multiple privacy parameters correspond to the privacy-sensitive information and related multimodal features of the medical data subject. Different privacy parameters correspond to different modality categories. Key privacy parameters are selected from the multiple privacy parameters to be analyzed. The current adaptive multimodal integrated recognition model is optimized based on the key privacy parameters to obtain the target adaptive multimodal integrated recognition model. Based on the convergence state, total number of parameters, AUC value, and importance assessment of privacy parameters of the target adaptive multimodal integrated recognition model, optimization iteration is performed until the model converges. The privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the peak AUC value are selected as the benchmark privacy parameters from the target adaptive multimodal integrated recognition model obtained during the iteration process. A benchmark feature identical to the benchmark privacy parameter is determined from the features of the medical data subject. The benchmark feature is composed of at least one feature corresponding to the medical data subject. The benchmark feature is used to enhance the modality fusion recognition reliability of the target adaptive multimodal integrated recognition model corresponding to the AUC peak. The benchmark features are loaded into the target adaptive multimodal ensemble recognition model corresponding to the peak AUC to obtain the privacy risk confidence coefficient of the medical data subject. Based on the privacy risk confidence coefficient, it is determined whether to provide corresponding privacy protection decisions for the medical data subject. The privacy risk confidence coefficient is used to assess the privacy risk confidence level of the medical data subject.

[0005] In this embodiment of the invention, the optimization iteration based on the convergence state, total number of parameters, AUC value, and importance assessment of privacy parameters of the target adaptive multimodal ensemble recognition model is performed until the model converges. The privacy parameters included in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are selected as benchmark privacy parameters from the target adaptive multimodal ensemble recognition models obtained during the iteration process, including: When it is determined that the target adaptive multimodal ensemble recognition model does not meet the model convergence condition, the first AUC value corresponding to the current adaptive multimodal ensemble recognition model and the second AUC value corresponding to the target adaptive multimodal ensemble recognition model are determined. When the total number of parameters in the target adaptive multimodal ensemble recognition model is not greater than a preset total parameter threshold, the second AUC value is greater than the first AUC value, and the importance assessment value of each privacy parameter in the target adaptive multimodal ensemble recognition model is not greater than a preset importance assessment threshold, the target adaptive multimodal ensemble recognition model is taken as the current adaptive multimodal ensemble recognition model, and the step of determining the key privacy parameters among the multiple privacy parameters to be analyzed is repeated; When the total number of parameters exceeds the preset threshold, multiple candidate adaptive multimodal integrated recognition models are generated based on each privacy parameter in the target adaptive multimodal integrated recognition model. The model corresponding to the AUC peak value among the multiple candidate adaptive multimodal integrated recognition models is used as the optimized target adaptive multimodal integrated recognition model. The optimized target adaptive multimodal integrated recognition model is used as the current adaptive multimodal integrated recognition model. The step of selecting key privacy parameters from the multiple privacy parameters to be analyzed is repeated. When the target adaptive multimodal ensemble recognition model is determined to meet the model convergence condition, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters.

[0006] In this embodiment of the invention, the step of generating multiple candidate adaptive multimodal ensemble recognition models based on each privacy parameter in the target adaptive multimodal ensemble recognition model includes: When it is determined that a privacy parameter with an importance evaluation value greater than a preset importance evaluation threshold is detected in the target adaptive multimodal integrated recognition model, the privacy parameter with an importance evaluation value greater than the preset importance evaluation threshold in the target adaptive multimodal integrated recognition model is used as a redundant privacy parameter; The privacy parameter whose importance evaluation value in the target adaptive multimodal integrated recognition model is not greater than the preset importance evaluation threshold is selected as the privacy parameter; Multiple privacy parameter groups are generated based on the redundant privacy parameters and the selected privacy parameters, and each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model to obtain multiple candidate adaptive multimodal ensemble recognition models. The basic adaptive multimodal ensemble recognition model is an adaptive multimodal ensemble recognition model without privacy parameters.

[0007] In this embodiment of the invention, the step of generating multiple candidate adaptive multimodal ensemble recognition models based on each privacy parameter in the target adaptive multimodal ensemble recognition model further includes: When no privacy parameter with an importance evaluation value greater than a preset importance evaluation threshold is detected in the target adaptive multimodal integrated recognition model, multiple privacy parameter groups are generated based on each privacy parameter in the target adaptive multimodal integrated recognition model; Each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model to obtain multiple candidate adaptive multimodal ensemble recognition models.

[0008] In this embodiment of the invention, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained from the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as benchmark privacy parameters, including: From the target adaptive multimodal integrated recognition model obtained during the iteration process, a first target adaptive multimodal integrated recognition model with a total number of parameters not greater than a preset total parameter threshold is determined; A second target adaptive multimodal integrated recognition model is determined in the first target adaptive multimodal integrated recognition model such that the importance evaluation value of each privacy parameter is not greater than a preset importance evaluation threshold; In the second target adaptive multimodal ensemble recognition model, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak is determined, and the privacy parameter of the target adaptive multimodal ensemble recognition model corresponding to the AUC peak is used as the benchmark privacy parameter.

[0009] In this embodiment of the invention, the method further includes: After generating the target adaptive multimodal integrated recognition model in each iteration, the target adaptive multimodal integrated recognition model of the previous optimization cycle is obtained; When the privacy parameters contained in the target adaptive multimodal ensemble recognition model of the previous optimization cycle are the same as those contained in the target adaptive multimodal ensemble recognition model, it is determined that the target adaptive multimodal ensemble recognition model satisfies the model convergence condition. The target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters. The method further includes: After generating the target adaptive multimodal integrated recognition model in each iteration, the optimization round corresponding to the target adaptive multimodal integrated recognition model is determined; When the optimization round reaches the preset optimization round, it is determined that the target adaptive multimodal integrated recognition model satisfies the model convergence condition. The target adaptive multimodal integrated recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal integrated recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters. The method further includes: When it is determined that the total number of parameters in the target adaptive multimodal integrated recognition model is not greater than the preset total number of parameters threshold, if the second AUC value is not greater than the first AUC value, or if the importance assessment value of any privacy parameter in the target adaptive multimodal integrated recognition model is greater than the preset importance assessment threshold, then the step of filtering out key privacy parameters from the multiple privacy parameters to be analyzed is repeated. The current adaptive multimodal ensemble recognition model is optimized based on the key privacy parameters to obtain the target adaptive multimodal ensemble recognition model.

[0010] In this embodiment of the invention, before using the target adaptive multimodal ensemble recognition model as the current adaptive multimodal ensemble recognition model, the method further includes: When a model identical to the target adaptive multimodal integrated recognition model is detected in the target adaptive multimodal integrated recognition model obtained during the determination iteration process, the step of filtering out key privacy parameters from the multiple privacy parameters to be analyzed is repeated; The step of using the target adaptive multimodal ensemble recognition model as the current adaptive multimodal ensemble recognition model includes: When no model identical to the target adaptive multimodal ensemble recognition model is detected in the target adaptive multimodal ensemble recognition model obtained during the iteration process, the target adaptive multimodal ensemble recognition model is taken as the current adaptive multimodal ensemble recognition model.

[0011] In this embodiment of the invention, before filtering out key privacy parameters from the plurality of privacy parameters to be analyzed, the method further includes: Among the plurality of privacy parameters to be analyzed, the importance assessment value corresponding to each privacy parameter to be analyzed is determined; Each privacy parameter to be analyzed is organized in ascending order according to the importance assessment value to obtain a priority queue of privacy parameters to be analyzed; The multiple privacy parameters to be analyzed are scheduled in stages according to the priority queue of the privacy parameters to be analyzed, resulting in multiple groups of privacy parameters to be analyzed in multiple processing stages.

[0012] In this embodiment of the invention, the step of filtering out key privacy parameters from the plurality of privacy parameters to be analyzed includes: Determine the set of privacy parameters to be analyzed corresponding to the current processing stage; When an unscheduled privacy parameter is detected in the group of privacy parameters to be analyzed corresponding to the current processing stage, key privacy parameters are selected from the unscheduled privacy parameters to be analyzed. If no unscheduled privacy parameters are detected in the privacy parameter group to be analyzed in the current processing stage, then the key privacy parameters are selected from the privacy parameter group to be analyzed in the next processing stage.

[0013] In a second aspect, embodiments of the present invention provide a server system, including a server, the server being used to perform the method described in the first aspect.

[0014] Compared to existing technologies, the beneficial effects provided by this invention include: Employing a privacy data intelligent identification method and system based on deep multimodal fusion disclosed in this invention, the method acquires multimodal privacy parameters to be analyzed from the subject of medical data, filters key privacy parameters to optimize the current adaptive multimodal integrated identification model to obtain the target model; iteratively optimizes the model based on convergence state, total number of parameters, AUC value, and privacy parameter importance assessment, selecting the privacy parameters contained in the target model corresponding to the AUC peak as the benchmark privacy parameters; matching benchmark features and loading them into the target model to obtain the privacy risk confidence coefficient, thereby determining privacy protection decisions. This method improves the accuracy of privacy identification through multimodal fusion and dynamic model optimization, providing a quantitative basis for protection decisions. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating the steps of the privacy data intelligent identification method based on deep multimodal fusion provided in an embodiment of the present invention; Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0018] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] In order to solve the technical problems mentioned in the background art Figure 1 This is a flowchart illustrating the privacy data intelligent recognition method based on deep multimodal fusion provided in this disclosure. The following is a detailed description of the privacy data intelligent recognition method based on deep multimodal fusion.

[0020] Step S201: Obtain multiple privacy parameters to be analyzed for the medical data subject. The multiple privacy parameters to be analyzed correspond to the privacy-sensitive information and related multimodal features of the medical data subject, respectively. Different privacy parameters to be analyzed correspond to different modal categories. Select key privacy parameters from the multiple privacy parameters to be analyzed, and optimize the current adaptive multimodal integrated recognition model based on the key privacy parameters to obtain the target adaptive multimodal integrated recognition model. Step S202: Based on the convergence state, total number of parameters, AUC value and importance assessment of privacy parameters of the target adaptive multimodal integrated recognition model, perform optimization iteration until the model converges. Select the privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the peak value of AUC from the target adaptive multimodal integrated recognition model obtained during the iteration process as the benchmark privacy parameters. Step S203: Determine a benchmark feature that is the same as the benchmark privacy parameter from the features of the medical data subject. The benchmark feature is composed of at least one feature corresponding to the medical data subject. The benchmark feature is used to enhance the modality fusion recognition reliability of the target adaptive multimodal integrated recognition model corresponding to the AUC peak. Step S204: Load the benchmark features into the target adaptive multimodal ensemble recognition model corresponding to the AUC peak to obtain the privacy risk confidence coefficient of the medical data subject. Determine whether to provide corresponding privacy protection decisions for the medical data subject based on the privacy risk confidence coefficient. The privacy risk confidence coefficient is used to assess the privacy risk confidence of the medical data subject.

[0021] In this embodiment of the invention, exemplarily, this embodiment takes Zhang San (45 years old, seeking medical treatment for lung discomfort) from a tertiary hospital as an example. The server acts as the execution entity, executing a privacy data intelligent identification method based on deep multimodal fusion. First, the server connects to multiple data sources, including the Hospital Information System (HIS), Medical Image Archiving and Communication System (PACS), Laboratory Information System (LIS), and voice consultation system, to obtain multiple privacy parameters of Zhang San to be analyzed. These parameters correspond to different modal categories: text modalities include the patient's name "Zhang San", ID number "110XXXX1978XXXX1234", home address "No. 123, XX Road, XX District, XX City", and doctor's consultation recording converted to text "The patient reported coughing for 3 days, without fever, and no history of allergies"; numerical modalities include blood routine white blood cell count "5.6×10 9The parameters included: alanine aminotransferase (ALT) of 45 U / L; image modalities included chest CT lesion location features (ground-glass nodules in the apical segment of the right upper lobe) and fundus photograph features (mild retinal artery sclerosis). Different modal parameters corresponded to specific processing modules within the model. Text modal parameters were input into the natural language processing sub-model, numerical modal parameters into the statistical feature extraction sub-model, and image modal parameters into the convolutional neural network sub-model. Before screening key privacy parameters, the server used a random forest feature importance algorithm to calculate the contribution of each privacy parameter to the privacy identification task, obtaining the following importance assessment values: ID number 0.8, chest CT lesion location features 0.75, patient name 0.6, blood routine white blood cell count 0.5, home address 0.4, doctor's consultation recording converted to text 0.2, liver and kidney function indicators 0.3, fundus photograph features 0. The server then organizes these parameters in ascending order according to their importance assessment values, forming a priority queue of privacy parameters to be analyzed: doctor's consultation recording text (0.2) → liver and kidney function indicators (0.3) → home address (0.4) → blood routine white blood cell count (0.5) → fundus photograph features (0.55) → patient name (0.6) → chest CT lesion location features (0.75) → ID number (0.8). The parameters are then scheduled in stages according to their importance intervals, resulting in three processing stages of privacy parameter groups to be analyzed: the first stage (importance 0.2-0.4) includes doctor's consultation recording text, liver and kidney function indicators, and home address; the second stage (importance 0.5-0.6) includes blood routine white blood cell count, fundus photograph features, and patient name; and the third stage (importance 0.7-0.8) includes chest CT lesion location features and ID number.

[0022] The server sequentially filters key privacy parameters according to the processing stage. First, it processes the parameter group of the first stage. Among them, the doctor's consultation recording text is a non-key symptom description, the liver and kidney function indicators are medical indicators with low privacy sensitivity, and the home address is a core parameter that is directly related to personal identity in the text modality. Therefore, the server selects the home address as the key privacy parameter of the first stage and loads it into the current adaptive multimodal ensemble recognition model (initial model, which contains three basic sub-models: text, numerical, and image, and has not loaded specific privacy parameters). The server also fine-tunes the weights of the fully connected layer of the text sub-model (learning rate 0.001, 50 iterations) so that the model can extract the regional identification features in the home address, resulting in the optimized target adaptive multimodal ensemble recognition model (model M1).

[0023] The server performs optimization iterations based on the convergence status, total number of parameters, AUC value, and importance assessment of privacy parameters of the target adaptive multimodal ensemble recognition model. First, it is determined that model M1 has not converged (the convergence condition is that the model parameters are exactly the same in two consecutive rounds or the optimization rounds reach 10 rounds). The first AUC value of the initial model is calculated to be 0.6 (the test set is 5000 medical data points labeled with privacy risks), and the second AUC value of model M1 is 0.65. Further checks reveal that the total number of parameters of model M1 is 1 (≤ the preset total number of parameters threshold 10), the second AUC value is greater than the first AUC value, and the importance assessment value of home address is 0.4 ≤ the preset importance assessment threshold 0.6, which meets the iteration conditions. The server uses model M1 as the current model to enter the second round of iteration. In the second iteration, the second-stage parameter set was processed. Patient name (text modality, direct identity identifier, importance 0.6) was selected as a key privacy parameter, loaded into model M1, and the weights of the text sub-model were fine-tuned to obtain model M2. The AUC value of model M2 was 0.7 (>0.65 of model M1), with a total of 2 parameters (≤ threshold). The importance evaluation values ​​of each parameter were all ≤ threshold, and the server continued to iterate. In the third iteration, the remaining parameters from the second stage were processed. Blood routine white blood cell count (numerical modality, importance 0.5) was selected as a key privacy parameter, loaded into model M2, and the numerical sub-model was fine-tuned (L2 regularization was added to prevent overfitting) to obtain model M3. The AUC value of model M3 was 0.72 (>0.7 of model M2), with a total of 3 parameters (≤ threshold). The importance evaluation values ​​of each parameter were all ≤ threshold, and the server continued to iterate. In the fourth iteration, the parameter set from the third stage was processed. Chest CT lesion location features (image modality, importance 0.75) were selected as key privacy parameters, loaded into model M3, and the image sub-model was fine-tuned (a dropout layer was added, scale 0.3) to obtain model M4. The AUC value of model M4 was 0.8 (> 0.72 of model M3). However, the importance assessment value of the chest CT lesion location features in model M4 (0.75) was greater than the preset importance assessment threshold of 0.6. The server marked this as a redundant privacy parameter and simultaneously removed home addresses, patient names, and blood routine white blood cell counts from model M4 with importance assessment values ​​≤ the threshold. The selected privacy parameter is used as the number of privacy parameters. Multiple privacy parameter groups are generated by combining redundant privacy parameters and the selected privacy parameters: {home address, patient name, blood routine white blood cell count}, {home address, patient name, blood routine white blood cell count, chest CT lesion location features}, {home address, patient name, chest CT lesion location features}. Each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model (the initial model without privacy parameters) to obtain candidate models C1, C2, and C3. After testing, it was found that candidate model C2 had the highest AUC value (0.8). The server uses candidate model C2 as the current model for further iteration.In the fifth iteration, the remaining fundus image features (image modality, importance 0.55) from the second stage are processed as key privacy parameters, loaded into candidate model C2, and the image sub-model weights are fine-tuned to obtain model M5. The AUC value of model M5 is 0.82 (>0.8 of candidate model C2), the total number of parameters is 5 (≤ threshold), and the importance evaluation value of each parameter is ≤ threshold. The server continues to iterate. In the sixth iteration, the remaining ID card numbers (text modality, importance 0.8) from the third stage were processed as key privacy parameters, loaded into model M5, and the weights of the text sub-model were fine-tuned to obtain model M6. The AUC value of model M6 was 0.83 (> 0.82 of model M5), but the importance evaluation value of the ID card number (0.8) was greater than the preset threshold of 0.6. The server marked it as a redundant privacy parameter, combined candidate parameter groups containing and not containing ID card numbers, and loaded them into the base model to obtain candidate models C4 (containing ID card numbers) and C5 (not containing ID card numbers). The test showed that the AUC value of candidate model C4 was 0.83 and the AUC value of candidate model C5 was 0.81. The server selected candidate model C4 as the current model to continue the iteration. In the seventh to ninth iterations, the server sequentially checked the total number of model parameters, AUC value, and importance assessment value, fine-tuning the weights and structure of each modal sub-model. The AUC value of the model increased to 0.84, 0.84, and 0.85 respectively, and the total number of parameters remained within 6 (≤ threshold). However, the parameters of the model in the ninth iteration were not exactly the same as those in the eighth iteration, and the convergence condition was not met. In the tenth iteration, the server performed a final fine-tuning of the model, resulting in model M10. Model M10 included privacy parameters such as home address, patient name, blood routine white blood cell count, chest CT lesion location features, and fundus image features, with a total of 5 parameters (≤ threshold). The importance assessment values ​​of each parameter were 0.4, 0.58, 0.49, 0.57, and 0.54 respectively (all ≤ preset importance assessment threshold 0.6). At this point, the server checked the convergence status and found that model M9 from the previous optimization cycle contained the same privacy parameters as model M10, thus determining that the model had converged.

[0024] It is worth noting that in this embodiment of the invention, the AUC value, or Area Under the Receiver Operating Characteristic Curve, is a core indicator for evaluating the performance of the binary classification model. The ROC curve is plotted with the "true positive rate" (the proportion of privacy-risk samples correctly identified by the model out of all actual privacy-risk samples, reflecting the model's "accuracy" in identifying privacy risks) on the vertical axis and the "false positive rate" (the proportion of non-privacy-risk samples incorrectly classified as risk samples by the model, reflecting the model's "misclassification rate") on the horizontal axis. The AUC value ranges from 0 to 1; the closer the value is to 1, the stronger the model's ability to distinguish between "high-privacy-risk medical data" and "low-privacy-risk medical data," with 0.5 corresponding to a random guessing level. In this scheme, AUC is used to quantitatively evaluate the privacy-risk identification performance of the multimodal ensemble model and is a key basis for judging whether the model iteration is effective.

[0025] After model convergence, the server selects baseline privacy parameters from all target adaptive multimodal ensemble recognition models obtained during the iteration process. First, it determines the first target adaptive multimodal ensemble recognition model (models M1 to M10) whose total number of parameters does not exceed a preset threshold of 10. Then, it determines the second target adaptive multimodal ensemble recognition models (models M2, M3, M5, M10, etc.) whose importance assessment value for each privacy parameter does not exceed a preset importance assessment threshold of 0.6 within the first target model. Finally, it determines the target adaptive multimodal ensemble recognition model corresponding to the peak AUC (AUC value 0.85) within the second target model, and uses the privacy parameters contained in model M10 as the baseline privacy parameters. Subsequently, the server determines the baseline features identical to the baseline privacy parameters from the features of the medical data subject Zhang San. Zhang San's feature set includes textual features "Patient Name: Zhang San" and "Home Address: No. 123, XX Road, XX District, XX City," and numerical features "Blood Routine White Blood Cell Count: 5.6 × 10⁻⁶". 9The server compares the following features with baseline privacy parameters: “ / L”, “Chest CT image lesion location: ground-glass nodule in the apical segment of the right upper lobe”, and “Fundus image features: mild retinal artery sclerosis”. These features are then compared one by one to the baseline features, which are defined as baseline features. These baseline features cover text, numerical, and image modalities, and the complementary verification of multimodal features enhances the modality fusion recognition reliability of model M10. The server loads the baseline features into model M10. Text modal features are input into the natural language processing sub-model, outputting a 768-dimensional feature vector; numerical modal features are input into the statistical feature extraction sub-model, outputting a 64-dimensional feature vector; and image modal features are input into the convolutional neural network sub-model, outputting a 2048-dimensional feature vector. Model M10 fuses these multimodal features (text weight 0) through an attention mechanism. With a weighting of 0.3 (numerical weight 0.2, image weight 0.5), the output privacy risk confidence coefficient for Zhang San is 0.85 (range 0-1, higher values ​​indicate higher privacy leakage risk). The server presets a privacy risk confidence coefficient threshold of 0.7 (determined through ROC curve analysis, false positive rate 5%). Since 0.85 > 0.7, the server provides corresponding privacy protection decisions for Zhang San: de-identify the home address, changing "No. 123, XX Road, XX District, XX City" to "***, XX Road, XX District, XX City"; set access controls for the patient's name and blood routine white blood cell count, allowing only the attending physician and department head to access, and recording all access logs; store the raw data corresponding to the baseline features in an AES-256 encrypted database, with the key managed by a designated person in the hospital's information department.

[0026] In this embodiment of the invention, the optimization iteration is performed based on the convergence state, total number of parameters, AUC value and importance assessment of privacy parameters of the target adaptive multimodal integrated recognition model until the model converges. The privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the peak value of AUC are selected as the benchmark privacy parameters from the target adaptive multimodal integrated recognition model obtained during the iteration process. This can be implemented through the following example.

[0027] When it is determined that the target adaptive multimodal ensemble recognition model does not meet the model convergence condition, the first AUC value corresponding to the current adaptive multimodal ensemble recognition model and the second AUC value corresponding to the target adaptive multimodal ensemble recognition model are determined. When the total number of parameters in the target adaptive multimodal ensemble recognition model is not greater than a preset total parameter threshold, the second AUC value is greater than the first AUC value, and the importance assessment value of each privacy parameter in the target adaptive multimodal ensemble recognition model is not greater than a preset importance assessment threshold, the target adaptive multimodal ensemble recognition model is taken as the current adaptive multimodal ensemble recognition model, and the step of determining the key privacy parameters among the multiple privacy parameters to be analyzed is repeated; When the total number of parameters exceeds the preset threshold, multiple candidate adaptive multimodal integrated recognition models are generated based on each privacy parameter in the target adaptive multimodal integrated recognition model. The model corresponding to the AUC peak value among the multiple candidate adaptive multimodal integrated recognition models is used as the optimized target adaptive multimodal integrated recognition model. The optimized target adaptive multimodal integrated recognition model is used as the current adaptive multimodal integrated recognition model. The step of selecting key privacy parameters from the multiple privacy parameters to be analyzed is repeated. When the target adaptive multimodal ensemble recognition model is determined to meet the model convergence condition, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters.

[0028] In an embodiment of the present invention, for example, when the server performs model optimization iteration for the privacy identification task of medical data subject Zhang San, it first performs convergence determination using the target adaptive multimodal integrated identification model M2 as the object: because the privacy parameters (home address, patient name) of M2 are not completely consistent with those of the previous round current model M1 and the optimization rounds are only 2 rounds, which does not reach the preset upper limit of 10 rounds, it is determined that M2 does not meet the convergence condition; then, the first AUC value of M1 (0.65) and the second AUC value of M2 (0.7) are retrieved, and the total number of parameters of M2 is verified to be 2 (including home address, patient name, and patient name). If the two privacy parameters (name and patient name) are ≤ the preset total parameter threshold of 6, the second AUC value of 0.7 is greater than the first AUC value of 0.65, and the importance assessment values ​​of the two privacy parameters (home address 0.4, patient name 0.6) are both ≤ the preset importance assessment threshold of 0.6, the iteration conditions are met. The server updates M2 to the current adaptive multimodal ensemble recognition model and continues to select blood routine white blood cell count as a key privacy parameter from the second stage group of the priority queue of privacy parameters to be analyzed and load it into M2. After fine-tuning the numerical sub-model (adding L2 regularization), the target model M3 is obtained.

[0029] When iterating to the target model M7, the server detected that its total number of parameters was 7 (including home address, patient name, blood routine white blood cell count, chest CT lesion location features, fundus photo features, ID number, and doctor's consultation recording text), which is greater than the preset total parameter threshold of 6. At this time, the importance assessment value of each privacy parameter in M7 was calculated, and it was found that the ID number (0.8) > the preset importance assessment threshold of 0.6, and was marked as a redundant parameter. The remaining 6 parameters were selected privacy parameters. Then, multiple privacy parameter groups were combined: one group retained the ID number (total 7, exceeding the threshold, excluded), one group removed the ID number (total 6), and one group removed the doctor's consultation recording text (total 6), resulting in candidate models C6 (containing 6 selected parameters) and C7 (containing 5 parameters). These parameter groups were loaded into the basic adaptive multimodal ensemble recognition model (the initial model without privacy parameters). The test showed that the AUC value of C6 was 0.82 and that of C7 was 0.81. The server selected C6, which corresponds to the peak AUC, as the optimized target adaptive multimodal ensemble recognition model and updated it to the current model for further iteration.

[0030] When iterating to the target model M10, the server detects that its privacy parameters are completely consistent with the previous model M9 (all are home address, patient name, blood routine white blood cell count, chest CT lesion location features, and fundus image features), and determines that the convergence condition is met. Then, it selects the model corresponding to the peak AUC from all the target adaptive multimodal integrated recognition models (M1 to M10) obtained during the iteration process: it traverses the AUC values ​​of all models (M1 is 0.65, M2 is 0.7, M3 is 0.72, M6 is 0.83, and M10 is 0.85), determines that the AUC value of 0.85 of M10 is the peak value, and finally uses the privacy parameters (home address, patient name, blood routine white blood cell count, chest CT lesion location features, and fundus image features) contained in M10 as the baseline privacy parameters.

[0031] In this embodiment of the invention, the step of generating multiple candidate adaptive multimodal integrated recognition models based on each privacy parameter in the target adaptive multimodal integrated recognition model can be implemented through the following example.

[0032] When it is determined that a privacy parameter with an importance evaluation value greater than a preset importance evaluation threshold is detected in the target adaptive multimodal integrated recognition model, the privacy parameter with an importance evaluation value greater than the preset importance evaluation threshold in the target adaptive multimodal integrated recognition model is used as a redundant privacy parameter; The privacy parameter whose importance evaluation value in the target adaptive multimodal integrated recognition model is not greater than the preset importance evaluation threshold is selected as the privacy parameter; Multiple privacy parameter groups are generated based on the redundant privacy parameters and the selected privacy parameters, and each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model to obtain multiple candidate adaptive multimodal ensemble recognition models. The basic adaptive multimodal ensemble recognition model is an adaptive multimodal ensemble recognition model without privacy parameters.

[0033] In an embodiment of the invention, for example, when the server iterates the privacy identification task for medical data subject Zhang San to the target adaptive multimodal integrated recognition model M7, it first calls the built-in feature importance verification module to evaluate each of the seven privacy parameters (home address, patient name, blood routine white blood cell count, chest CT lesion location features, fundus photograph features, ID number, and doctor's consultation recording text) contained in M7. At this time, the preset importance evaluation threshold is 0.6, and the preset total parameter threshold is 6. The server detects that the importance evaluation value of the ID number is 0.8 (higher than 0.6), and then determines that there are privacy parameters in M7 with an importance evaluation value greater than the threshold, and marks the ID number as a redundant privacy parameter; at the same time, it confirms that the importance evaluation values ​​of the remaining 6 parameters (home address 0.4, patient name 0.6, blood routine white blood cell count 0.5, chest CT lesion location features 0.57, fundus photograph features 0.54, and doctor's consultation recording text 0.2) are all no greater than 0.6, and determines them as selected privacy parameters.

[0034] The server then combines multiple compliant privacy parameter groups based on redundancy and selected privacy parameters: the first group retains all 6 selected parameters (total parameters 6, meeting the threshold requirement); the second group removes the doctor's consultation audio text (total parameters 5); and the third group removes fundus image features (total parameters 5). The server loads these three sets of parameters into the basic adaptive multimodal ensemble recognition model (the initial model without privacy parameters). By fine-tuning each modality sub-model (the fully connected layer of BERT in the text sub-model, the L2 regularization term in the numerical sub-model, and the dropout layer in the image sub-model), candidate adaptive multimodal ensemble recognition models C6, C7, and C8 are obtained. Test results show that C6 has an AUC of 0.82, C7 has 0.81, and C8 has 0.8. The server ultimately selects C6, corresponding to the peak AUC, as the optimized target adaptive multimodal ensemble recognition model and continues the subsequent iteration process.

[0035] In this embodiment of the invention, the method of combining multiple candidate adaptive multimodal integrated recognition models based on each privacy parameter in the target adaptive multimodal integrated recognition model is further provided by the following implementation method.

[0036] When no privacy parameter with an importance evaluation value greater than a preset importance evaluation threshold is detected in the target adaptive multimodal integrated recognition model, multiple privacy parameter groups are generated based on each privacy parameter in the target adaptive multimodal integrated recognition model; Each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model to obtain multiple candidate adaptive multimodal ensemble recognition models.

[0037] In an embodiment of the invention, for example, when the server iterates to the target adaptive multimodal integrated recognition model M5, it first performs an importance check on the five privacy parameters it contains (home address, patient name, blood routine white blood cell count, chest CT lesion location features, and fundus image features). The preset importance assessment threshold is 0.6. The detection finds that the importance assessment values ​​of all parameters (home address 0.4, patient name 0.6, blood routine white blood cell count 0.5, chest CT lesion location features 0.57, fundus image features 0.54) are all no greater than 0.6, and it is determined that no privacy parameters exceeding the threshold are detected. Subsequently, the server combines multiple privacy parameter groups based on these parameters: the first group retains all five parameters, the second group removes the blood routine white blood cell count, and the third group removes the fundus image features. Next, these three sets of parameters were loaded into the basic adaptive multimodal ensemble recognition model (the initial model without privacy parameters). By fine-tuning the weights of each modality sub-model (learning rate of 0.001 for the text sub-model and dropout ratio of 0.3 for the image sub-model), candidate adaptive multimodal ensemble recognition models C3, C4, and C5 were obtained. Test results showed that the AUC value of C3 was 0.82, C4 was 0.81, and C5 was 0.8, and the server completed the generation of candidate models.

[0038] In this embodiment of the invention, the target adaptive multimodal integrated recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal integrated recognition model obtained from the iteration process, and the privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters. This can be implemented through the following example.

[0039] From the target adaptive multimodal integrated recognition model obtained during the iteration process, a first target adaptive multimodal integrated recognition model with a total number of parameters not greater than a preset total parameter threshold is determined; A second target adaptive multimodal integrated recognition model is determined in the first target adaptive multimodal integrated recognition model such that the importance evaluation value of each privacy parameter is not greater than a preset importance evaluation threshold; In the second target adaptive multimodal ensemble recognition model, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak is determined, and the privacy parameter of the target adaptive multimodal ensemble recognition model corresponding to the AUC peak is used as the benchmark privacy parameter.

[0040] In an embodiment of the present invention, for example, the server, for the privacy identification iteration process of medical data subject Zhang San, first retrieves all target adaptive multimodal integrated recognition models (including M1 to M10) generated during the iteration process, and performs the first target model screening: the preset total parameter threshold is 6, and it is found that the total parameter of M7 is 7 (including 7 parameters such as ID number), which exceeds the threshold. The total parameter of the remaining models (M1-M6, M8-M10) is ≤6. These models are determined as the first target adaptive multimodal integrated recognition models.

[0041] Next, the server verifies the importance of privacy parameters for each of the first target models: the preset importance assessment threshold is 0.6. The detection found that the importance assessment value of the chest CT lesion location features contained in M4 is 0.75 (exceeding the threshold). The importance assessment values ​​of all privacy parameters of the other first target models (M1-M3, M5-M6, M8-M10) are ≤0.6 (such as home address 0.4, patient name 0.6, blood routine white blood cell count 0.5 for M5, chest CT lesion location features 0.57, fundus photo features 0.54 for M10, etc.). These models are determined as the second target adaptive multimodal integrated recognition models.

[0042] Subsequently, the server extracted the AUC values ​​of the second target model: M1 was 0.65, M2 was 0.7, M3 was 0.72, M5 was 0.8, M6 was 0.82, M8 was 0.83, M9 was 0.84, and M10 was 0.85. By comparison, the peak AUC value of 0.85 was determined, corresponding to model M10. Finally, the server determined the privacy parameters contained in M10 (home address, patient name, white blood cell count, chest CT lesion location features, and fundus image features) as the baseline privacy parameters.

[0043] In this embodiment of the invention, the following implementation methods are also provided.

[0044] After generating the target adaptive multimodal integrated recognition model in each iteration, the target adaptive multimodal integrated recognition model of the previous optimization cycle is obtained; When the privacy parameters contained in the target adaptive multimodal ensemble recognition model of the previous optimization cycle are the same as those contained in the target adaptive multimodal ensemble recognition model, it is determined that the target adaptive multimodal ensemble recognition model satisfies the model convergence condition. The target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters.

[0045] In this embodiment of the invention, for example, the server executes an iterative process for the privacy identification task of medical data subject Zhang San. After generating a target adaptive multimodal integrated identification model in each round, the server automatically retrieves the target model from the previous optimization cycle. When the target model M10 is generated in the 10th round, the server obtains the target model M9 from the previous cycle and compares the privacy parameters contained in the two: the privacy parameters of M9 are home address, patient name, blood routine white blood cell count, chest CT lesion location features, and fundus image features; the privacy parameters of M10 are completely consistent with those of M9. The server then determines that M10 meets the model convergence condition and immediately retrieves the AUC values ​​of all target models (M1 to M10) during the iteration process, where the AUC value of M10 is 0.85 (the peak value among all models). The server finally determines the privacy parameters contained in M10 as the baseline privacy parameters.

[0046] In this embodiment of the invention, the following implementation methods are also provided.

[0047] After generating the target adaptive multimodal integrated recognition model in each iteration, the optimization round corresponding to the target adaptive multimodal integrated recognition model is determined; When the optimization round reaches the preset optimization round, it is determined that the target adaptive multimodal ensemble recognition model satisfies the model convergence condition. The target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters.

[0048] In this embodiment of the invention, for example, the server presets 10 optimization rounds for the privacy identification task of medical data subject Zhang San. After generating the target adaptive multimodal ensemble recognition model in each round, the server automatically marks the optimization round corresponding to the current model: iterating to generate M1 marks round 1, generating M2 marks round 2, until the target model M10 in the 10th round is generated, at which point the server determines that its corresponding optimization round is 10, reaching the preset optimization round. At this time, the server directly determines that M10 meets the model convergence condition, and then retrieves the AUC values ​​of all target adaptive multimodal ensemble recognition models (M1 to M10) during the iteration process: M1 is 0.65, M2 is 0.7, M3 is 0.72, M5 is 0.8, M6 is 0.82, M8 is 0.83, M9 is 0.84, and M10 is 0.85. After comparison with the server, the peak AUC was determined to be 0.85, corresponding to model M10. Finally, the privacy parameters included in M10 (home address, patient name, blood routine white blood cell count, chest CT lesion location characteristics, fundus image characteristics) were determined as the baseline privacy parameters.

[0049] In this embodiment of the invention, the following implementation methods are also provided.

[0050] When it is determined that the total number of parameters in the target adaptive multimodal integrated recognition model is not greater than the preset total number of parameters threshold, if the second AUC value is not greater than the first AUC value, or if the importance assessment value of any privacy parameter in the target adaptive multimodal integrated recognition model is greater than the preset importance assessment threshold, then the step of filtering out key privacy parameters from the multiple privacy parameters to be analyzed is repeated. The current adaptive multimodal ensemble recognition model is optimized based on the key privacy parameters to obtain the target adaptive multimodal ensemble recognition model.

[0051] In an embodiment of the invention, for example, when the server iteratively generates a target adaptive multimodal integrated recognition model M4 for privacy identification of medical data subject Zhang San, it first verifies that the total number of parameters is 4 (including home address, patient name, blood routine white blood cell count, and chest CT lesion location features), confirming that it is ≤ the preset total parameter threshold of 6. Then, it retrieves the first AUC value of the previous current model M3 (0.72) and calculates the second AUC value of M4 (0.71, ≤0.72). At the same time, it detects that the importance assessment value of the chest CT lesion location features (0.75) is > the preset importance assessment threshold of 0.6. The server determines that the repeated screening condition is met, immediately returns to the second stage group of the privacy parameter priority queue to be analyzed, re-screens fundus image features (importance assessment value 0.54 ≤ 0.6) as key privacy parameters, replaces the original chest CT lesion location features and loads them into the current model M3, fine-tunes the dropout layer ratio of the image sub-model to 0.35, and optimizes to obtain a new target adaptive multimodal integrated recognition model M4-1.

[0052] In this embodiment of the invention, before using the target adaptive multimodal integrated recognition model as the current adaptive multimodal integrated recognition model, the following implementation methods are also provided.

[0053] When a model identical to the target adaptive multimodal integrated recognition model is detected in the target adaptive multimodal integrated recognition model obtained during the determination iteration process, the step of filtering out key privacy parameters from the multiple privacy parameters to be analyzed is repeated; The step of using the target adaptive multimodal ensemble recognition model as the current adaptive multimodal ensemble recognition model includes: When no model identical to the target adaptive multimodal ensemble recognition model is detected in the target adaptive multimodal ensemble recognition model obtained during the iteration process, the target adaptive multimodal ensemble recognition model is taken as the current adaptive multimodal ensemble recognition model.

[0054] In this embodiment of the invention, for example, after the server iteratively generates the target adaptive multimodal integrated recognition model M5-1, it first retrieves all target models (M1 to M5) stored during the iteration process for consistency verification: the verification dimensions include privacy parameter combinations, sub-model structure and weights. The results show that the privacy parameters of M5-1 (home address, patient name, blood routine white blood cell count, fundus photo features) are completely consistent with the historical model M5, and the fully connected layer weights of the text sub-model BERT (0.32), the L2 regularization coefficient of the numerical sub-model (0.01), and the dropout ratio of the image sub-model (0.3) are all no different from M5. The server determines that there are identical models in the iteration process.

[0055] At this point, the server immediately returns to the first stage group of the priority queue for privacy parameters to be analyzed, and re-screens key privacy parameters: the original unscheduled liver and kidney function indicators (importance assessment value 0.3) replace the fundus photo features in the original M5-1, load them into the current adaptive multimodal ensemble recognition model M4, and fine-tune the numerical sub-model (adjust the L2 regularization coefficient to 0.02) to generate a new target adaptive multimodal ensemble recognition model M5-2.

[0056] The server performed a second validation on M5-2: its privacy parameter combination included home address, patient name, blood routine white blood cell count, and liver and kidney function indicators. The sub-model weights were updated to 0.35 for the text sub-model and 0.02 for the numerical sub-model regularization coefficient. No matching entries were found in the iteration history model library, confirming that no identical model was detected. The server then adopted M5-2 as the current adaptive multimodal ensemble recognition model and continued the subsequent iteration process.

[0057] In this embodiment of the invention, before filtering out key privacy parameters from the plurality of privacy parameters to be analyzed, the following implementation method is also provided.

[0058] Among the plurality of privacy parameters to be analyzed, the importance assessment value corresponding to each privacy parameter to be analyzed is determined; Each privacy parameter to be analyzed is organized in ascending order according to the importance assessment value to obtain a priority queue of privacy parameters to be analyzed; The multiple privacy parameters to be analyzed are scheduled in stages according to the priority queue of the privacy parameters to be analyzed, resulting in multiple groups of privacy parameters to be analyzed in multiple processing stages.

[0059] In an exemplary embodiment of the invention, for the privacy identification task of medical data subject Zhang San, the server first performs a privacy parameter preprocessing process before screening key privacy parameters. The server first retrieves all privacy parameters of Zhang San to be analyzed, including text modal patient name, ID number, home address, and doctor's consultation recording converted to text; numerical modal blood routine white blood cell count and liver and kidney function indicators; and image modal chest CT lesion location features and fundus photograph features.

[0060] Next, the server calls the Random Forest Feature Importance Algorithm to quantify the privacy recognition contribution of each privacy parameter to be analyzed: ID number (0.8), chest CT lesion location features (0.75), patient name (0.6), fundus photo features (0.55), blood routine white blood cell count (0.5), home address (0.4), liver and kidney function indicators (0.3), and doctor's consultation recording converted to text (0.2), to obtain the importance assessment value corresponding to each parameter.

[0061] Subsequently, the server organizes all privacy parameters to be analyzed in ascending order according to their importance assessment values, and generates a priority queue of privacy parameters to be analyzed: Doctor's consultation recording converted to text (0.2) → Liver and kidney function indicators (0.3) → Home address (0.4) → Blood routine white blood cell count (0.5) → Fundus photograph features (0.55) → Patient name (0.6) → Chest CT lesion location features (0.75) → ID number (0.8).

[0062] Finally, the server performs phased scheduling based on the importance gradient of the priority queue: parameters with importance assessment values ​​in the range of 0.2-0.4 are assigned to the first processing stage, corresponding to the parameter group of doctor's consultation recording converted to text, liver and kidney function indicators, and home address; parameters in the range of 0.5-0.6 are assigned to the second processing stage, corresponding to the parameter group of blood routine white blood cell count, fundus image features, and patient name; and parameters in the range of 0.7-0.8 are assigned to the third processing stage, corresponding to the parameter group of chest CT lesion location features and ID number. This yields the privacy parameter groups to be analyzed in the three processing stages, providing a scheduling basis for subsequent phased screening of key privacy parameters.

[0063] In this embodiment of the invention, the step of filtering out key privacy parameters from the plurality of privacy parameters to be analyzed can be implemented through the following example: Determine the group of privacy parameters to be analyzed corresponding to the current processing stage; When an unscheduled privacy parameter is detected in the group of privacy parameters to be analyzed corresponding to the current processing stage, key privacy parameters are selected from the unscheduled privacy parameters to be analyzed. If no unscheduled privacy parameters are detected in the privacy parameter group to be analyzed in the current processing stage, then the key privacy parameters are selected from the privacy parameter group to be analyzed in the next processing stage.

[0064] In an embodiment of the invention, for example, when the server performs a key privacy parameter screening process for the privacy identification task of medical data subject Zhang San, it first retrieves the pre-generated phased results of the privacy parameters to be analyzed: the parameter group corresponding to the first processing stage is the doctor's consultation recording converted to text, liver and kidney function indicators, and home address (importance range 0.2-0.4); the parameter group corresponding to the second processing stage is the blood routine white blood cell count, fundus photograph features, and patient name (importance range 0.5-0.6); and the parameter group corresponding to the third processing stage is the chest CT lesion location features and ID number (importance range 0.7-0.8). The server determines that the current processing stage is the first processing stage by reading the iteration status flag.

[0065] Next, the server retrieves the scheduling log of the privacy parameters to be analyzed and checks the unscheduled parameters in the privacy parameter group corresponding to the first processing stage: the log shows that the doctor's consultation recording converted to text (importance 0.2) and liver and kidney function indicators (importance 0.3) have been scheduled in previous iterations, only the home address (importance 0.4) has not been scheduled. The server then focuses on the unscheduled home address, and combines privacy-sensitive attribute verification (home address is directly associated with personal identity and belongs to high-risk privacy information) and model adaptability analysis (text modal parameters can be directly input into the natural language processing sub-model), and filters the home address as a key privacy parameter.

[0066] After all privacy parameters to be analyzed in the first processing stage have been scheduled, the server retrieves the scheduling log again. If it detects that there are no unscheduled parameters in the parameter group corresponding to the first processing stage, it automatically switches to the second processing stage. The server performs an unscheduled check on the parameter group corresponding to the second processing stage: the log shows that the blood routine white blood cell count (importance 0.5) and fundus photograph features (importance 0.55) have not been scheduled, while the patient name (importance 0.6) has been scheduled. The server prioritizes the blood routine white blood cell count among the unscheduled parameters, combining its numerical modal attributes (which can be directly input into the statistical feature extraction sub-model) and privacy relevance (although blood routine indicators are medical data, they can easily form a privacy profile when combined with other parameters), and selects the blood routine white blood cell count as a key privacy parameter.

[0067] If all parameters in the second processing stage are scheduled, the server will switch to the third processing stage to perform unscheduled detection and key parameter screening on the location features of chest CT lesions and ID numbers, ensuring that the key screening of all privacy parameters to be analyzed is completed in an orderly manner according to the phased scheduling logic.

[0068] This invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned privacy data intelligent identification method based on deep multimodal fusion. Figure 2 As shown, Figure 2 This is a structural block diagram of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or interaction, the memory 111, processor 112, and communication unit 113 are electrically connected to each other directly or indirectly. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0069] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the foregoing illustrative discussions are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in accordance with the foregoing teachings. These embodiments were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and to employ various embodiments with different modifications to suit a particular intended application.

Claims

1. A privacy data intelligent identification method based on deep multimodal fusion, characterized in that, include: Multiple privacy parameters to be analyzed are obtained from the medical data subject. These multiple privacy parameters correspond to the privacy-sensitive information and related multimodal features of the medical data subject. Different privacy parameters correspond to different modality categories. Key privacy parameters are selected from the multiple privacy parameters to be analyzed. The current adaptive multimodal integrated recognition model is optimized based on the key privacy parameters to obtain the target adaptive multimodal integrated recognition model. Based on the convergence state, total number of parameters, AUC value, and importance assessment of privacy parameters of the target adaptive multimodal integrated recognition model, optimization iteration is performed until the model converges. The privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the peak AUC value are selected as the benchmark privacy parameters from the target adaptive multimodal integrated recognition model obtained during the iteration process. A benchmark feature identical to the benchmark privacy parameter is determined from the features of the medical data subject. The benchmark feature is composed of at least one feature corresponding to the medical data subject. The benchmark feature is used to enhance the modality fusion recognition reliability of the target adaptive multimodal integrated recognition model corresponding to the AUC peak. The benchmark features are loaded into the target adaptive multimodal ensemble recognition model corresponding to the peak AUC to obtain the privacy risk confidence coefficient of the medical data subject. Based on the privacy risk confidence coefficient, it is determined whether to provide corresponding privacy protection decisions for the medical data subject. The privacy risk confidence coefficient is used to assess the privacy risk confidence level of the medical data subject.

2. The method according to claim 1, characterized in that, The optimization iteration is performed based on the convergence state, total number of parameters, AUC value, and importance assessment of privacy parameters of the target adaptive multimodal ensemble recognition model until the model converges. From the target adaptive multimodal ensemble recognition models obtained during the iteration process, the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak are selected as benchmark privacy parameters, including: When it is determined that the target adaptive multimodal ensemble recognition model does not meet the model convergence condition, the first AUC value corresponding to the current adaptive multimodal ensemble recognition model and the second AUC value corresponding to the target adaptive multimodal ensemble recognition model are determined. When the total number of parameters in the target adaptive multimodal ensemble recognition model is not greater than a preset total parameter threshold, the second AUC value is greater than the first AUC value, and the importance assessment value of each privacy parameter in the target adaptive multimodal ensemble recognition model is not greater than a preset importance assessment threshold, the target adaptive multimodal ensemble recognition model is taken as the current adaptive multimodal ensemble recognition model, and the step of determining the key privacy parameters among the multiple privacy parameters to be analyzed is repeated; When the total number of parameters exceeds the preset threshold, multiple candidate adaptive multimodal integrated recognition models are generated based on each privacy parameter in the target adaptive multimodal integrated recognition model. The model corresponding to the AUC peak value among the multiple candidate adaptive multimodal integrated recognition models is used as the optimized target adaptive multimodal integrated recognition model. The optimized target adaptive multimodal integrated recognition model is used as the current adaptive multimodal integrated recognition model. The step of selecting key privacy parameters from the multiple privacy parameters to be analyzed is repeated. When the target adaptive multimodal ensemble recognition model is determined to meet the model convergence condition, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters.

3. The method according to claim 2, characterized in that, The step of generating multiple candidate adaptive multimodal ensemble recognition models based on each privacy parameter in the target adaptive multimodal ensemble recognition model includes: When it is determined that a privacy parameter with an importance evaluation value greater than a preset importance evaluation threshold is detected in the target adaptive multimodal integrated recognition model, the privacy parameter with an importance evaluation value greater than the preset importance evaluation threshold in the target adaptive multimodal integrated recognition model is used as a redundant privacy parameter; The privacy parameter whose importance evaluation value in the target adaptive multimodal integrated recognition model is not greater than the preset importance evaluation threshold is selected as the privacy parameter; Multiple privacy parameter groups are generated based on the redundant privacy parameters and the selected privacy parameters, and each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model to obtain multiple candidate adaptive multimodal ensemble recognition models. The basic adaptive multimodal ensemble recognition model is an adaptive multimodal ensemble recognition model without privacy parameters.

4. The method according to claim 3, characterized in that, The step of combining multiple candidate adaptive multimodal ensemble recognition models based on each privacy parameter in the target adaptive multimodal ensemble recognition model further includes: When no privacy parameter with an importance evaluation value greater than a preset importance evaluation threshold is detected in the target adaptive multimodal integrated recognition model, multiple privacy parameter groups are generated based on each privacy parameter in the target adaptive multimodal integrated recognition model; Each privacy parameter group is loaded into the basic adaptive multimodal ensemble recognition model to obtain multiple candidate adaptive multimodal ensemble recognition models.

5. The method according to claim 2, characterized in that, The target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained from the iteration process. The privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the baseline privacy parameters, including: From the target adaptive multimodal integrated recognition model obtained during the iteration process, a first target adaptive multimodal integrated recognition model with a total number of parameters not greater than a preset total parameter threshold is determined; A second target adaptive multimodal integrated recognition model is determined in the first target adaptive multimodal integrated recognition model such that the importance evaluation value of each privacy parameter is not greater than a preset importance evaluation threshold; In the second target adaptive multimodal ensemble recognition model, the target adaptive multimodal ensemble recognition model corresponding to the AUC peak is determined, and the privacy parameter of the target adaptive multimodal ensemble recognition model corresponding to the AUC peak is used as the benchmark privacy parameter.

6. The method according to claim 2, characterized in that, The method further includes: After generating the target adaptive multimodal integrated recognition model in each iteration, the target adaptive multimodal integrated recognition model of the previous optimization cycle is obtained; When the privacy parameters contained in the target adaptive multimodal ensemble recognition model of the previous optimization cycle are the same as those contained in the target adaptive multimodal ensemble recognition model, it is determined that the target adaptive multimodal ensemble recognition model satisfies the model convergence condition. The target adaptive multimodal ensemble recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal ensemble recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal ensemble recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters. The method further includes: After generating the target adaptive multimodal integrated recognition model in each iteration, the optimization round corresponding to the target adaptive multimodal integrated recognition model is determined; When the optimization round reaches the preset optimization round, it is determined that the target adaptive multimodal integrated recognition model satisfies the model convergence condition. The target adaptive multimodal integrated recognition model corresponding to the AUC peak value is selected from the target adaptive multimodal integrated recognition models obtained during the iteration process, and the privacy parameters contained in the target adaptive multimodal integrated recognition model corresponding to the AUC peak value are used as the benchmark privacy parameters. The method further includes: When it is determined that the total number of parameters in the target adaptive multimodal integrated recognition model is not greater than the preset total number of parameters threshold, if the second AUC value is not greater than the first AUC value, or if the importance assessment value of any privacy parameter in the target adaptive multimodal integrated recognition model is greater than the preset importance assessment threshold, then the step of filtering out key privacy parameters from the multiple privacy parameters to be analyzed is repeated. The current adaptive multimodal ensemble recognition model is optimized based on the key privacy parameters to obtain the target adaptive multimodal ensemble recognition model.

7. The method according to claim 2, characterized in that, Before using the target adaptive multimodal ensemble recognition model as the current adaptive multimodal ensemble recognition model, the method further includes: When a model identical to the target adaptive multimodal integrated recognition model is detected in the target adaptive multimodal integrated recognition model obtained during the determination iteration process, the step of filtering out key privacy parameters from the multiple privacy parameters to be analyzed is repeated; The step of using the target adaptive multimodal ensemble recognition model as the current adaptive multimodal ensemble recognition model includes: When no model identical to the target adaptive multimodal ensemble recognition model is detected in the target adaptive multimodal ensemble recognition model obtained during the iteration process, the target adaptive multimodal ensemble recognition model is taken as the current adaptive multimodal ensemble recognition model.

8. The method according to claim 1, characterized in that, Before filtering out key privacy parameters from the multiple privacy parameters to be analyzed, the process also includes: Among the plurality of privacy parameters to be analyzed, the importance assessment value corresponding to each privacy parameter to be analyzed is determined; Each privacy parameter to be analyzed is organized in ascending order according to the importance assessment value to obtain a priority queue of privacy parameters to be analyzed; The multiple privacy parameters to be analyzed are scheduled in stages according to the priority queue of the privacy parameters to be analyzed, resulting in multiple groups of privacy parameters to be analyzed in multiple processing stages.

9. The method according to claim 8, characterized in that, The process of filtering out key privacy parameters from the plurality of privacy parameters to be analyzed includes: Determine the set of privacy parameters to be analyzed corresponding to the current processing stage; When an unscheduled privacy parameter is detected in the group of privacy parameters to be analyzed corresponding to the current processing stage, key privacy parameters are selected from the unscheduled privacy parameters to be analyzed. If no unscheduled privacy parameters are detected in the privacy parameter group to be analyzed in the current processing stage, then the key privacy parameters are selected from the privacy parameter group to be analyzed in the next processing stage.

10. A server system, characterized in that, Includes a server, the server being used to perform the method according to any one of claims 1-9.

Images