Model robustness enhancement method and communication apparatus

Abstract

Embodiments of the present application relate to the field of communications, provide a model robustness enhancement method and a communication apparatus, and can improve the robustness of an AI / ML model. The method comprises: a model training function can send detected abnormal data to an NDT system for data enhancement, acquire, from the NDT system, robustness enhancement data corresponding to the abnormal data, and input the robustness enhancement data corresponding to the abnormal data as adversarial data into a model for adversarial training, so that a target model automatically learns prior knowledge of an adversary attack and prior knowledge of a feature domain drift trajectory of the target model itself in a training process, thereby improving the robustness of the model, such as attack defense robustness and feature domain drift robustness of the model.

Description

Model robustness enhancement method and communication device

[0001] The present application claims priority from the Chinese patent application No. 202410775640.3 filed on June 14, 2024, and entitled "Robustness Enhancement Method and Communication Device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of communication, in particular to a model robustness enhancement method and a communication device. BACKGROUND

[0003] The life cycle management of artificial intelligence (AI) / machine learning (ML) models in wireless networks includes model training, inference, performance evaluation, deployment, testing, updating, etc., but at each stage of its life cycle, it will face security threats such as poisoning attacks, theft attacks and adversarial attacks. Therefore, the 3rd generation partnership project (3GPP) has made clear requirements for the trusted management of models: trusted management should be added in the model training, testing and inference stages to enhance the robustness and security of AI / ML models, and AI / ML models should be updated and maintained regularly to ensure that the model still maintains high prediction accuracy when the data distribution changes (which may be caused by an adversary attack).

[0004] The AI / ML model training and retraining stage needs to collect training data to update the model parameters, and the AI / ML model training data is generally taken from the existing network. There is a security risk that an adversary will cause the accuracy of the trained model to decrease by injecting toxic sample points into the data collected from the existing network. Therefore, how to improve the robustness of AI / ML models has become a problem to be solved. SUMMARY

[0005] The present application provides a model robustness enhancement method and a communication device, which can improve the robustness of AI / ML models.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, a model robustness enhancement method is provided. The method can be performed by a model training function, a component of the model training function, such as a processor, a chip, or a chip system of the model training function, or a logic module or software that can implement all or part of the model training function. The method includes: the model training function sending abnormal data to a network digital twin (NDT) system, and receiving robustness enhancement data corresponding to the abnormal data from the NDT system, the robustness enhancement data corresponding to the abnormal data being used for model training.

[0008] In the method, the model training function can send the detected abnormal data to the NDT system for data enhancement, obtain the robustness enhancement data corresponding to the abnormal data from the NDT system, and input the robustness enhancement data corresponding to the abnormal data as adversarial data into a model for adversarial training, so that the target model automatically learns the prior knowledge of an adversary attack and the prior knowledge of a feature domain drift trajectory of the target model itself in the training process, thereby improving the robustness of the model, such as the attack defense robustness and the feature domain drift robustness of the model.

[0009] In a possible design, the method of the first aspect can further include: the model training function receiving first information from a trusted artificial intelligence (AI) management function, the first information being used to instruct the model training function to send abnormal data detected from a training data set to the NDT system for data enhancement; and the model training function sending the abnormal data to the NDT system, which can include: the model training function sending the abnormal data to the NDT system according to the first information. In this way, the model training function can trigger detection of the abnormal data and data enhancement processing of the abnormal data based on the first information sent by the trusted AI management function, to improve the robustness of the model.

[0010] In a possible design, the method of the first aspect can further include: the model training function sending first indication information to the NDT system, the first indication information being used to instruct the NDT system to perform data enhancement on the abnormal data. In this way, the model training function can also send the first indication information to the NDT system to trigger the NDT to perform data enhancement on the abnormal data.

[0011] In a possible design, the first indication information can include information of an algorithm used to perform data enhancement on the abnormal data.

[0012] In a possible design, the method of the first aspect can further include: the model training function performing model training according to the robustness enhancement data corresponding to the abnormal data.

[0013] In a possible design, the method of the first aspect can further include: the model training function sending high-uncertainty data to the NDT system, the high-uncertainty data being data in the training data set that causes the model to have a confidence level lower than a first preset threshold. The model training function receives robustness-enhanced data corresponding to the high-uncertainty data from the NDT system, and the robustness-enhanced data corresponding to the high-uncertainty data is used for model training. In this way, on the basis of improving the robustness of the model by using the robustness-enhanced data corresponding to the abnormal data, the model training function can also sample the high-uncertainty data for data enhancement, use robustness-enhanced data with more feature types to learn more prior knowledge, and further improve the generalization ability and robustness of the model.

[0014] In a possible design, the method of the first aspect can further include: the model training function receiving second information from the trusted AI management function, the second information being used to instruct the model training function to send high-uncertainty data sampled from the training data set to the NDT system for data enhancement; and the model training function sending the high-uncertainty data to the NDT system, which can include: the model training function sending the high-uncertainty data to the NDT system according to the second information. In this way, the model training function can also trigger the collection of the high-uncertainty data and the data enhancement processing of the high-uncertainty data based on the second information sent by the trusted AI management function, to further improve the robustness of the model.

[0015] In a possible design, the second information can include information used to instruct an algorithm for sampling the high-uncertainty data.

[0016] In a possible design, the method of the first aspect can further include: the model training function sending second indication information to the NDT system, the second indication information being used to instruct the NDT system to perform data enhancement on the high-uncertainty data. In this way, the model training function can also send the second indication information to the NDT system to trigger the NDT to perform the data enhancement on the high-uncertainty data.

[0017] In a possible design, the second indication information can include information of an algorithm used to perform the data enhancement on the high-uncertainty data.

[0018] In a possible design, the method of the first aspect can further include: the model training function performing model training according to the robustness-enhanced data corresponding to the abnormal data and the robustness-enhanced data corresponding to the high-uncertainty data.

[0019] In a possible design, the method in the first aspect can further include: the model training function performing clustering on the high-uncertainty data sampled from the training data set for fairness, to obtain data of different categories. The model training function samples data of a category in which the fairness bias is greater than a second preset threshold from the data of different categories, to obtain unfair category data. The model training function sends the unfair category data to the NDT system. The model training function receives robustness enhancement data corresponding to the unfair category data from the NDT system, and the robustness enhancement data corresponding to the unfair category data is used for model training. In this way, on the basis of improving the robustness of the model by using the robustness enhancement data corresponding to the abnormal data, the model training function can also sample the unfair category data for data enhancement, increase the adversarial training based on the fairness-based robustness enhancement data, enable the target model to automatically learn fairness-related knowledge in the (re)training process, better realize the value alignment of the target model, and finally realize the model robustness enhancement based on data fairness.

[0020] In a possible design, the method in the first aspect can further include: the model training function receiving third information from the trusted AI management function, where the third information is used to instruct the model training function to obtain the robustness enhancement data corresponding to the unfair category data from the NDT system for model training. The model training function sending the unfair category data to the NDT system can include: the model training function sending the unfair category data to the NDT system according to the third information. In this way, the model training function can also trigger the collection of the unfair category data and the data enhancement processing of the unfair category data based on the third information sent by the trusted AI management function, to further improve the robustness of the model.

[0021] In a possible design, the third information can include information used to instruct a clustering algorithm for fairness and information used to instruct an algorithm for sampling data after clustering for fairness.

[0022] In a possible design, the method in the first aspect can further include: the model training function sending third indication information to the NDT system, where the third indication information is used to instruct the NDT system to perform data enhancement on the unfair category data. In this way, the model training function can also send the third indication information to the NDT system, to trigger the NDT to perform the data enhancement of the unfair category data.

[0023] In a possible design, the third indication information can include information of an algorithm used to perform the data enhancement on the unfair category data.

[0024] In a possible design, the method of the first aspect can further include: the model training function performing model training according to the robustness-enhanced data corresponding to the abnormal data, the robustness-enhanced data corresponding to the high-uncertainty data, and the robustness-enhanced data corresponding to the unfair-class data.

[0025] In a possible design, the method of the first aspect can further include: the model training function sending, to the NDT system, model parameter information of the model after the model training, to align the models stored by the two parties in real time.

[0026] In a possible design, the method of the first aspect can further include: after performing model training according to the robustness-enhanced data, the model training function sending, to the NDT system, information indicating parameters for data enhancement that need to be updated, to further improve the performance of the data enhancement algorithm.

[0027] In a possible design, the method of the first aspect can further include: after performing model training according to the robustness-enhanced data, the model training function sending, to the NDT system, information indicating parameters for data enhancement that need to be updated, to further improve the performance of the data enhancement algorithm.

[0028] In a possible design, the method of the first aspect can further include: after performing model training according to the robustness-enhanced data, the model training function sending, to the NDT system, information indicating parameters for data enhancement that need to be updated, to further improve the performance of the data enhancement algorithm.

[0029] In a possible design, the method of the first aspect can further include: after performing model training according to the robustness-enhanced data, the model training function sending, to the NDT system, information indicating parameters for data enhancement that need to be updated, to further improve the performance of the data enhancement algorithm.

[0030] In a possible design, the method in the second aspect can further include: receiving, by the NDT system, high-uncertainty data from the model training function, the high-uncertainty data being data in the training data set that causes the model to have a confidence lower than a first preset threshold in prediction. performing, by the NDT system, data augmentation on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data, the robustness-enhanced data corresponding to the high-uncertainty data being used for model training. sending, by the NDT system, the robustness-enhanced data corresponding to the high-uncertainty data to the model training function.

[0031] In a possible design, performing, by the NDT system, data augmentation on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data can include: performing, by the NDT system, model prediction on the high-uncertainty data to obtain a first prediction result. determining, by the NDT system, a counterfactual sample generation algorithm according to the first prediction result. obtaining, by the NDT system, the robustness-enhanced data corresponding to the high-uncertainty data according to the counterfactual sample generation algorithm and the high-uncertainty data.

[0032] In a possible design, the method in the second aspect can further include: receiving, by the NDT system, second indication information from the model training function, the second indication information being used to instruct the NDT system to perform data augmentation on the high-uncertainty data. Performing, by the NDT system, data augmentation on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data can include: performing, by the NDT system, data augmentation on the high-uncertainty data according to the second indication information to obtain the robustness-enhanced data corresponding to the high-uncertainty data.

[0033] In a possible design, the method in the second aspect can further include: receiving, by the NDT system, unfair-class data from the model training function, the unfair-class data being data of a class with a fairness bias greater than a second preset threshold. Performing, by the NDT system, data augmentation on the unfair-class data to obtain robustness-enhanced data corresponding to the unfair-class data, the robustness-enhanced data corresponding to the unfair-class data being used for model training. Sending, by the NDT system, the robustness-enhanced data corresponding to the unfair-class data to the model training function.

[0034] In a possible design, performing, by the NDT system, data augmentation on the unfair-class data to obtain robustness-enhanced data corresponding to the unfair-class data can include: performing, by the NDT system, model prediction on the unfair-class data to obtain a second prediction result. Determining, by the NDT system, a fairness sample generation algorithm according to the second prediction result. Obtaining, by the NDT system, the robustness-enhanced data corresponding to the unfair-class data according to the fairness sample generation algorithm and the unfair-class data.

[0035] In a possible design, the method of the second aspect further can include: the NDT system receiving third indication information from the model training function, the third indication information being used to instruct the NDT system to perform data augmentation on the unfair category data. The NDT system performing data augmentation on the unfair category data to obtain robustness augmented data corresponding to the unfair category data can include: the NDT system performing data augmentation on the unfair category data according to the third indication information to obtain the robustness augmented data corresponding to the unfair category data.

[0036] In a possible design, the method of the second aspect further can include: the NDT system receiving model parameter information of a model updated by the model training function.

[0037] In a possible design, the method of the second aspect further can include: the NDT system receiving information about parameters for data augmentation that need to be updated from the model training function.

[0038] In a possible design, the method of the second aspect further can include: the NDT system receiving information about parameters for data augmentation from the trusted artificial intelligence (AI) management function, the information being used to instruct the NDT system to perform data augmentation.

[0039] The technical effects of the method of the second aspect can refer to the related descriptions of the technical effects of the method of the first aspect, and details are not repeated here.

[0040] In a third aspect, a model robustness augmentation method is provided. The method can be executed by a trusted AI management function, or a component of the trusted AI management function, such as a processor, a chip, or a chip system of the trusted AI management function, or a logic module or software that can implement all or part of the trusted AI management function. The method includes: the trusted AI management function obtaining first information, the first information being used to instruct a model training function to send abnormal data detected from a training data set to an NDT system for data augmentation. The trusted AI management function sends the first information to the model training function.

[0041] In the method, the trusted AI management function can trigger the model training function to send the detected abnormal data to the NDT for data augmentation instead of discarding the abnormal data, by sending the first information to the model training function. Thus, the model training function can obtain robustness augmented data corresponding to the abnormal data from the NDT system, and input the robustness augmented data corresponding to the abnormal data as adversarial data into a model for adversarial training. The target model can automatically learn prior knowledge of an enemy attack and prior knowledge of a feature domain drift trajectory of the target model itself in the training process, thereby improving the robustness of the model, such as attack defense robustness and feature domain drift robustness.

[0042] In a possible design, the method of the third aspect further can include: the trusted AI management function sending second information to the model training function, the second information being used to instruct the model training function to send high-uncertainty data sampled from the training data set to the NDT system for data enhancement, the high-uncertainty data being data in the training data set that causes the model prediction confidence to be lower than a first preset threshold. In this way, the trusted AI management function can further send second information to the model training function, triggering the model training function to collect high-uncertainty data and perform data enhancement processing on the high-uncertainty data, so as to further improve the robustness of the model.

[0043] In a possible design, the second information can include information used to instruct an algorithm for sampling high-uncertainty data.

[0044] In a possible design, the method of the third aspect further can include: the trusted AI management function sending third information to the model training function, the third information being used to instruct the model training function to perform model training from the NDT system by using robustness enhancement data corresponding to unfair category data, the unfair category data being data of a category with a fairness bias greater than a second preset threshold. In this way, the trusted AI management function can further send third information to the model training function, triggering the model training function to collect unfair category data and perform data enhancement processing on the unfair category data, so as to further improve the robustness of the model.

[0045] In a possible design, the third information can include information used to instruct a clustering algorithm targeting fairness, and information used to instruct an algorithm for sampling data after targeting fairness.

[0046] In a possible design, the method of the third aspect further can include: the trusted AI management function sending, to the NDT system, information used to instruct the NDT system to perform data enhancement.

[0047] In a fourth aspect, a communication apparatus is provided for implementing various methods described above. The communication apparatus can be the model training function in the first aspect, or an apparatus including the model training function, or an apparatus included in the model training function, such as a chip. The communication apparatus includes corresponding modules, units, or means for implementing the method of the first aspect, which can be implemented by hardware, software, or by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0048] In some possible design, the communication apparatus includes a processing module and a communication module. The communication module is configured to send abnormal data to a network digital twin (NDT) system, and receive robustness enhancement data corresponding to the abnormal data from the NDT system, the robustness enhancement data corresponding to the abnormal data being used for model training. The processing module is configured to perform the model training.

[0049] In a possible design, the communication module is further configured to receive first information from a trusted artificial intelligence (AI) management function, the first information being used to instruct a model training function to send abnormal data detected from a training data set to the NDT system for data enhancement. The communication module is configured to send the abnormal data to the NDT system, which can include that the communication module is configured to send the abnormal data to the NDT system according to the first information.

[0050] In a possible design, the communication module is further configured to send first indication information to the NDT system, the first indication information being used to instruct the NDT system to perform data enhancement on the abnormal data.

[0051] In a possible design, the first indication information can include information of an algorithm used to perform the data enhancement on the abnormal data.

[0052] In a possible design, the processing module is configured to perform the model training according to the robustness enhancement data corresponding to the abnormal data.

[0053] In a possible design, the communication module is further configured to send high-uncertainty data to the NDT system, the high-uncertainty data being data in the training data set that causes a confidence of a model prediction to be lower than a first preset threshold. The communication module is further configured to receive robustness enhancement data corresponding to the high-uncertainty data from the NDT system, the robustness enhancement data corresponding to the high-uncertainty data being used for model training.

[0054] In a possible design, the communication module is further configured to receive second information from a trusted AI management function, the second information being used to instruct a model training function to send high-uncertainty data sampled from the training data set to the NDT system for data enhancement. The communication module is further configured to send the high-uncertainty data to the NDT system, which can include that the communication module is configured to send the high-uncertainty data to the NDT system according to the second information.

[0055] In a possible design, the second information can include information of an algorithm used to sample the high-uncertainty data.

[0056] In a possible design, the communication module is further configured to send second indication information to the NDT system, the second indication information being used to instruct the NDT system to perform data enhancement on the high-uncertainty data.

[0057] In a possible design, the second indication information can include information of an algorithm for data enhancement on the high-uncertainty data.

[0058] In a possible design, the processing module is further configured to perform model training according to the robustness-enhanced data corresponding to the abnormal data and the robustness-enhanced data corresponding to the high-uncertainty data.

[0059] In a possible design, the processing module is further configured to perform clustering on the high-uncertainty data sampled from the training data set to obtain data of different categories, and to sample data of a category whose fairness deviation is greater than a second preset threshold from the data of different categories to obtain unfair category data. The communication module is further configured to send the unfair category data to the NDT system. The model training function receives robustness-enhanced data corresponding to the unfair category data from the NDT system, and the robustness-enhanced data corresponding to the unfair category data is used for model training.

[0060] In a possible design, the communication module is further configured to receive third information from the trusted AI management function, where the third information is used to instruct the model training function to perform model training by obtaining the robustness-enhanced data corresponding to the unfair category data from the NDT system. The communication module is further configured to send the unfair category data to the NDT system, which can include that the communication module is configured to send the unfair category data to the NDT system according to the third information.

[0061] In a possible design, the third information can include information of a clustering algorithm for fairness and information of an algorithm for sampling data after clustering for fairness.

[0062] In a possible design, the communication module is further configured to send third indication information to the NDT system, where the third indication information is used to instruct the NDT system to perform data enhancement on the unfair category data.

[0063] In a possible design, the third indication information can include information of an algorithm for data enhancement on the unfair category data.

[0064] In a possible design, the processing module is further configured to perform model training according to the robustness-enhanced data corresponding to the abnormal data, the robustness-enhanced data corresponding to the high-uncertainty data, and the robustness-enhanced data corresponding to the unfair category data.

[0065] In a possible design, the communication module is further configured to send model parameter information of the model training to the NDT system to real-time align the models stored by both parties.

[0066] In a possible design, the communication module is further configured to send, to the NDT system, information indicating parameters for data augmentation that need to be updated after the model training based on the robustness-enhanced data.

[0067] In a possible design, the communication module can include a receiving module and a sending module. The sending module is configured to implement the sending function of the communication apparatus in the fourth aspect, and the receiving module is configured to implement the receiving function of the communication apparatus in the fourth aspect.

[0068] In a possible design, the communication apparatus in the fourth aspect can further include a storage module that stores programs or instructions. When the processing module executes the programs or instructions, the communication apparatus in the fourth aspect can execute the method in the first aspect.

[0069] In the fifth aspect, a communication apparatus is provided for implementing the methods described above. The communication apparatus can be the NDT system in the second aspect, or an apparatus including the NDT system, or an apparatus included in the NDT system, such as a chip. The communication apparatus includes corresponding modules, units, or means for implementing the methods in the second aspect, which can be implemented by hardware, software, or by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0070] In some possible designs, the communication apparatus includes a processing module and a communication module. The communication module is configured to receive abnormal data from the model training function. The processing module is configured to perform data augmentation on the abnormal data to obtain robustness-enhanced data corresponding to the abnormal data, and the robustness-enhanced data corresponding to the abnormal data is used for model training. The communication module is further configured to send the robustness-enhanced data corresponding to the abnormal data to the model training function.

[0071] In a possible design, the processing module configured to perform data augmentation on the abnormal data to obtain robustness-enhanced data corresponding to the abnormal data can include: the processing module is configured to learn abnormal behavior based on the abnormal data. The NDT system generates the robustness-enhanced data corresponding to the abnormal data based on the abnormal behavior.

[0072] In a possible design, the communication module is further configured to receive first indication information from the model training function, and the first indication information is used to instruct the NDT system to perform data augmentation on the abnormal data. The processing module configured to perform data augmentation on the abnormal data to obtain robustness-enhanced data corresponding to the abnormal data can include: the processing module is configured to perform data augmentation on the abnormal data based on the first indication information to obtain the robustness-enhanced data corresponding to the abnormal data.

[0073] In a possible design, the communication module is further configured to receive high-uncertainty data from the model training function, the high-uncertainty data being data in the training data set that causes the model to have a confidence lower than a first preset threshold in prediction. The processing module is further configured to perform data augmentation on the high-uncertainty data to obtain robustness augmented data corresponding to the high-uncertainty data, and the robustness augmented data corresponding to the high-uncertainty data is used for model training. The communication module is further configured to send the robustness augmented data corresponding to the high-uncertainty data to the model training function.

[0074] In a possible design, the processing module is further configured to perform data augmentation on the high-uncertainty data to obtain robustness augmented data corresponding to the high-uncertainty data, which can include that the processing module is configured to perform model prediction according to the high-uncertainty data to obtain a first prediction result. The processing module is further configured to determine an counterfactual sample generation algorithm according to the first prediction result. The processing module is further configured to obtain the robustness augmented data corresponding to the high-uncertainty data according to the counterfactual sample generation algorithm and the high-uncertainty data.

[0075] In a possible design, the communication module is further configured to receive second indication information from the model training function, the second indication information being used to instruct the NDT system to perform data augmentation on the high-uncertainty data. The processing module configured to perform data augmentation on the high-uncertainty data to obtain robustness augmented data corresponding to the high-uncertainty data can include that the processing module is configured to perform data augmentation on the high-uncertainty data according to the second indication information to obtain the robustness augmented data corresponding to the high-uncertainty data.

[0076] In a possible design, the communication module is further configured to receive unfair category data from the model training function, the unfair category data being data of a category with a fairness bias greater than a second preset threshold. The processing module is further configured to perform data augmentation on the unfair category data to obtain robustness augmented data corresponding to the unfair category data, and the robustness augmented data corresponding to the unfair category data is used for model training. The communication module is further configured to send the robustness augmented data corresponding to the unfair category data to the model training function.

[0077] In a possible design, the processing module is further configured to perform data augmentation on the unfair category data to obtain robustness augmented data corresponding to the unfair category data, which can include that the processing module is configured to perform model prediction according to the unfair category data to obtain a second prediction result. The processing module is configured to determine a fairness sample generation algorithm according to the second prediction result. The processing module is configured to obtain the robustness augmented data corresponding to the unfair category data according to the fairness sample generation algorithm and the unfair category data.

[0078] In a possible design, the communication module is further configured to receive third indication information from the model training function, where the third indication information is used to instruct the NDT system to perform data augmentation on the unfair category data. The processing module is configured to perform data augmentation on the unfair category data to obtain robustness augmented data corresponding to the unfair category data, and can include: the processing module is configured to perform data augmentation on the unfair category data according to the third indication information to obtain the robustness augmented data corresponding to the unfair category data.

[0079] In a possible design, the communication module is further configured to receive model parameter information of a model updated by the model training function.

[0080] In a possible design, the communication module is further configured to receive information used to instruct parameters for data augmentation that need to be updated from the model training function.

[0081] In a possible design, the communication module is further configured to receive information used to instruct parameters for data augmentation of the NDT system from the trusted artificial intelligence (AI) management function.

[0082] In a possible design, the communication module can include a receiving module and a sending module. The sending module is configured to implement the sending function of the communication apparatus in the fifth aspect, and the receiving module is configured to implement the receiving function of the communication apparatus in the fifth aspect.

[0083] In a possible design, the communication apparatus in the fifth aspect can further include a storage module that stores programs or instructions. When the processing module executes the programs or instructions, the communication apparatus in the fifth aspect can execute the method in the second aspect.

[0084] In a sixth aspect, a communication apparatus is provided for implementing various methods described above. The communication apparatus can be the trusted AI management function in the third aspect, or an apparatus including the trusted AI management function, or an apparatus included in the trusted AI management function, such as a chip. The communication apparatus includes corresponding modules, units, or means for implementing the methods in the third aspect, which can be implemented by hardware, software, or by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0085] In some possible designs, the communication apparatus includes a processing module and a communication module. The processing module is configured to obtain first information used to instruct a model training function to send abnormal data detected from a training data set to an NDT system for data augmentation. The communication module is configured to send the first information to the model training function.

[0086] In a possible design, the communication module is further configured to send second information to the model training function, where the second information is used to instruct the model training function to send high-uncertainty data sampled from the training data set to the NDT system for data enhancement, and the high-uncertainty data is data in the training data set that causes the confidence of the model prediction to be lower than a first preset threshold.

[0087] In a possible design, the second information can include information used to instruct an algorithm for sampling the high-uncertainty data.

[0088] In a possible design, the communication module is further configured to send third information to the model training function, where the third information is used to instruct the model training function to perform model training on robustness enhancement data corresponding to unfair category data obtained from the NDT system, and the unfair category data is data of a category with a fairness deviation greater than a second preset threshold.

[0089] In a possible design, the third information can include information used to instruct a clustering algorithm targeting fairness and information used to instruct an algorithm for sampling data after the data is processed by the clustering algorithm targeting fairness.

[0090] In a possible design, the communication module is further configured to send information used to instruct the NDT system to perform data enhancement to the NDT system.

[0091] In a possible design, the communication module can include a receiving module and a sending module. The sending module is configured to implement the sending function of the communication apparatus in the sixth aspect, and the receiving module is configured to implement the receiving function of the communication apparatus in the sixth aspect.

[0092] In a possible design, the communication apparatus in the sixth aspect can further include a storage module that stores programs or instructions. When the processing module executes the programs or instructions, the communication apparatus in the sixth aspect can execute the method in the third aspect.

[0093] In a seventh aspect, a communication apparatus (for example, the communication apparatus can be a chip or a chip system) is provided. The communication apparatus includes a processor configured to implement the functions involved in any of the above aspects.

[0094] In a possible design, the communication apparatus can further include a memory configured to save necessary programs, instructions, and data. The processor is coupled to the memory, and is configured to execute the computer programs or instructions stored in the memory, so that the communication apparatus executes the method in any of the possible implementation manners of the first aspect to the third aspect.

[0095] In a possible design, the communication apparatus in the seventh aspect further includes a transceiver. The transceiver can be a transceiver circuit or an interface circuit. The transceiver can be configured to enable the communication apparatus to communicate with another communication apparatus.

[0096] In a possible design, the processor can be integrated with the memory.

[0097] In some possible designs, when the apparatus is a chip system, the apparatus can be formed by a chip, or can include a chip and other discrete devices.

[0098] In an eighth aspect, a communication apparatus is provided. The communication apparatus includes a processor and an interface circuit. The interface circuit is configured to receive a signal from another communication apparatus outside the communication apparatus and transmit the signal to the processor, or send a signal from the processor to the other communication apparatus outside the communication apparatus. The processor is configured to implement the method in any possible implementation manner of the first aspect to the third aspect by using a logic circuit or executing code instructions.

[0099] It can be understood that, when the communication apparatus in any one of the seventh aspect or the eighth aspect is a chip, the sending action / functionality described above can be understood as output, and the receiving action / functionality described above can be understood as input.

[0100] In a ninth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program or instructions. When the computer program or instructions are executed on a communication apparatus, the communication apparatus can perform the method in any one of the first aspect to the third aspect.

[0101] In a tenth aspect, a computer program product is provided. The computer program product includes instructions, which, when executed on a communication apparatus, enable the communication apparatus to perform the method in any one of the first aspect to the third aspect.

[0102] In an eleventh aspect, a communication system is provided. The communication system includes a model training function configured to implement the method in the first aspect, and an NDT system configured to implement the method in the second aspect.

[0103] In a possible design, the system in the eleventh aspect further includes a trusted AI management function configured to implement the method in the third aspect. DETAILED DESCRIPTION

[0104] FIG. 1 is a schematic diagram of a scenario in which a model is attacked by an adversary in a lifecycle management phase of the model;

[0105] FIG. 2 is a schematic diagram of an architecture of a communication system according to an embodiment of the present application;

[0106] FIG. 3 is an architecture diagram of an applicable application scenario according to an embodiment of the present application;

[0107] FIG. 4 is an architecture diagram of another applicable application scenario according to an embodiment of the present application;

[0108] FIG. 5 is a schematic diagram of model training function and NDT system deployment according to an embodiment of the present application;

[0109] FIG. 6 is a flow diagram of a model robustness enhancement method according to an embodiment of the present application;

[0110] FIG. 7 is a flow diagram of another model robustness enhancement method according to an embodiment of the present application;

[0111] FIG. 8 is an architecture diagram of a communication apparatus according to an embodiment of the present application;

[0112] FIG. 9 is an architecture diagram of another communication apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0113] Embodiments of the present application will be presented around various aspects, embodiments or features of systems that can include a plurality of devices, components, modules, etc. It should be understood and appreciated that each of the various systems can include additional devices, components, modules, etc., and / or can not include all of the devices, components, modules, etc. discussed in connection with the figures. Furthermore, combinations of these approaches can also be used.

[0114] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as a wireless fidelity (Wi-Fi) system, a vehicle to everything (V2X) communication system, a device-to-device (D2D) communication system, a vehicle networking communication system, a worldwide interoperability for microwave access (WiMAX) communication system, a 4th generation (4G) mobile communication system such as a long term evolution (LTE) system, a worldwide interoperability for microwave access (WiMAX) communication system, a 5th generation (5G) mobile communication system such as a new radio (NR) system, and future communication systems, etc.

[0115] For ease of understanding, the following first introduces the related terms, concepts or technologies that may be involved in the embodiments of the present application:

[0116] 1. Inference model (may also be referred to as model)

[0117] Learned from data, a function that can implement a specific function / mapping. The model can be obtained based on AI or ML technology, and therefore can also be referred to as an artificial intelligence / AI model, a machine learning / ML model, etc. Common algorithms used to generate AI / ML models include supervised learning, unsupervised learning, reinforcement learning, and the corresponding models can be referred to as supervised learning models, unsupervised learning models, and reinforcement learning models. For example, a supervised learning model can be a classification model, a prediction model, a regression model, etc., and an unsupervised learning model can be a clustering model. In addition, the model can also be obtained based on neural network (NN) technology, and such a model can also be referred to as a neural network model, a deep learning model, etc.

[0118] The main process of building a model includes problem analysis (determining the data to be collected and the type of model to be used, etc.), data collection, model training, and model inference, etc. Among them, model training is the process of determining the parameters of the model using data, and model inference is the process of predicting the output result according to the new input using the trained model.

[0119] It should be understood that in the embodiments of the present application, AI refers to a technology that presents human intelligence through a computer program, and ML focuses on developing computer programs that can access data and use these data for self-learning. In the following description, artificial intelligence and machine learning are no longer distinguished, for example, an ML model can also be referred to as an AI model.

[0120] 2. Data augmentation

[0121] Data augmentation is a technique that generates new training data with slight differences by performing a series of transformation operations on the original training data, thereby expanding the size and diversity of the training data set. Data augmentation techniques not only alleviate the problem of data scarcity, but also improve the robustness of the model to noise and changes, and enhance the generalization ability of the model.

[0122] 3. Robustness enhancement data for machine learning

[0123] Generally refers to the addition of some specific data or certain processing of existing data in the training data, aiming to improve the stability and reliability of machine learning models in the face of uncertainty and noise. Its main purpose is to enable the model to better cope with various disturbances and abnormal situations during the training process, so as to exhibit higher robustness in actual application.

[0124] 4、Data drift

[0125] The accuracy and reliability of machine learning models are of great importance. However, the data on which the model relies is rarely static and will change in unpredictable ways over time. This phenomenon is known as data drift, which poses a major challenge to the effectiveness of the model.

[0126] Data drift, also known as covariate shift, occurs when the statistical properties of input data change over time, resulting in scenarios where there is a difference between the distribution of data used during model training and the distribution of data encountered in model deployment or the real world. In short, data drift means that the data on which the model is built no longer represents the data for which predictions are expected.

[0127] Data drift can significantly affect the performance and accuracy of machine learning models. When the underlying data distribution changes, the model's assumptions become invalid, leading to suboptimal predictions and potentially inaccurate results. For example, a model trained to predict customer preferences based on historical data may fail to capture changing trends or external events, resulting in decreased predictive power.

[0128] 5、AI / ML model application in wireless networks

[0129] In order to improve the level of intelligence and automation of the network, AI and ML technology is being applied in more and more fields, including the management domain, core network (CN) domain and radio access network (RAN) domain. Multiple fields are researching how to apply AI / ML technology to enable network intelligence.

[0130] To support the use of models in the network, the lifecycle management (LCM) of the model needs to be studied. The model management topic of 3GPP standalone (SA) 5, Study on Artificial Intelligence / Machine Learning (AIMLMGMT), is successfully established and discussed, focusing on the lifecycle management of models in the 5G system (5GS) (including management domain, RAN domain, core network domain), including model training, inference, performance evaluation, deployment, testing, updating, and other capabilities.

[0131] However, models may face security threats at various stages of their lifecycle management, including poisoning attacks, backdoor attacks, theft attacks, and adversarial attacks. As shown in FIG. 1, in the data preparation stage, an adversary can pose a security threat to the model through poisoning attacks and backdoor attacks, where the adversary injects poison data into the training data set through poisoning attacks, causing the model to produce incorrect output or abnormal behavior after deployment, which can cause network service interruption and affect network user experience. Through backdoor attacks, specific neurons are implanted in the neural network model, affecting the model's judgment of specific inputs. In the inference stage, an adversary can pose a security threat to the model through theft attacks and adversarial attacks, where illegal acquisition of model parameters or structure through theft attacks can lead to intellectual property leakage; through adversarial attacks, an adversarial input is constructed to mislead the model, causing it to produce incorrect output, which can lead to critical decision errors or network service termination. To reduce the above security risks in the wireless field, necessary security protection measures need to be taken from multiple stages of the model lifecycle, such as training, deployment, and updating, to improve the security of the model.

[0132] To this end, 3GPP also discussed the trusted management of AI / ML models in the wireless field in Release (R) 18. In Technical Report (TR) 28.908, 3GPP made clear requirements for the trusted management of AI / ML models: trusted management should be added in the model training, testing, and inference stages to enhance the robustness and security of AI / ML models, and AI / ML models should be regularly updated and maintained to ensure that the model still maintains high prediction accuracy when the data distribution changes (possibly due to an adversary attack).

[0133] Since training data needs to be collected for model parameter updates during model training and retraining, the training data for AI / ML model training in the wireless field is generally taken from the live network, so there is a security risk that an adversary can inject poison sample points into the training data collected from the live network, causing the accuracy of the trained model to decrease.

[0134] To reduce the security risk of the adversary poisoning, the 3GPP standard only improves the quality of the training data and detects and removes outliers. Specifically, the training data collected from the live network is subjected to outlier detection, and abnormal data is removed, and the proportion of missing values in the processed training data set is calculated.

[0135] A large number of literatures in the current academic field have proved that the data poisoning attack algorithm designed by the adversary for the target model has a learnable pattern, and the outlier detection model will have a higher correct rate of outlier detection under the condition of increasing the attack pattern and other prior knowledge. Therefore, wireless field AI models urgently need to learn the attack pattern of the adversary and integrate the prior knowledge into the training data preprocessing stage to enhance the robustness of the wireless field AI model. In addition, the 3GPP TR 28.908 protocol also points out that during the training process of the ML model, the adversarial samples can also enter the model training stage together with the normal samples to enhance the recognition ability of the adversary attack, and thus improve the overall robustness of the ML model.

[0136] However, the current ML model in the live network cannot collect a large amount of high-quality data with characteristics such as adversary attack algorithm characteristics and model distribution characteristic drift trajectory in a short time, so it is difficult for the target ML model to obtain valuable prior knowledge to enhance the robustness of the model. Therefore, the embodiment of the present application provides a model robustness enhancement method and a communication device, which generates robustness enhancement data by using a network digital twin (NDT) system, and then guides the AI / ML model to obtain prior knowledge such as adversary attack patterns, and finally obtains an AI / ML model with enhanced robustness.

[0137] In order to better understand the embodiments of the present application, before introducing the embodiments of the present application, the following points are explained.

[0138] First, in the embodiments of the present application, "for indicating" can include for directly indicating and for indirectly indicating. When describing that certain "indication information" is used to indicate A, it can include that the indication information directly indicates A or indirectly indicates A, and does not mean that A must be carried in the indication information.

[0139] The information indicated by the indication information is referred to as to-be-indicated information. In a specific implementation process, there are many ways to indicate the to-be-indicated information, for example, but not limited to, the to-be-indicated information can be directly indicated, such as the to-be-indicated information itself or an index of the to-be-indicated information. The to-be-indicated information can also be indirectly indicated by indicating other information, where the other information and the to-be-indicated information have an association relationship. The to-be-indicated information can also be only indicated in part, and the other part of the to-be-indicated information is known or agreed in advance. For example, the indication of specific information can also be achieved by means of the arrangement order of each information agreed in advance (for example, specified by a protocol), thereby reducing the indication overhead to a certain extent. Meanwhile, the common part of each information can be identified and uniformly indicated, so as to reduce the indication overhead caused by separately indicating the same information.

[0140] In addition, the specific indication manner can also be various existing indication manners, for example, but not limited to, the above indication manners and various combinations thereof. Specific details of various indication manners can be referred to the prior art, and will not be described herein. As known from the above, for example, when multiple information of the same type needs to be indicated, the indication manners of different information can be different. In a specific implementation process, the required indication manner can be selected according to specific needs, and the selected indication manner is not limited by the embodiments of the present application. In this way, the indication manner involved in the embodiments of the present application should be understood as covering various methods that can enable the to-be-indicated party to know the to-be-indicated information.

[0141] The to-be-indicated information can be sent as a whole, or can be sent separately into multiple sub-information, and the sending period and / or sending time of the sub-information can be the same or different. The specific sending method is not limited by the present application. The sending period and / or sending time of the sub-information can be predefined, for example, predefined according to a protocol, or configured by the transmitting end device by sending configuration information to the receiving end device.

[0142] Secondly, in the embodiments of the present application, the first, second and various numerical numbers are only for differentiation for convenience of description, and do not limit the scope of the embodiments of the present application. For example, different indication information is differentiated. For another example, the first indication information and the second indication information are only for differentiating different indication information, and the sequence thereof is not limited. Those skilled in the art can understand that the words of "first", "second" and the like do not limit the quantity and execution sequence, and the words of "first", "second" and the like do not necessarily mean different.

[0143] Third, in the embodiments of the present application, "when", "in the case of", "if" and the like all refer to the device making corresponding processing under certain objective circumstances, and are not limited to time, and do not require the device to have a judgment action when implemented, nor does it mean that there are other limitations.

[0144] Meanwhile, in the embodiments of the present application, "exemplary" or "for example" and the like are used to indicate an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" and the like is intended to present the relevant concept in a specific manner for ease of understanding.

[0145] Finally, the network architecture and service scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as network architectures evolve and new service scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0146] Please refer to FIG. 2, which is a schematic diagram of an architecture of a communication system to which the embodiments of the present application are applied. As an example, as shown in FIG. 2, the communication system includes a model training function and an NDT system, which can indirectly or directly communicate with each other. Optionally, the communication system can also include a model inference function, and / or a trusted AI management function.

[0147] Among them, the trusted AI management function: responsible for unified trusted management of AI model (such as robustness management, explainability management and fairness management, etc.), issues relevant trusted policies to the model training function, model inference function and NDT system and receives the trusted related reports reported by the model training function and inference function after training is completed. The trusted AI management function can be deployed in the network management system (network management system, NMS) or the element management system (element management system, EMS), such as the model management function in the NMS or EMS. It should be noted that the deployment position of the trusted AI management function needs to be higher or flat than the deployment position of the model training function.

[0148] Model training function: responsible for model training, and generating inference model after training, which can also be referred to as training function, model training entity, training entity, model training network element, model training function network element, training network element, etc., which are not limited. The model training function can be deployed in the NMS, EMS, network equipment in the RAN domain, or core network element in the core network domain, such as network data analytics function (NWDAF) network element, application function (AF) network element.

[0149] For example, the capability or function of the model training function can be deployed on a certain network element or device, or a certain network element or device has the capability or function of model training, which can be referred to as a model training network element or a model training device, and the embodiments of the present application do not limit this; for the sake of description, the embodiments of the present application take the model training function as an example for description, but all can be replaced by other devices with the capability or function of training and inference model.

[0150] Model inference function: responsible for model inference or prediction, and obtaining corresponding prediction output by inputting data into the model, which can also be referred to as inference function, model inference entity, inference entity, model inference network element, model inference function network element, inference network element, etc., which are not limited. The model inference function can be deployed in the EMS, such as the management data analytics function (MDAF) in the EMS, or in the network equipment in the RAN domain, or in the core network element in the core network domain, such as the NWDAF network element and the AF network element.

[0151] For example, the capability or function of the model inference function can be deployed on a certain network element or device, or a certain network element or device has the capability or function of model inference, which can be referred to as a model inference network element or a model inference device, and the embodiments of the present application do not limit this; for the sake of description, the embodiments of the present application take the model inference function as an example for description, but all can be replaced by other devices with the capability or function of inference or prediction based on model.

[0152] The model inference function and the model training function can be deployed in different devices, such as the model training function can be deployed in the NMS, and the model inference function can be deployed in the network equipment; the model inference function and the model training function can also be deployed in the same device, such as the model inference function and the model training function can be deployed in the same network equipment or core network element, and the present application does not particularly limit this.

[0153] NDT system: can be referred to as NDT, responsible for synthesizing data required for adversarial training according to trusted correlation strategies and sampled training data.

[0154] In the embodiments of the present application, the trusted AI management function, the model training function and the model inference function can be deployed together or separately.

[0155] It should be noted that although Figure 2 does not show it, multiple model inference functions can be included in the communication system, such as the model training function can also be connected to a model inference function. The number of model training functions and model inference functions is not limited in the embodiments of the present application.

[0156] In order to facilitate understanding of the embodiments of the present application, the following describes possible application scenarios applicable to the embodiments of the present application in conjunction with Figures 3-5.

[0157] Taking a service architecture as an example, Figure 3 is a schematic architecture diagram of one application scenario applicable to the embodiments of the present application. As shown in (a) of Figure 3, the trusted AI management function, the model training function, the model inference function and the NDT system can all be deployed in the management service (MnS) producer, which can provide management services to the outside, and the MnS consumer can invoke the management service from the MnS producer.

[0158] As shown in (b) of Figure 3, unlike (a) of Figure 3, the trusted AI management function, the model training function and the model inference function can be deployed in the MnS producer, while the NDT system is independently deployed and connected to the MnS producer.

[0159] Taking a management architecture as an example, Figure 4 is a schematic architecture diagram of another application scenario applicable to the embodiments of the present application. As shown in Figure 4, the NMS and the EMS both belong to the management domain device, the EMS can manage the single-domain network element or device, for example: RAN device or CN network element, and the NMS can manage the cross-domain device, for example: EMS, in addition, the NMS can also directly manage the single-domain network element. The NMS / EMS can be a traditional cross-domain / single-domain network management device, or a part of the management function set of cross-domain management (Cross-domain management) / single-domain management (Domain management).

[0160] The NMS can be deployed with a trusted AI management function, a model training function, and an NDT system; the EMS can be deployed with a trusted AI management function, a model training function, a model inference function, and an NDT system; and the RAN device or the CN network element can also be deployed with a model training function, a model inference function, and an NDT system. In the architecture shown in FIG. 3, the EMS and the RAN / CN can be the MnS producer in FIG. 3, and the NMS can be the MnS consumer.

[0161] The NMS and the EMS can also be collectively referred to as a 3GPP management system, or an operations administration and maintenance (OAM) module.

[0162] In the embodiments of the present application, there are several ways as shown in FIG. 5 for the deployment of the model training function and the NDT system: the model training function is deployed in the NMS, and the NDT system can be deployed in the NMS, the EMS, or the network device in the RAN domain (such as the distributed unit (DU) or the centralized unit (CU) of the base station); the model training function is deployed in the EMS, and the NDT system can be deployed in the EMS or the network device in the RAN domain; the model training function is deployed in the network device in the RAN domain, and the NDT system can be deployed in the network device in the RAN domain.

[0163] In a possible design, in the case where the trusted AI management function is deployed in the EMS, the model training function cannot be deployed in the NMS, and the NDT system cannot be deployed in the NMS, the EMS, or the network device in the RAN domain.

[0164] In addition, in the case where the model training function is deployed in the NMS and the NDT system is deployed in the EMS or the network device in the RAN domain, and in the case where the model training function is deployed in the EMS and the NDT system is deployed in the network device in the RAN domain, the model training function and the NDT system have the possibility of cross-operator deployment.

[0165] The model robustness enhancement method provided in the embodiments of the present application will be described in detail below with reference to FIGS. 6 and 7.

[0166] For example, FIG. 6 is a flowchart illustrating a model robustness enhancement method provided by an embodiment of the present application. The model robustness enhancement method is described by taking the communication between the trusted AI management function, the model training function, and the NDT system shown in FIG. 2 as an example. Of course, the subject performing the action of the trusted AI management function in the method can also be a device / module in the trusted AI management function, such as a chip, processor, processing unit, etc. in the trusted AI management function, without limitation; the subject performing the action of the model training function in the method can also be a device / module in the model training function, such as a chip, processor, processing unit, etc. in the model training function, without limitation; and the subject performing the action of the NDT system in the method can also be a device / module in the NDT system, such as a chip, processor, processing unit, etc. in the NDT system, without limitation.

[0167] As shown in FIG. 6, the model robustness enhancement method includes the following steps.

[0168] S601, the model training function sends abnormal data to the NDT system. Correspondingly, the NDT system receives the abnormal data from the model training function.

[0169] The abnormal data refers to data detected by the model training function from the training data set, which causes the model prediction performance to decrease, and can include adversary poisoning data, drift data, etc. The drift data refers to data that has undergone data drift, i.e., the distribution and properties of the data have changed over time, resulting in a decrease in model performance.

[0170] The training data set is data collected from the live network, which contains a large amount of sample data and corresponding labels. The sample data refers to input data for training the model, while the label refers to the output result or category corresponding to each sample data, or in other words, the comparison data between the sample data as the input of the model training and the output data of the model training, which is used to converge the model. For example, the sample data and the label can constitute one or more sets of training data in the form of (x j , y j ), j is the serial number of the training data in the training data set, x j is the sample data, and y j is the label. During model training, the y j can be compared with the output data when the model input x j is input, and the trainable parameters in the model are adjusted based on the comparison result by, for example, back propagation and gradient descent, so as to obtain the model used for inference.

[0171] The model training function can collect (or gather or obtain) training data sets of corresponding types according to the requirements of a target model (i.e., a model obtained after model training is completed), for example, the target model is used for network performance analysis, used for air interface NLOS (non-light of sight) positioning / LOS (light of sight) positioning classification, used for predicting terminal device positions, etc., and different types of training data are used for training of models with different functions.

[0172] When performing model training, the model training function can detect abnormal data in the training data set according to local configuration or triggering of other devices or functions, obtain the abnormal data, and send the abnormal data to the NDT system to trigger the NDT system to perform data enhancement on the abnormal data.

[0173] For example, the target model is used for air interface LOS / NLOS positioning classification, i.e., NLOS / LOS classification of a channel matrix, sample data in the training data set can be a channel matrix, and the corresponding label can be NLOS or LOS. The abnormal data detected by the model training function from the training data set can be training data in which the upper right corner element of the channel matrix is tampered with by an adversary.

[0174] In a possible design, the model training function can perform abnormal data detection and abnormal data sending according to triggering of the trusted AI management function. In this design, the trusted AI management function can send first information to the model training function, and correspondingly, the model training function can receive the first information from the trusted AI management function. The first information is used to instruct the model training function to send abnormal data detected from the training data set to the NDT system for data enhancement, or in other words, the first information is used to instruct the model training function to use the detected abnormal data for model training after the abnormal data is enhanced by the NDT system.

[0175] Therefore, the model training function can detect abnormal data and send the abnormal data to the NDT system according to the first information. That is, the model training function can determine not to discard the detected abnormal data according to the first information, but to send the abnormal data to the NDT system for data enhancement and then use the abnormal data for model training.

[0176] The first information can be considered as strategy information for processing the abnormal data. In some designs, the first information can also be used to instruct the model training function to discard the abnormal data detected from the training data set, in which case the abnormal data is not used for model training. For example, the first information is indicated by 1 bit, and a bit value of 1 indicates that the model training function sends the abnormal data detected from the training data set to the NDT system for data enhancement, and a bit value of 0 indicates that the model training function discards the abnormal data detected from the training data set.

[0177] Optionally, the first information can also include information of an algorithm for abnormal data detection, for example, the information of the algorithm for abnormal data detection can include an abnormal data detection algorithm identification (ID) and parameter values involved in the abnormal data detection algorithm, in which case it can be understood that the model training function and the NDT system can be pre-configured with one or more abnormal data detection algorithms, and each abnormal data detection algorithm corresponds to an algorithm ID. In some cases, the information of the algorithm for abnormal data detection can also be sent separately from the first information, which is not limited.

[0178] Optionally, the trusted AI management function can also send other information related to model training to the model training function, such as model update strategy indication information, which can be used to instruct the model training function how to update the model training without affecting the model performance, how to achieve efficient model training update under limited computing resources, or how to maintain the stability of the model under changing data distribution, etc.

[0179] S602, the NDT system performs data enhancement on the abnormal data to obtain robustness enhanced data corresponding to the abnormal data.

[0180] The robustness enhanced data corresponding to the abnormal data is used for model training, and the robustness enhanced data corresponding to the abnormal data can also be referred to as enhanced data of the abnormal data, which can be understood as data obtained by performing data enhancement on the abnormal data, and can be used to improve the robustness of the model.

[0181] In the embodiments of the present application, after the NDT system obtains the abnormal data, the NDT system can perform data enhancement on the abnormal data to obtain robustness enhanced data corresponding to the abnormal data. The robustness enhanced data corresponding to the abnormal data can be used as adversarial data for the model training function to input the model for adversarial training. The adversarial data and the training data in the original training data set except the abnormal data constitute a new training data set, so that the model automatically learns the prior knowledge of the enemy attack and the prior knowledge of the feature domain drift trajectory of the target model itself in the training process, to improve the attack defense robustness and feature domain drift robustness of the target model.

[0182] In a possible implementation, the NDT system generating robustness-enhanced data corresponding to the abnormal data can include: the NDT system can learn abnormal behavior from the abnormal data, so that the NDT system can generate robustness-enhanced data corresponding to the abnormal data according to the abnormal behavior. That is, the NDT system can learn abnormal behavior from the abnormal data, and can generate robustness-enhanced data corresponding to more abnormal data with the characteristics of the abnormal behavior based on the abnormal behavior.

[0183] With reference to the above examples, taking the target model for air interface LOS / NLOS positioning classification as an example, the abnormal behavior learned by the NDT system from the abnormal data can be that the enemy will tamper with data at the upper right corner of the channel matrix, and therefore, the NDT system can adjust the abnormal data obtained, such as increasing the adversarial perturbation, generate more channel matrices in which the data at the upper right corner is tampered with according to the abnormal data and the abnormal behavior, or flip the label in the abnormal data, such as modifying LOS to NLOS or modifying NLOS to LOS. Therefore, the robustness-enhanced data corresponding to the abnormal data can also be referred to as robustness-enhanced data based on abnormal behavior, which is not limited in this regard.

[0184] In a possible design, the NDT system performing data enhancement on the abnormal data can be triggered directly after the NDT system obtains the abnormal data, that is, after the NDT system receives the abnormal data, the NDT system selects a suitable (good data enhancement effect) data enhancement algorithm from the data enhancement algorithms preconfigured locally, and performs data enhancement according to the selected data enhancement algorithm to obtain robustness-enhanced data corresponding to the abnormal data.

[0185] For example, the NDT system can determine the data enhancement algorithm to be used according to the proportion of the abnormal data in all training data, the data distribution, and the basic statistical characteristics (such as mean, variance, etc.) of the abnormal data, query the data enhancement function table preconfigured locally, and generate robustness-enhanced data corresponding to the abnormal data. The proportion of the abnormal data in all training data can be sent to the NDT system by the model training function.

[0186] For another example, the NDT system can obtain historical data enhancement results, determine the robustness-enhanced effect of different data enhancement algorithms according to the historical data enhancement results, train a reinforcement learning model locally or adopt a preconfigured scoring mechanism to dynamically adjust the selection strategy, and select a data enhancement algorithm with better data enhancement effect from the data enhancement algorithms preconfigured locally to generate robustness-enhanced data corresponding to the abnormal data.

[0187] In another possible design, the NDT system triggering data augmentation on the abnormal data can also be triggered according to the indication of the model training function, that is, the NDT system does not immediately perform data augmentation after obtaining the abnormal data, but needs to start performing data augmentation according to the indication information of the model training function. In this design, the model training function can send first indication information to the NDT system, and correspondingly, the NDT system can receive the first indication information from the model training function. The first indication information is used to instruct the NDT system to perform data augmentation on the abnormal data, and the first indication information can be understood as a switch for starting the NDT system to perform data augmentation on the abnormal data. Thus, the NDT system can perform data augmentation on the abnormal data according to the first indication information to obtain the robustness enhanced data corresponding to the abnormal data.

[0188] Optionally, the first indication information can include information for instructing an algorithm for data augmentation on the abnormal data, such as an abnormal data augmentation algorithm ID, that is, the model training function instructs the NDT system of the algorithm for data augmentation on the abnormal data at the same time of triggering the NDT system to perform data augmentation on the abnormal data. At this time, the NDT system can perform data augmentation on the abnormal data according to the instructed abnormal data augmentation algorithm.

[0189] The process of the model training function determining the abnormal data augmentation algorithm is similar to the NDT system described above, such as querying a data augmentation function table pre-stored in the model training function according to the proportion of the abnormal data in all training data, data distribution and basic statistical characteristics (such as mean, variance, etc.), or such as obtaining historical data augmentation results, determining the robustness enhancement effect of different data augmentation algorithms according to the historical data augmentation results, and locally training a reinforcement learning model or adopting a pre-stored scoring mechanism to dynamically adjust a selection strategy, so as to select a data augmentation algorithm with better data augmentation effect from the locally configured data augmentation algorithms.

[0190] Optionally, the first indication information can be sent together with the abnormal data, such as being sent in the same message or signaling, or can be sent separately from the abnormal data, such as being sent in different messages or signaling, which is not limited.

[0191] It should be understood that in the model training configuration phase, the model training function, the NDT system and the trusted AI management function are all configured with the identifier of the model that needs to be trained with the enhanced data. In other words, the training models at the model training function, the NDT system and the trusted AI management function are kept aligned.

[0192] S603, the NDT system sends the robustness enhanced data corresponding to the abnormal data to the model training function. Correspondingly, the model training function receives the robustness enhanced data corresponding to the abnormal data from the NDT system.

[0193] The NDT system obtains robustness-enhanced data corresponding to the abnormal data, and sends the robustness-enhanced data corresponding to the abnormal data to the model training function for model training.

[0194] Optionally, after completing data enhancement of the abnormal data, the NDT system can also send a data enhancement report to the model training function, which can be used to feed back parameter values used for data enhancement of the current type of data, time for generating enhanced data, and data enhancement accuracy achieved, etc. The data enhancement report can be used to feed back execution of data enhancement of the abnormal data.

[0195] Optionally, the data enhancement report corresponding to the abnormal data can be sent together with the robustness-enhanced data corresponding to the abnormal data, or can be sent separately, which is not limited.

[0196] Correspondingly, the model training function can use the robustness-enhanced data corresponding to the abnormal data for model training. For example, the model training function can automatically design a loss function based on the type of training data according to the model update strategy issued by the trusted AI management model, and perform differential perception model training to obtain an updated model. The differential perception model training is an efficient model active learning strategy that forces the model to pay attention to different causal features between real sampling data (such as abnormal data) and counterfactual data (such as robustness-enhanced data) during the training process.

[0197] For example, the model training function performs differential perception model training as follows:

[0198] (1) Calculate the difference between the real sampling data (x) and its corresponding counterfactual data

[0199] (2) Remove similar features (mask) between the real sampling data and its corresponding counterfactual data, and only keep the features with large differences:

[0200] (2-1) Mask generation:

[0201] where m i is the a-th element of the mask m. If the difference δ a exceeds the threshold τ, then m a = 1; otherwise, m a = 1.

[0202] (2-2) Remove similar features between the real sampling data and its corresponding counterfactual data, and only keep the features with large differences:

[0203] where, ​the masked real sampled data x, the masked counterfactual data is an element-wise multiplication operation.

[0204] (3) Model training with difference perception:

[0205] wherein, θ is a model parameter (such as a weight), is the actual training data set used, and l(·) is a loss function (usually cross-entropy loss), is the expectation of minimizing the sum of the loss of the real sampled data and the corresponding counterfactual data.

[0206] Thus, the model training function only retains features that are significantly different between the real sampled data and the corresponding counterfactual data through the masking strategy, so that the model training process pays more attention to the differences in causal features, and the generalization ability of the model is improved.

[0207] Optionally, after completing the model training based on the robustness-enhanced data corresponding to the abnormal data, the model training function can also send the updated model parameter information, such as the updated model weight, to the NDT system. Correspondingly, the NDT system can receive the updated model parameter information from the model training function, so as to update the training model stored by the NDT system according to the updated model parameter information.

[0208] Optionally, the model training function can also adjust the parameters for data enhancement executed by the NDT system according to the robustness-enhanced effect of the model after training based on the robustness-enhanced data corresponding to the abnormal data. For example, the model training function sends information indicating the parameters for data enhancement that need to be updated to the NDT system. Correspondingly, the NDT system can receive the information indicating the parameters for data enhancement that need to be updated from the model training function to adjust the parameters for data enhancement, and execute with new parameters in the next data enhancement, so as to improve the robustness of the model.

[0209] Thus, in the model training process, the model training function can send the detected abnormal data to the NDT system for data enhancement, obtain the robustness-enhanced data corresponding to the abnormal data from the NDT system, input the robustness-enhanced data corresponding to the abnormal data into the model as the adversarial data for adversarial training, so that the target model automatically learns the prior knowledge of the enemy attack and the prior knowledge of the feature domain drift trajectory of the target model itself in the training process, thereby improving the robustness of the model, such as the attack defense robustness and the feature domain drift robustness of the model.

[0210] In this embodiment of the application, the model training function, based on using robust enhancement data corresponding to abnormal data as new training data to improve the robustness of the model, can also combine robust enhancement data of other data features to further improve the robustness of the model. In one possible design, the model training function can also improve the robustness of the model based on robust enhancement data corresponding to high uncertainty data, which may include the following S604-1 to S604-3:

[0211] S604-1, The model training function sends high-uncertainty data to the NDT system. Correspondingly, the NDT system receives high-uncertainty data from the model training function.

[0212] High uncertainty data refers to data in the training dataset that causes the confidence level of the model's predictions (such as classification or generation) to be lower than a first preset threshold. It is the most uncertain type of uncertain data. Confidence level refers to the probability estimate of the model generating a certain output or classification label given an input, reflecting the model's confidence in its predictions. High uncertainty data can also be called high variability data, without further categorization.

[0213] The model training function can sample the training data that causes the greatest change after the model update, based on information obtained from historical model training data. This sampled data is considered high-uncertainty data (typically drift data). In one possible implementation, high-uncertainty data can refer to the training data that changes the most on historical models with different weight parameters. The sampling process can be implemented as follows: v(x) = max{v i (x)|i∈[1,K]}, v i (x)=Var(P(y i |x i ,θ s ));

[0214] Where v(x) is the total variability of the sample data x, which is the variability v among all categories i. i The maximum value of (x) is determined by selecting the class with the greatest variability when classifying the sample data x as the representative variability sample of the sample data;

[0215] v i (x) represents the standard deviation of the variability of the sample data x in the i-th class; specifically, it represents the standard deviation of the model under different parameters θ. s The standard for predicting the i-th class probability of sample data x;

[0216] Var(P(y i |x i ,θ s )) is for parameter θ sa standard deviation of the i-th class prediction probability of the model on the sample data x;

[0217] K is the total number of classes for classifying the sample data x, K is an integer greater than 1;

[0218] P(y i |x i ,θ s ) is the probability that the sample data x is predicted to be the i-th class given the model parameters θ s

[0219] θ s is a specific parameter in the model parameter set s.

[0220] Therefore, the model training function can sample the sample data of the class with the maximum variability standard deviation in the K classes as high uncertainty data, so as to send the sampled high uncertainty data to the NDT for data enhancement.

[0221] Similar to the above abnormal data, whether the model training function sends the high uncertainty data to the NDT system, or whether the model training function needs to obtain the robustness enhanced data corresponding to the high uncertainty data for model training, in addition to triggering the sampling and sending of the high uncertainty data by the model training function according to the model training requirements, it can also be triggered according to the trusted AI management function.

[0222] In a possible design, the trusted AI management function can send second information to the model training function, and correspondingly, the model training function receives the second information from the trusted AI management function. Wherein, the second information is used to instruct the model training function to send the high uncertainty data sampled from the training data set to the NDT system for data enhancement, or in other words, the second information is used to instruct the model training function to use the high uncertainty data sampled after data enhancement by the NDT system for model training. The second information can be understood as strategy information for handling high uncertainty.

[0223] Optionally, the second information can include information for indicating the algorithm for sampling the high uncertainty data, such as a sampling algorithm ID, at this time it can be understood that the model training function and the NDT system on both sides can be pre-configured with one or more sampling algorithms, and each sampling algorithm corresponds to an algorithm ID.

[0224] Therefore, the model training function can sample the high uncertainty data from the training data set and send the high uncertainty data to the NDT system according to the second information. That is, the model training function can send the sampled high uncertainty data to the NDT system for data enhancement according to the second information, for model training.

[0225] ​Optionally, the second information can also be sent together with the first information, such as being carried in the same message or signaling, or can be sent separately, such as being carried in different messages or signaling, and no limitation is made in this regard.

[0226] Optionally, the high-uncertainty data can be sent together with the abnormal data, or can be sent separately from the abnormal data, and no limitation is made in this regard.

[0227] S604-2, the NDT system performs data enhancement on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data.

[0228] The robustness-enhanced data corresponding to the high-uncertainty data is used for model training, and the robustness-enhanced data corresponding to the high-uncertainty data can also be referred to as enhanced data of the high-uncertainty data, which can be understood as data obtained by performing data enhancement on the high-uncertainty data and can be used to improve the robustness of the model.

[0229] After the NDT system obtains the high-uncertainty data, the NDT system can perform data enhancement on the high-uncertainty data, and the high-uncertainty data after the data enhancement can be used as new training data for model training.

[0230] In one possible implementation, the NDT generates robustness-enhanced data corresponding to the high-uncertainty data, which can include: the NDT system performs model prediction on the high-uncertainty data to obtain a first prediction result, determines an counterfactual sample generation algorithm according to the first prediction result, and generates robustness-enhanced data corresponding to the high-uncertainty data according to the counterfactual sample generation algorithm and the high-uncertainty data.

[0231] That is, the NDT system inputs the obtained high-uncertainty data into the model obtained by the current training to perform prediction and obtains a first prediction result, then selects a counterfactual sample generation algorithm with better robustness-enhanced effect from one or more counterfactual sample generation algorithms pre-configured locally according to the first prediction result, and performs enhancement processing such as label flipping and adding adversarial perturbation on the high-uncertainty data according to the selected counterfactual sample generation algorithm to generate counterfactual data, i.e., robustness-enhanced data corresponding to the high-uncertainty data. Exemplarily, the counterfactual sample generation algorithm can include the following: an optimal transport-based counterfactual sample generation algorithm, a model latent space interpolation-based counterfactual sample generation algorithm, and a linear programming-based counterfactual sample generation algorithm.

[0232] Exemplarily, taking the target model for air interface LOS / NLOS positioning classification as an example, the channel matrix is classified as NLOS / LOS, the sample data in the high-uncertainty data can be the channel matrix, and the corresponding label can be NLOS or LOS. The robustness-enhanced data corresponding to the certain high-uncertainty data can be the label corresponding to the channel matrix (whose original label is NLOS) in the high-uncertainty data, which is reversed to LOS, and is marked as counterfactual data, for subsequent model training. Therefore, the robustness-enhanced data corresponding to the high-uncertainty data can also be referred to as counterfactual robustness-enhanced data, which is not limited.

[0233] In a possible design, the data enhancement of the high-uncertainty data by the NDT system can be triggered directly after the NDT system obtains the high-uncertainty data, that is, after the NDT system receives the high-uncertainty data, the NDT system selects a suitable (good data enhancement effect) data enhancement algorithm (such as a counterfactual sample generation algorithm) from the preconfigured data enhancement algorithm in the local, and performs data enhancement according to the selected data enhancement algorithm to obtain the robustness-enhanced data corresponding to the high-uncertainty data.

[0234] Exemplarily, the NDT system can determine the data enhancement algorithm to be used according to the proportion, data distribution, and basic statistical characteristics (such as mean, variance, etc.) of the high-uncertainty data in all training data, query the data enhancement function table preconfigured in the local, and generate the robustness-enhanced data corresponding to the high-uncertainty data. The proportion of the high-uncertainty data in all training data can be sent to the NDT system by the model training function.

[0235] Exemplarily, the NDT system can obtain historical data enhancement results, determine the robustness-enhanced effect of different data enhancement algorithms according to the historical data enhancement results, train a reinforcement learning model locally or adopt a preconfigured scoring mechanism to dynamically adjust the selection strategy, so as to select a data enhancement algorithm with better data enhancement effect from the locally configured data enhancement algorithms, and generate the robustness-enhanced data corresponding to the high-uncertainty data.

[0236] In another possible design, the NDT system can also trigger data augmentation on the high-uncertainty data according to an indication from the model training function, that is, the NDT system does not immediately perform data augmentation on the high-uncertainty data, but initiates the data augmentation according to the indication from the model training function. In this design, the model training function can send second indication information to the NDT system, and the NDT system can receive the second indication information from the model training function, so that the NDT system can perform data augmentation on the high-uncertainty data according to the second indication information. The second indication information can be used to instruct the NDT system to perform data augmentation on the high-uncertainty data, and the second indication information can be understood as a switch for instructing the NDT system to perform data augmentation on the high-uncertainty data.

[0237] Optionally, the second indication information can include information of an algorithm used for data augmentation on the high-uncertainty data, such as a high-uncertainty data augmentation algorithm ID. That is, the model training function instructs the NDT system of the algorithm used for data augmentation on the high-uncertainty data when triggering the NDT system to perform data augmentation on the high-uncertainty data. In this case, the NDT system can perform data augmentation on the high-uncertainty data according to the instructed algorithm. The model training function determines the high-uncertainty data augmentation algorithm in a manner similar to the NDT system, and details are not repeated herein.

[0238] Optionally, the second indication information can be sent together with the high-uncertainty data, or can be sent separately from the high-uncertainty data, which is not limited herein.

[0239] In S604-3, the NDT system sends the robustness-enhanced data corresponding to the high-uncertainty data to the model training function. Correspondingly, the model training function receives the robustness-enhanced data corresponding to the high-uncertainty data from the NDT system.

[0240] The NDT system obtains the robustness-enhanced data corresponding to the high-uncertainty data, and sends the robustness-enhanced data corresponding to the high-uncertainty data to the model training function for model training.

[0241] Optionally, after completing the data augmentation on the high-uncertainty data, the NDT system can also send a data augmentation report corresponding to the high-uncertainty data to the model training function, which can be used to feed back the parameter value used for data augmentation on the current type of data, the time for generating the augmented data, and the data augmentation accuracy achieved, and the like. The data augmentation report can be used to feed back the execution of the data augmentation on the high-uncertainty data.

[0242] Optionally, the data augmentation report corresponding to the high-uncertainty data can be sent together with the robustness-enhanced data corresponding to the abnormal data, or can be sent separately, which is not limited herein.

[0243] In a possible implementation, when the model training function needs to train based on robustness enhancement data corresponding to abnormal data and high-uncertainty data, the NDT system can use one data enhancement report to feed back the data enhancement situation of the abnormal data and the data enhancement situation of the high-uncertainty data. Correspondingly, the robustness enhancement data corresponding to the abnormal data and the robustness enhancement data corresponding to the high-uncertainty data can also be sent together, and no limitation is made in this regard.

[0244] Correspondingly, after the model training function obtains the robustness enhancement data corresponding to the high-uncertainty data, the model training function can use the robustness enhancement data corresponding to the high-uncertainty data and the robustness enhancement data corresponding to the abnormal data as training data, and perform model training based on the robustness enhancement data corresponding to the abnormal data and the robustness enhancement data corresponding to the high-uncertainty data. For details of the specific model training process, reference can be made to the related description in S603 described above, and no limitation is made in this regard.

[0245] At this time, the model update information and the information used to indicate the parameters for data enhancement that need to be updated, which are fed back by the model training function to the NDT, are determined based on the model training result of the robustness enhancement data corresponding to the abnormal data and the robustness enhancement data corresponding to the high-uncertainty data.

[0246] Therefore, the model training function can further perform model training in combination with the robustness enhancement data corresponding to the high-uncertainty data, and use more training data with more features to improve the robustness of the model.

[0247] Further, the model training function can also improve the robustness of the model based on the robustness enhancement data corresponding to the unfair class data, and can include the following S605-1 to S605-3.

[0248] S605-1, the model training function sends the unfair class data to the NDT system. Correspondingly, the NDT system receives the unfair class data from the model training function.

[0249] The unfair class data can be data of a class whose fairness deviation is greater than a second preset threshold, which is obtained by clustering and sampling high-uncertainty data sampled from the training data set with fairness as the target.

[0250] That is, the model training function can cluster the high-uncertainty data sampled from the training data set with fairness as the target, obtain data of different classes, and then sample data of a class whose fairness deviation is greater than a second preset threshold from the data of different classes, to obtain the unfair class data.

[0251] It should be understood that the high-uncertainty data used for clustering with fairness as the goal is a portion of the training data sampled from the live network, which can be used as the training dataset for clustering. In addition, in some implementations, unfair data can also be clustered based on the entire training dataset sampled from the live network. In this case, the training dataset for clustering includes the entire training dataset sampled from the live network.

[0252] Clustering training datasets (such as high-uncertainty data or the entire training dataset sampled from the live network) with fairness as the goal can be simply understood as follows: the model training function divides the training data into different categories based on a specific data feature, obtaining the proportion of data from each category in the training data obtained from clustering based on that specific data feature. The data augmentation process based on the unfair category data described below ensures that different categories of data are treated fairly in model predictions, preventing a larger volume of data from one category from leading to a higher probability of outputting predictions for that category.

[0253] Since data from categories with a small percentage of the data may be treated unfairly in model predictions, fairness bias can be understood as the difference between the percentage of a certain category of data in the training data and the percentage of other categories. The smaller the bias, the better the fairness of that category of data in model predictions. Therefore, the model training function can compare the fairness bias of each category of data with a set second preset threshold to determine whether each category of data is treated fairly under a specific data feature. If it is not treated fairly, it can determine which categories of data will be treated unfairly in model predictions under that specific data feature, sort the data according to the fairness bias, and sample the unfair category of data. This unfair category of data can then be augmented to ensure that it is treated fairly in model predictions compared to other categories of data.

[0254] After determining the sensitive features of the training data based on the task and model design, the fairness difference of the training data across different cluster categories under these sensitive features is calculated. In some possible implementations, calculating the fairness bias of the sensitive features (the features that have the greatest impact on the training and inference performance of the task model) during the current task model training and inference process can be achieved in the following ways:

[0255] in, This represents the category (or cluster) C in the training dataset for clustering. The fairness or unfairness of the data, where |C| represents the amount of data in category C. Let S be the amount of data in the training dataset for clustering, and S be the sensitive feature set. The percentage of data in category C that has sensitive feature s. Let s be the proportion of data in the training data that has a sensitive feature s, where s is a certain sensitive feature.

[0256] After determining the sensitive features and the training data categories based on those features, unfair category data can be sampled from the training dataset, which can be achieved in the following way:

[0257] Data on unfairness categories sampled: β is the impact factor;

[0258] in, Rep(x) represents the representativeness score of sample data x, which measures the representativeness of a sample data in category C. The higher the representativeness score, the more representative the sample data is in category C. n is the total number of sample data in category C. For the p-th sample data in category C, For the q-th sample data in category C, for and The distance between them (usually Euclidean distance);

[0259] This represents the uncertainty score of the sample data x. For all possible category labels, For sample data x in the classifier The probability of being classified as label y.

[0260] For example, taking the target model for air interface LOS / NLOS positioning classification as an example, the channel matrix is ​​classified as NLOS / LOS. The training data can be the channel matrix, and the corresponding label can be NLOS or LOS. The training dataset for clustering is clustered according to the label type, that is, the data labeled NLOS is in one class, and the data labeled LOS is in another class. If the amount of data labeled NLOS in the clustered training dataset is 90%, and the amount of data labeled LOS is only 10%, then the data labeled LOS is unfair class data.

[0261] Similar to the aforementioned anomalous and highly uncertain data, whether the model training function sends unfair category data to the NDT system, or whether the model training function needs to obtain the robustness enhancement data corresponding to the unfair category data for model training, can be determined not only by its local triggering of sampling and sending of unfair category data based on model training needs, but also by the triggering of the Trusted AI Management function.

[0262] In a possible design, the trusted AI management function can send third information to the model training function, and correspondingly, the model training function can receive the third information from the trusted AI management function. The third information is used to instruct the model training function to perform model training on robustness enhancement data corresponding to the unfair category data obtained from the NDT system, or in other words, the third information is used to instruct the model training function to perform model training on the sampled unfair category data after data enhancement by the NDT system. The third information can be understood as policy information for processing the unfair category data.

[0263] Optionally, the third information can include information used to indicate a clustering algorithm targeting fairness, such as a clustering algorithm ID, and information used to indicate an algorithm for sampling data after clustering targeting fairness, such as a sampling algorithm ID.

[0264] In this way, the model training function can send the unfair category data to the NDT system according to the third information. That is, the model training function can send the sampled unfair category data to the NDT system for data enhancement according to the third information, for model training.

[0265] Optionally, the third information can be sent together with the first information, for example, carried in the same message or signaling, or can be sent separately, for example, carried in different messages or signaling, which is not limited in this regard.

[0266] Optionally, the unfair category data can be sent together with the abnormal data, or can be sent separately from the abnormal data, which is not limited in this regard.

[0267] In S605-2, the NDT system performs data enhancement on the unfair category data to obtain robustness enhancement data corresponding to the unfair category data.

[0268] The robustness enhancement data corresponding to the unfair category data is used for model training. The robustness enhancement data corresponding to the unfair category data can also be referred to as enhancement data of the unfair category data, which can be understood as data obtained by performing data enhancement on the unfair category data and can be used to improve the robustness of the model.

[0269] After the NDT system obtains the unfair category data, the NDT system can also perform data enhancement on the unfair category data. The unfair category data after data enhancement can be used as a new training data for model training of the model training function.

[0270] In a possible implementation, the NDT can generate robustness-enhanced data corresponding to unfair category data, which can include: the NDT system can perform model prediction on the unfair category data to obtain a second prediction result, determine a fairness sample generation algorithm according to the second prediction result, and generate fairness-based robustness-enhanced data, i.e., robustness-enhanced data corresponding to the unfair category data, according to the fairness sample generation algorithm and the unfair category data.

[0271] That is, the NDT system inputs the obtained unfair category data into the model obtained by the current training for prediction to obtain a second prediction result, which is used to represent the fairness performance of the data, and then selects a fairness sample generation algorithm with better robustness-enhanced effect from one or more fairness sample generation algorithms pre-configured locally according to the second prediction result, so as to perform enhancement processing on the unfair category data according to the selected fairness sample generation algorithm, and generate robustness-enhanced data corresponding to the unfair category data. For example, the fairness sample generation algorithm can include the following: an optimal transport-based fairness sample generation algorithm, a model latent space interpolation-based fairness sample generation algorithm, and a generative adversarial network (GAN) model for generating more data similar to the data distribution of the category.

[0272] For example, taking the target model for air interface LOS / NLOS positioning classification as an example, when the channel matrix is classified as NLOS / LOS, the amount of data with the label NLOS in the high-uncertainty data accounts for 90%, and the amount of data with the label LOS accounts for only 10% without fairness data enhancement. After fairness sampling and data enhancement are performed on the data with the label LOS, the proportions of the two in the high-uncertainty data can become 1:1.

[0273] For how the NDT system performs data enhancement on the unfair category data, similar to the data enhancement of the abnormal data or the high-uncertainty data, a possible design is that the NDT system can perform data enhancement on the unfair category data after the NDT system obtains the unfair category data. For details of the implementation process, refer to the related description of the NDT system directly triggering data enhancement in S602 or S604-2 above, which will not be described herein.

[0274] Another possible design is that the NDT system can also perform data enhancement on the high-uncertainty data according to the indication of the model training function. In this design, the model training function can send third indication information to the NDT system, and correspondingly, the NDT system can receive the third indication information from the model training function. The third indication information is used to instruct the NDT system to perform data enhancement on the unfair category data.

[0275] Optionally, the third indication information can include information of an algorithm for data enhancement on the unfair category data, such as a fairness sample generation algorithm ID.

[0276] In this way, the NDT system can perform data enhancement on the unfair category data according to the third indication information to obtain robustness enhanced data corresponding to the unfair category data. For specific description, please refer to the description of the first indication information or the second indication information above, which will not be repeated here.

[0277] It should be understood that in some possible implementations, the first indication information, the second indication information, and the third indication information can be indicated by the same indication information, and different values of the indication information correspond to data enhancement of different types of feature data.

[0278] S605-3, the NDT system sends the robustness enhanced data corresponding to the unfair category data to the model training function. Correspondingly, the model training function receives the robustness enhanced data corresponding to the unfair category data from the NDT system.

[0279] The NDT system obtains the robustness enhanced data corresponding to the unfair category data, and sends the robustness enhanced data corresponding to the unfair category data to the model training function for model training.

[0280] Optionally, after completing the data enhancement on the unfair category data, the NDT system can also send a corresponding data enhancement report to the model training function, which can be used to feed back the parameter value used for data enhancement of the current type of data, the time of generating the enhanced data, and the data enhancement accuracy achieved, etc. The data enhancement report can be used to feed back the execution of the data enhancement of the unfair category data.

[0281] Optionally, the data enhancement report corresponding to the unfair category data can be sent together with the robustness enhanced data corresponding to the abnormal data, or can be sent separately, which is not limited.

[0282] In one possible implementation, when the model training function needs to be trained based on the robustness enhanced data corresponding to the abnormal data and the unfair category data, the NDT system can also use one data enhancement report to feed back the data enhancement situation of the abnormal data and the data enhancement situation of the unfair category data. Correspondingly, the robustness enhanced data corresponding to the abnormal data and the robustness enhanced data corresponding to the unfair category data can also be sent together, which is not limited.

[0283] In this implementation, the model training function can perform model training according to the robustness enhanced data corresponding to the abnormal data and the robustness enhanced data corresponding to the unfair category data. For specific model training process, please refer to the description in S603 above, which is not limited.

[0284] Correspondingly, the model update information and the information indicating the parameters for data augmentation that need to be updated fed back by the model training function to the NDT are determined according to the model training results of the robustness augmented data corresponding to the abnormal data and the robustness augmented data corresponding to the unfair category data.

[0285] Of course, when the model training function needs to be trained based on the robustness augmented data corresponding to the abnormal data, the high uncertainty data and the unfair category data, the NDT system can also use one data augmentation report to feed back the data augmentation situations of the three kinds of data. Correspondingly, the robustness augmented data corresponding to the abnormal data, the high uncertainty data and the unfair category data can also be sent together, which is not limited.

[0286] At this time, the model training function can perform model training according to the robustness augmented data corresponding to the abnormal data, the robustness augmented data corresponding to the high uncertainty data and the robustness augmented data corresponding to the unfair category data. The specific model training process can refer to the related description in S603 above, which is not limited.

[0287] Correspondingly, the model update information and the information indicating the parameters for data augmentation that need to be updated fed back by the model training function to the NDT are determined according to the model training results of the robustness augmented data corresponding to the abnormal data, the robustness augmented data corresponding to the high uncertainty data and the robustness augmented data corresponding to the unfair category data.

[0288] Therefore, on the basis of the robustness augmented data corresponding to the abnormal data or the robustness augmented data corresponding to the abnormal data and the robustness augmented data corresponding to the high uncertainty data, the requirement for fairness of the model training data is newly added, the robustness augmented data corresponding to the unfair category is generated based on the fairness, and this part of data is also sent to the model training function for adversarial training. This operation enables the target model to automatically learn the fairness related knowledge in the (re)training process, can better realize the value alignment of the target model, and finally realizes the model robustness enhancement based on data fairness.

[0289] It should be understood that the embodiments of the present application do not limit the execution order between S601-S603, S604-1-S604-3 and S605-1-S605-3.

[0290] In the embodiments of the present application, the training data set can also be referred to as a training sample set, the abnormal data can also be referred to as abnormal samples, the adversarial data can also be referred to as adversarial samples, the high uncertainty data can also be referred to as high uncertainty samples, the unfair category data can also be referred to as unfair category samples, and the robustness enhanced data can also be referred to as robustness enhanced samples, without limitation.

[0291] The model robustness enhancement method provided by the embodiments of the present application will be described in detail below in combination with specific implementations. For example, as shown in FIG. 7, a flowchart of a model robustness enhancement method provided by the embodiments of the present application can include the following steps:

[0292] S700-1, the trusted AI management function, the model training function, and the model inference function obtain a model identifier.

[0293] The model identifier is used to identify a training model. The model inference function can also obtain training data collection strategy information.

[0294] S700-2, the trusted AI management function sends the model identifier and data enhancement parameter information to the NDT system. Correspondingly, the NDT system receives the model identifier and data enhancement parameter information from the trusted AI management function.

[0295] The data enhancement parameter information is used to indicate related parameters of data enhancement of the NDT system, such as a parameter list of a data enhancement algorithm, a dimension of a solution space, a precision of generated data, etc.

[0296] S701, the trusted AI management function generates a trusted management strategy.

[0297] The trusted management strategy, such as model robustness enhancement and data privacy protection, can be specifically described in the related description of the existing implementation, and will not be repeated here. The trusted management strategy in the embodiments of the present application mainly refers to model robustness enhancement.

[0298] S702, the trusted AI management function sends abnormal data detection indication information and model update strategy indication information to the model training function.

[0299] The abnormal data detection indication information can include an abnormal data detection algorithm ID, parameter values related to the abnormal data detection algorithm, and an abnormal data processing strategy. The abnormal data processing strategy is used to instruct the model training function to send the detected abnormal data to the NDT system for data enhancement, which corresponds to the first information described above. The related description of the first information will not be repeated here.

[0300] The model update strategy indication information can include an algorithm of model training, a required prediction accuracy of the model, etc.

[0301] Optionally, the trusted AI management function can also send high uncertainty data detection indication information and / or unfair class data detection indication information to the model training function. The high uncertainty data detection indication information can include a high uncertainty data sampling algorithm ID, a parameter value related to the high uncertainty data sampling algorithm, and a high uncertainty data processing strategy, wherein the high uncertainty data processing strategy is used to instruct the model training function to send the detected high uncertainty data to the NDT system for data augmentation. The above corresponds to the second information, and specific descriptions can be found in the description of the second information above, which will not be repeated here.

[0302] The unfair class data detection indication information can include an unfair class data clustering algorithm ID, an unfair class data sampling algorithm ID, a parameter value related to the unfair class data clustering algorithm, a parameter value related to the sampling algorithm, and an unfair class data processing strategy, wherein the unfair class data processing strategy is used to instruct the model training function to send the detected unfair class data to the NDT system for data augmentation. The above corresponds to the third information, and specific descriptions can be found in the description of the third information above, which will not be repeated here.

[0303] S703, the model training function detects abnormal data and labels the abnormal data.

[0304] S704, the model training function sends the abnormal data and data augmentation indication information #1 to the NDT system. Correspondingly, the NDT system receives the abnormal data and data augmentation indication information #1 from the model training function.

[0305] The data augmentation indication information #1 is used to instruct the NDT system to perform data augmentation on the abnormal data, which corresponds to the first indication information above. Specific descriptions can be found in the description of the first indication information above, which will not be repeated here.

[0306] S705, the NDT system performs data augmentation on the abnormal data according to the data augmentation indication information #1 to generate robustness enhanced data based on abnormal behavior.

[0307] The robustness enhanced data based on abnormal behavior corresponds to the robustness enhanced data corresponding to the abnormal data above. Specific descriptions can be found in the description of S602 above, which will not be repeated here.

[0308] S706, the NDT system sends the robustness enhanced data based on abnormal behavior to the model training function. Correspondingly, the model training function receives the robustness enhanced data based on abnormal behavior from the NDT system.

[0309] Optionally, the NDT system can also send a data augmentation report corresponding to the abnormal data to the model training function.

[0310] Correspondingly, the model training function can perform model training according to the robustness enhancement data based on abnormal behavior to obtain an updated model. The specific implementation of the model training function performing model training can be referred to the related description in S603 above, and details are not described herein.

[0311] S707, the model training function samples high uncertainty data.

[0312] The related description of the model training function sampling high uncertainty data can be referred to the related description in S604-1 above, and details are not described herein.

[0313] S708, the model training function sends the high uncertainty data and data enhancement indication information #2 to the NDT system. Correspondingly, the NDT system receives the high uncertainty data and data enhancement indication information #2 from the model training function.

[0314] The data enhancement indication information #2 is used to instruct the NDT system to perform data enhancement on the high uncertainty data, which corresponds to the second indication information described above. Details can be referred to the related description of the second indication information, and details are not described herein.

[0315] S709, the NDT system performs data enhancement on the high uncertainty data according to the data enhancement indication information #2 to generate robustness enhancement data based on counterfactuals.

[0316] The robustness enhancement data based on counterfactuals corresponds to the robustness enhancement data corresponding to the high uncertainty data described above. Details can be referred to the related description in S604-2 above, and details are not described herein.

[0317] S710, the NDT system sends the robustness enhancement data based on counterfactuals to the model training function. Correspondingly, the model training function receives the robustness enhancement data based on counterfactuals from the NDT system.

[0318] Optionally, the NDT system can also send the data enhancement report corresponding to the high uncertainty data to the model training function.

[0319] At this time, the model training function can perform model training according to the robustness enhancement data based on abnormal behavior and the robustness enhancement data based on counterfactuals to obtain an updated model. The specific implementation of the model training function performing model training can be referred to the related description in S603 or S604-3 above, and details are not described herein.

[0320] S711, the model training function performs clustering and sampling of the high uncertainty data for fairness to obtain unfair category data.

[0321] The specific implementation of S711 can be referred to the related description in S605-1 above, and details are not described herein.

[0322] S712, the model training function sends unfair category data and data augmentation instruction information #3 to the NDT system. Correspondingly, the NDT system receives the unfair category data and data augmentation instruction information #3 from the model training function.

[0323] The data augmentation instruction information #3 is used to instruct the NDT system to perform data augmentation on the unfair category data, which corresponds to the third instruction information described above. For details, please refer to the description of the third instruction information above, which will not be repeated here.

[0324] S713, the NDT system performs data augmentation on the unfair category data according to the data augmentation instruction information #3 to obtain fairness-based robustness enhancement data.

[0325] The fairness-based robustness enhancement data corresponds to the robustness enhancement data corresponding to the unfair category data. For details of the implementation process of S713, please refer to the description of S605-2 above, which will not be repeated here.

[0326] S714, the NDT system sends the fairness-based robustness enhancement data to the model training function. Correspondingly, the model training function receives the fairness-based robustness enhancement data from the NDT system.

[0327] At this time, the model training function can perform model training according to the abnormal behavior-based robustness enhancement data, the counterfactual-based robustness enhancement data, and the fairness-based robustness enhancement data to obtain an updated model. For details of the implementation of model training by the model training function, please refer to the description of S603 or S604-3 above, which will not be repeated here.

[0328] S715, the model training function sends updated model information to the model inference function. Correspondingly, the model inference function receives the updated model information from the model training function.

[0329] The updated model information corresponds to the model parameter information of the updated model described above. The updated model information can include model identification, updated model parameters, model address, etc.

[0330] S716, the model training function sends the updated model information to the NDT system. Correspondingly, the NDT system receives the updated model information from the model training function.

[0331] Optionally, the model training function can also send data augmentation parameters updated based on the training results of the model training based on the three types of robustness enhancement data to the NDT system to further improve the robustness of the model.

[0332] S717、The model training function sends the updated model information and the trusted related log to the trusted AI management function. Correspondingly, the trusted AI management function receives the updated model information and the trusted related log from the model training function.

[0333] The trusted related log can record information related to the execution of the trusted management policy (such as model robustness enhancement) by the model training function.

[0334] In the above embodiments, the methods and / or steps implemented by the trusted AI management function can also be implemented by components (such as processors, chips, chip systems, circuits, logic modules, or software) available to the trusted AI management function; the methods and / or steps implemented by the model training function can also be implemented by components (such as processors, chips, chip systems, circuits, logic modules, or software) available to the model training function; and the methods and / or steps implemented by the NDT system can also be implemented by components (such as processors, chips, chip systems, circuits, logic modules, or software) available to the NDT system.

[0335] The above mainly introduces the schemes provided in the present application. Correspondingly, the present application also provides a communication device for implementing various methods in the above method embodiments. The communication device can be the trusted AI management function in the above method embodiments, or a device containing the trusted AI management function, or a component available to the trusted AI management function, such as a chip or a chip system. Alternatively, the communication device can be the model training function in the above method embodiments, or a device containing the model training function, or a component available to the model training function, such as a chip or a chip system. Alternatively, the communication device can be the NDT system in the above method embodiments, or a device containing the NDT system, or a component available to the NDT system, such as a chip or a chip system.

[0336] In some embodiments, the communication device contains hardware structures and / or software modules for implementing various functions. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is driven by hardware or computer software to drive hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0337] The embodiments of the present application can divide the functional modules of the communication device according to the method embodiments described above. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be realized in the form of hardware or in the form of a software functional module. It should be noted that the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. When actually implemented, another division method can be used.

[0338] Taking the communication device as the trusted AI management function or the model training function or the NDT system in the method embodiments described above, FIG. 8 is a structural schematic diagram of a communication device provided by an embodiment of the present application. As shown in FIG. 8, the communication device 800 includes a processing module 801 and a communication module 802. The processing module 801 is configured to perform the processing function of the trusted AI management function or the model training function or the NDT system in the method embodiments described above. The communication module 802 is configured to perform the communication function of the trusted AI management function or the model training function or the NDT system in the method embodiments described above.

[0339] The above method embodiments involve all related contents of each step, which can be cited to the function description of the corresponding functional module, and will not be repeated here.

[0340] In a possible design scheme, in the embodiments of the present application, the communication module 802 can include a receiving module and a sending module (not shown in FIG. 8). The sending module and the receiving module are respectively configured to realize the sending function and the receiving function of the communication device 800.

[0341] In a possible design scheme, the communication device 800 can further include a storage module (not shown in FIG. 8), which stores a program or instructions. When the processing module 801 executes the program or instructions, the communication device 800 can execute the functions of the trusted AI management function or the model training function or the NDT system in the method shown in any one of FIGS. 6 or 7.

[0342] In some embodiments, the processing module 801 involved in the communication device 800 can be realized by a processor or a processor-related circuit component, and can be a processor or a processing unit. The communication module 802 can be realized by a transceiver or a transceiver-related circuit component, and can be a transceiver or a receiving unit.

[0343] Exemplarily, FIG. 9 is a structural schematic diagram of another communication apparatus provided by an embodiment of the present application. The communication apparatus can be the trusted AI management function or the model training function or the NDT system in the above method embodiments, or can be a chip (system) or other components or assemblies that can be arranged in the trusted AI management function or the model training function or the NDT system. As shown in FIG. 9, the communication apparatus 900 can include a processor 901, a bus 902, a communication interface 903, and a memory 904. The processor 901, the memory 904, and the communication interface 903 communicate with each other through the bus 902. The communication apparatus 900 can be the trusted AI management function or the model training function or the NDT system. It should be understood that the number of processors and memories in the communication apparatus 900 is not limited by the present application.

[0344] The bus 902 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in FIG. 9, but it does not mean that there is only one bus or only one type of bus. The bus 902 can include a path for transmitting information between various components (e.g., the memory 904, the processor 901, the communication interface 903) of the communication apparatus 900.

[0345] The processor 901 can include any one or more of a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0346] The memory 904 can include a volatile memory, such as a random access memory (RAM). The processor 901 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0347] The communication interface 903 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, etc., to realize the communication between the communication apparatus 900 and other devices or communication networks.

[0348] The executable program code is stored in the memory 904, and the processor 901 executes the executable program code to implement the functions of the first device or the second device in the foregoing method embodiments respectively. That is, the memory 904 stores instructions for executing the foregoing method.

[0349] In another aspect, the embodiments of the present application further provide a computer program product containing instructions, which, when executed on a communication device, enable the communication device to perform the method according to any one of the foregoing embodiments.

[0350] In another aspect, the embodiments of the present application further provide a computer readable storage medium. The computer readable storage medium stores computer programs or instructions, which, when executed on a communication device, enable the communication device to perform the method according to any one of the foregoing embodiments.

[0351] In another aspect, the embodiments of the present application further provide a communication system, which includes a model training function and an NDT system for implementing the foregoing method embodiments.

[0352] Optionally, the communication system can further include a trusted AI management function and / or a model training function for implementing the foregoing method embodiments.

[0353] In the foregoing embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or include one or more data storage devices such as servers, data centers, etc. that can be integrated with the medium. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, SSD), etc.

[0354] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0355] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0356] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0357] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0358] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0359] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or an access network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, random access memory (RAM), magnetic disks, or optical disks.

[0360] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0361] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A model robustness enhancement method, characterized by, The method comprises: The model training function sends abnormal data to a network digital twin NDT system; The model training function receives robustness enhancement data corresponding to the abnormal data from the NDT system, which is used for model training.

2. The method of claim 1, wherein, The method further comprises: The model training function receives first information from a trusted artificial intelligence AI management function, which is used to instruct the model training function to send the abnormal data detected from a training data set to the NDT system for data enhancement; The model training function sends abnormal data to the NDT system, comprising: The model training function sends the abnormal data to the NDT system according to the first information.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: The model training function sends first indication information to the NDT system, which is used to instruct the NDT system to perform data enhancement on the abnormal data.

4. The method according to any one of claims 1-3, characterized in that, The method comprises: The model training function performs model training according to the robustness enhancement data corresponding to the abnormal data.

5. The method according to any one of claims 1-3, characterized in that, The method further comprises: The model training function sends high uncertainty data to the NDT system, which is data in the training data set that makes the confidence of model prediction lower than a first preset threshold; The model training function receives robustness enhancement data corresponding to the high uncertainty data from the NDT system, which is used for model training.

6. The method of claim 5, wherein, The method further comprises: The model training function receives second information from a trusted AI management function, which is used to instruct the model training function to send the high uncertainty data sampled from the training data set to the NDT system for data enhancement; The model training function sends high uncertainty data to the NDT system, comprising: The model training function sends the high uncertainty data to the NDT system according to the second information.

7. The method of claim 6, wherein, The second information includes information indicating an algorithm for sampling the high uncertainty data.

8. The method according to any one of claims 5-7, characterized in that, The method further comprises: The model training function sends second indication information to the NDT system, which is used to instruct the NDT system to perform data enhancement on the high uncertainty data.

9. The method according to any one of claims 5-8, characterized in that, The method comprises: The model training function performs model training according to the robustness enhancement data corresponding to the abnormal data and the robustness enhancement data corresponding to the high uncertainty data.

10. The method according to any one of claims 5-8, characterized in that, The method further comprises: The model training function clusters the high uncertainty data sampled from the training data set for fairness, obtaining data of different categories; The model training function samples data of a category whose fairness bias is greater than a second preset threshold from the data of different categories, obtaining unfair category data; The model training function sends the unfair category data to the NDT system; The model training function receives robustness enhanced data corresponding to the unfair category data from the NDT system, and the robustness enhanced data corresponding to the unfair category data is used for model training.

11. The method of claim 10, wherein, The method further comprises: The model training function receives third information from the trusted AI management function, and the third information is used to instruct the model training function to acquire robustness enhanced data corresponding to unfair category data from the NDT system for model training; The model training function sends the unfair category data to the NDT system, comprising: The model training function sends the unfair category data to the NDT system according to the third information.

12. The method of claim 11, wherein, The third information comprises information indicating a clustering algorithm targeting fairness and information indicating an algorithm for sampling data after clustering targeting fairness.

13. The method according to any one of claims 10-12, characterized in that, The method further comprises: The model training function sends third indication information to the NDT system, and the third indication information is used to instruct the NDT system to perform data enhancement on the unfair category data.

14. The method according to any one of claims 10-13, characterized in that, The method further comprises: The model training function performs model training according to the robustness enhanced data corresponding to the abnormal data, the robustness enhanced data corresponding to the high uncertainty data, and the robustness enhanced data corresponding to the unfair category data.

15. The method of any one of claims 1-14, wherein, The method further comprises: The model training function sends model training updated model parameter information to the NDT system.

16. The method of any one of claims 1-15, wherein, The method further comprises: After model training according to the robustness enhanced data, the model training function sends information indicating that the parameters for data enhancement need to be updated to the NDT system.

17. A model robustness enhancement method, comprising: The method comprises: The NDT system receives abnormal data from the model training function; The NDT system performs data enhancement on the abnormal data to obtain robustness enhanced data corresponding to the abnormal data, and the robustness enhanced data corresponding to the abnormal data is used for model training; The NDT system sends the robustness enhanced data corresponding to the abnormal data to the model training function.

18. The method of claim 17, wherein, The NDT system performs data enhancement on the abnormal data to obtain robustness enhanced data corresponding to the abnormal data, comprising: The NDT system learns abnormal behavior according to the abnormal data; The NDT system generates robustness enhanced data corresponding to the abnormal data according to the abnormal behavior.

19. The method of claim 17 or 18, wherein, The method further comprises: The NDT system receives first indication information from the model training function, and the first indication information is used to instruct the NDT system to perform data enhancement on the abnormal data; The NDT system performs data enhancement on the abnormal data to obtain robustness enhanced data corresponding to the abnormal data, comprising: The NDT system performs data enhancement on the abnormal data according to the first indication information to obtain robustness enhanced data corresponding to the abnormal data.

20. The method of any one of claims 17-19, wherein, The method further comprises: The NDT system receives high uncertainty data from the model training function, and the high uncertainty data is data in a training data set that makes the confidence of model prediction lower than a first preset threshold; The NDT system performs data enhancement on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data, and the robustness-enhanced data corresponding to the high-uncertainty data is used for model training. The NDT system sends the robustness-enhanced data corresponding to the high-uncertainty data to the model training function.

21. The method of claim 20, wherein, The NDT system performs data enhancement on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data, and the robustness-enhanced data corresponding to the high-uncertainty data is used for model training. The NDT system performs model prediction according to the high-uncertainty data to obtain a first prediction result. The NDT system determines a counterfactual sample generation algorithm according to the first prediction result. The NDT system obtains the robustness-enhanced data corresponding to the high-uncertainty data according to the counterfactual sample generation algorithm and the high-uncertainty data.

22. The method of claim 20 or 21, wherein, The method further comprises: The NDT system receives second indication information from the model training function, and the second indication information is used to instruct the NDT system to perform data enhancement on the high-uncertainty data. The NDT system performs data enhancement on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data, and the robustness-enhanced data corresponding to the high-uncertainty data is used for model training. The NDT system performs data enhancement on the high-uncertainty data to obtain robustness-enhanced data corresponding to the high-uncertainty data, and the robustness-enhanced data corresponding to the high-uncertainty data is used for model training.

23. The method of any one of claims 17-22, wherein, The method further comprises: The NDT system receives unfair category data from the model training function, and the unfair category data is data of a category with a fairness bias greater than a second preset threshold. The NDT system performs data enhancement on the unfair category data to obtain robustness-enhanced data corresponding to the unfair category data, and the robustness-enhanced data corresponding to the unfair category data is used for model training. The NDT system sends the robustness-enhanced data corresponding to the unfair category data to the model training function.

24. The method of claim 23, wherein, The NDT system performs data enhancement on the unfair category data to obtain robustness-enhanced data corresponding to the unfair category data, and the robustness-enhanced data corresponding to the unfair category data is used for model training. The NDT system performs model prediction according to the unfair category data to obtain a second prediction result. The NDT system determines a fairness sample generation algorithm according to the second prediction result. The NDT system obtains the robustness-enhanced data corresponding to the unfair category data according to the fairness sample generation algorithm and the unfair category data.

25. The method of claim 23 or 24, wherein, The method further comprises: The NDT system receives third indication information from the model training function, and the third indication information is used to instruct the NDT system to perform data enhancement on the unfair category data. The NDT system performs data enhancement on the unfair category data to obtain robustness-enhanced data corresponding to the unfair category data, and the robustness-enhanced data corresponding to the unfair category data is used for model training. The method further comprises:

26. The method of any one of claims 17-25, wherein, ​ The NDT system receives model training updated model parameter information from the model training function.

27. The method of any one of claims 17-26, wherein, The method further includes: The NDT system receives information from the model training function indicating that parameters for data augmentation need to be updated.

28. The method of any one of claims 17-27, wherein, The method further includes: The NDT system receives information from a trusted artificial intelligence (AI) management function indicating parameters for the NDT system to perform data augmentation.

29. A model robustness enhancement method, comprising: The method includes: The trusted AI management function obtains first information indicating that a model training function sends abnormal data detected from a training data set to an NDT system for data augmentation. The trusted AI management function sends the first information to the model training function.

30. The method of claim 29, wherein, The method further includes: The trusted AI management function sends second information to the model training function, the second information indicating that the model training function sends high uncertainty data sampled from the training data set to the NDT system for data augmentation, the high uncertainty data being data in the training data set that causes a model prediction to have a confidence level below a first preset threshold.

31. The method of claim 30, wherein, The second information includes information indicating an algorithm for sampling the high uncertainty data.

32. The method of any one of claims 29-31, wherein, The method further includes: The trusted AI management function sends third information to the model training function, the third information indicating that the model training function performs model training using robustness enhancement data corresponding to unfair category data obtained from the NDT system, the unfair category data being data of a category having a fairness bias greater than a second preset threshold.

33. The method of claim 32, wherein, The third information includes information indicating a clustering algorithm targeting fairness and information indicating an algorithm for sampling data after targeting fairness.

34. The method of any one of claims 29-33, wherein, The method further includes: The trusted AI management function sends information to the NDT system indicating parameters for the NDT system to perform data augmentation.

35. A communications device, characterized by A module for performing the method of any of claims 1-16, or 17-28, or 29-34.

36. A communications device, characterized by A processor; The processor is configured to execute a computer program or instructions to cause the method of any of claims 1-16, or 17-28, or 29-34 to be implemented. The chip has instructions stored therein that, when the chip is executed on a communication device, cause the method of any of claims 1-16, or 17-28, or 29-34 to be implemented.

37. A communication chip, comprising: A module for performing the method of any of claims 1-16, and an NDT system for performing the method of any of claims 17-28.

38. A communication system, characterized by A trusted AI management function for performing the method of any of claims 29-34. The computer readable storage medium has a computer program or instructions stored therein that, when executed by a communication device, cause the method of any of claims 1-16, or 17-28, or 29-34 to be implemented.

39. The system of claim 38, wherein, ​ ​ 40. A computer-readable storage medium, comprising: ​ 41. A computer program product, characterised in that, including computer program code which, when run on a communications device, implements the method of any of claims 1-16, or 17-28, or 29-34.

Images