24 results about "Gradient noise" patented technology

The invention discloses a large model training method, medium and system based on a differential privacy mechanism, and the method comprises the steps: S101, obtaining historical data, carrying out the preprocessing, and carrying out the grouping and marking of the preprocessed historical data, so as to form a training data set; s102, initializing parameters of the large language model; s103, performing training based on the training data set, and calculating a gradient; s104, performing gradient noise addition to obtain a noise addition gradient, and calculating a corresponding comprehensive unfairness index value based on the noise addition gradient; s105, judging whether the comprehensive unfairness index value is within a preset value range or not; s106, if the comprehensive unfair index value is within a preset value range, judging whether the current large language model meets a training requirement or not; if yes, taking the current large language model as a final model; and if not, returning to the step S103. According to the invention, the privacy of the user can be effectively protected, and meanwhile, the unfair phenomenon caused by the use of a differential privacy mechanism is avoided.

The invention discloses an AI model training data protection method and device based on local differential privacy. The method comprises the steps of noise scale determination, wherein the noise scale is determined through a Laplace differential privacy mechanism according to a preset privacy budget value and obtained input data; a data noise adding step: generating Laplacian noise according to the noise scale, and adding the Laplacian noise to the input data; a gradient noise adding step: determining the gradient of each data point in the input data after the Laplacian noise is added according to a preset initial model training parameter, and adding the Laplacian noise to the gradient of each data point to obtain a noise-added gradient value; and a parameter determination step: determining a prediction model training parameter according to the noise-added gradient value. According to the invention, the accuracy and efficiency of model training can be improved under the condition of effectively avoiding privacy leakage of training data.

The embodiment of the invention relates to the field of time series anomaly detection, in particular to a time series data anomaly detection method and device of a dynamic condition diffusion model based on gradient noise. A specific embodiment of the method comprises the following steps: determining a GNDC-DM model; performing trend noise detection processing on the acquired initial industrial equipment data based on a trend noise diffusion model included in the GNDC-DM model to generate trend noise reduction data; performing seasonal noise detection processing on the obtained initial industrial equipment data based on a seasonal noise diffusion model included in the GNDC-DM model to generate seasonal noise reduction data; and performing hybrid anomaly detection processing on the acquired initial industrial equipment data, trend noise reduction data and seasonal noise reduction data based on a hybrid anomaly diffusion model included in the GNDC-DM model to generate equipment noise reduction data. According to the embodiment, the detection based on the reconstruction score can be more effective.

The invention relates to a longitudinal federal learning method and device based on feature importance and electronic equipment, and aims to realize a local privacy budget generator of a client through a lightweight neural network by taking local feature importance as drive. Therefore, a cooperative closed loop from'feature-level privacy budget 'to'embedded dimension disturbance' to'suggested privacy budget uploading 'to'server-side gradient noise addition' is realized, and meanwhile, discrete grading and suggested privacy budget disturbance mechanisms are introduced to prevent the server-side from reversely deducing privacy contents, so that the differential privacy security and practicability of the whole system are improved.

The invention discloses a model decision interpretability method fusing integral gradient and class activation mapping, and belongs to the technical field of deep learning interpretability. Aiming at the problems of gradient noise interference, insufficient space positioning and the like existing in an existing single interpretability method, the method is optimized through five key steps: firstly, extracting a feature map set of the last convolutional layer of a deep learning model; secondly, calculating an integral gradient of the feature map to a target category based on a path integral idea; thirdly, obtaining a feature map weight through global average pooling; then, weighted summation is carried out, and an initial attribution thermodynamic diagram is generated through ReLU activation; and finally, a high-resolution thermodynamic diagram is obtained in combination with guided gradient optimization. According to the method, gradient stability of Integrating Gradients and spatial positioning advantages of Grad-CAM are fused, invariance and sensitivity axioms are realized, model adaptability is maintained, accuracy and robustness of interpretation results are remarkably improved, and the method is suitable for key fields such as signal processing and image classification.

The application belongs to the technical field of artificial intelligence, and particularly relates to a private AI model calling method and system based on encrypted data interaction, which comprises the following steps: S1, a client generates a series of structured random numbers through a key derivation function based on a main random number seed and each dimension index, encrypts an original input vector into an initial ciphertext vector by using a partial homomorphism encryption public key and the structured random numbers, generates a conditional re-encoding key pair for nonlinear calculation in the AI model, and sends a re-encoding public key to a server. The application can eliminate the confusion ciphertext, so that the server side constructs a quasi-gradient noise based on the intermediate state of the calculation process and injects the result, resisting side channel attacks on the output ciphertext; and the client can reconstruct and remove the noise, so that the lossless calling result is obtained without sacrificing any calculation accuracy, and end-to-end privacy protection of the whole model reasoning process is realized.

The invention discloses a neural network training method, a neural network training device, electronic equipment and a computer readable storage medium, and the neural network training method is at least applied to a visual processing scene, and comprises the steps: inputting a training sample into a to-be-trained neural network, and obtaining an output result of the neural network under each current connection weight; the training sample at least comprises visual data; determining a first gradient of each connection weight based on the output result and the labeling information of the training sample; directed gradient noise is generated for each connection weight; for the target connection weight meeting the relation requirement of the first gradient and a set gradient threshold value, the directional gradient noise corresponding to the target connection weight and the first gradient are fused to obtain a fusion gradient, and updating of the target connection weight is achieved according to the fusion gradient; and the training is repeated until the obtained neural network meets the set requirement. The neural network training efficiency can be improved, the training time is shortened, and the training cost is reduced.

The application belongs to the technical field of model training, and provides a differential privacy federated learning method based on noise scale allocation and related equipment. Embodiments of the application obtain a total noise scale sequence of a target client under a total training step number, and allocate a noise scale sub-sequence to a plurality of local training steps of the target client in a current global iteration based on the total noise scale sequence; the total noise scale sequence is an increasing sequence determined by minimizing the convergence upper bound of a global model corresponding to a server under the constraint of a total privacy budget of the target client; the target client performs the current global iteration based on a decreasing learning rate sequence and a mini-batch stochastic gradient descent algorithm, and performs gradient noise processing according to the noise scale sub-sequence. Finally, the current global model is updated based on the aggregated cumulative noise gradient. Embodiments of the application can be applied to differential privacy federated learning of mini-batch local training, and can improve model performance while protecting privacy.

The present application discloses a training method for a neural network, a training device for a neural network, an electronic device, and a computer-readable storage medium. The training method for the neural network is at least applied to a visual processing scenario and includes: inputting a training sample into the neural network to be trained to obtain an output result of the neural network under each current connection weight; the training sample at least includes visual data; determining a first gradient of each connection weight based on the output result and the annotation information of the training sample; generating directional gradient noise for each connection weight respectively; for a target connection weight that satisfies the relationship requirement between the first gradient and a set gradient threshold, fusing the directional gradient noise corresponding to the target connection weight and the first gradient to obtain a fused gradient, and updating the target connection weight according to the fused gradient; repeating the training until the obtained neural network meets the set requirements. It can improve the training efficiency of the neural network, reduce the training time, and compress the training cost.

Embodiments of the present application provide a communication method and apparatus, and a storage medium. The method comprises: receiving first information from multiple terminals, wherein the first information indicates a first local parameter, a second local parameter, and a third local parameter; on the basis of the total power of gradient noise and the first information from the multiple terminals, obtaining an update step size; and sending an updated global model or the update step size to the multiple terminals, respectively, wherein the updated global model is obtained by processing the global model on the basis of the update step size. The step size is dynamically adjusted on the basis of the gradient noise and a training state indicated by each local parameter, thus accelerating the gradient descent of neural network learning, and solving the problem of slow neural network training or even non-convergence in the presence of gradient noise. By accelerating the neural network training process, "Green AI" can be achieved, and the communication overhead can be saved. The present solution can be compatible with digital transmission of federated learning, analog transmission, and over-the-air aggregation.

The present invention discloses an image generation method based on a gradient noise suppression generative adversarial network, the method comprising the following steps: obtaining a training set for a positive sample of a discriminator; generating an image set as a negative sample; training a gradient noise perceptron; setting training parameters in GNS-GAN adversarial learning, calculating and updating the weights of the discriminator and the generator; finally, using the generator to generate an image and perform pixel value scale exchange to complete the generation of the final image. The advantages of the present invention are: improving the quality of the generated image and the stability of the adversarial learning, effectively reducing the influence of noise in the training process through the gradient noise suppression technology; reducing the training time and computing resource consumption, avoiding the complex forward and reverse diffusion steps in the traditional diffusion model; improving the network structure, making the number of parameters of the generator and the discriminator smaller, and improving the training efficiency of the model.

The invention provides a refraction measurement system and method, and the method comprises the steps: determining angular spectrum information related to the refraction attribute of a target sample through a sample image of each illumination angle in the illumination process of the target sample through a light-emitting diode array, carrying out the enhancement processing of the angular spectrum information in each sample image, and obtaining a feature enhancement image set; determining refraction interference characteristics of the target sample in the refraction measurement process, and performing gradient noise reduction on refraction interference of the target sample according to the refraction interference characteristics and the characteristic enhancement image set to obtain refraction phase gradient characteristics of the target sample after refraction noise reduction; determining quantitative phase distribution of refraction of the target sample according to a preset refraction characteristic quantity and a phase gradient characteristic of refraction of the target sample; and determining diopter distribution of the target sample according to the quantitative phase distribution and the refractive index and thickness of the target sample. By adopting the scheme of the invention, the influence of optical interference and noise on refraction measurement can be overcome under multi-angle illumination.

The application provides a time step mixed cross-modal pulse neural network training method and system. The method comprises the following steps: constructing a static modal time sequence and an event modal time sequence; constructing a mixed modal time sequence according to a target mixing ratio; calculating an event stream classification loss; calculating a mixed stream classification loss; generating a domain alignment loss; generating a modal perception loss; calculating a mixed proportion perception loss; and fusing the event stream classification loss, the mixed stream classification loss, the domain alignment loss, the modal perception loss and the mixed proportion perception loss to update parameters of the pulse neural network. The application can significantly reduce the variance of the gradient in the training process, thereby reducing the gradient noise, improving the training stability and convergence speed of the pulse neural network, and can be directly integrated into a mainstream pulse neural network or a neural network training pipeline with a time dimension, has low deployment cost and strong universality, and the model has higher generalization performance on event visual tasks, and has wide application prospects.

The present application relates to a longitudinal federated learning method and device based on feature importance, and an electronic device, which is driven by local feature importance, and realizes a private budget generator at the client end through a lightweight neural network, and further realizes a collaborative closed loop from "feature-level privacy budget" to "embedding dimension perturbation" to "recommended privacy budget upload" to "gradient noise addition at the server end", and introduces a discrete level division and a recommended privacy budget perturbation mechanism to avoid the server end from deducing privacy content, thereby improving the overall differential privacy security and practicability of the system.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a privatized AI model calling method and system based on encrypted data interaction, and the method comprises the following steps: S1, a client generates a series of structured random numbers through a key derivation function based on a main random number seed and indexes of all dimensions, an original input vector is encrypted into an initial ciphertext vector by using a partially homomorphic encryption public key and a structured random number, a conditional recoding key pair is generated for nonlinear calculation in an AI model, and a recoding public key is sent to a server. According to the method, the obfuscated ciphertext can be eliminated, so that a server side constructs quasi-gradient noise based on an intermediate state of a calculation process and injects a result, and side channel attacks on the output ciphertext are resisted; and the client can reconstruct and remove the noise, so that a lossless calling result is obtained on the premise of not sacrificing any calculation precision, and end-to-end privacy protection of the whole process of model reasoning is realized.

The invention discloses a data training optimization method and system, and relates to the technical field of optimization algorithms, and the method comprises the steps: constructing a frequency domain noise shaping filtering function through a gradient distribution entropy value, carrying out the Fourier transform of an original gradient tensor, and then applying the frequency domain noise shaping filtering function, and obtaining a filtering frequency domain gradient spectrum; reconstructing a filtering frequency domain gradient spectrum as an optimized gradient tensor through inverse Fourier transform; calculating a dynamic learning rate through an exponential decay formula according to the gradient distribution entropy; and carrying out next batch training on the updated data, and terminating the training when the entropy fluctuation intensity is smaller than an entropy fluctuation convergence threshold to obtain optimized data. By introducing phase alignment calibration, an entropy-driven regularization item and a convergence judgment mechanism based on entropy fluctuation intensity, the data updating quality and the intelligent level of training termination judgment are improved; the method has good interpretability and engineering application value, and is suitable for large-scale data training and various task scenes sensitive to gradient noise.

The present application relates to a kind of frequency domain-space domain fusion perception methods for 2.5D integrated circuit fast high-precision thermal prediction, the network includes frequency space thermal encoder module, frequency domain cross-scale interaction module, frequency domain-space domain hybrid loss function (FSL) and model input and pre-processing network (PPNet).Frequency space thermal encoder module combines adaptive multi-frequency embedding and 3D convolutional neural network, extracts high-to-low frequency thermal dissipation gradient feature and detailed spatial feature;Frequency domain cross-scale interaction module realizes multi-scale feature interaction through cross attention mechanism, solves semantic gap problem;Frequency domain-space domain hybrid loss function loss function combines spatial mean square error and frequency domain loss, suppresses high-frequency thermal gradient noise.The present application first realizes the thermal prediction of frequency domain-space domain dual-domain perception, significantly improves prediction accuracy and speed, and has excellent generalization ability, and can be widely applied to the thermal management field of 2.5D integrated circuit.

The invention provides a three-dimensional reconstruction model training method, a three-dimensional reconstruction method, a training device, electronic equipment, a computer readable storage medium and a computer program product, and relates to the technical field of computers. The method comprises the steps of calculating a reconstruction error of a pixel based on a difference between a predicted pixel value and a real pixel value of the pixel in an input image; determining a discarding probability of a Gaussian primitive corresponding to the pixel according to the reconstruction error of the pixel; wherein the Gaussian primitive is obtained by outputting the three-dimensional reconstruction model; and determining the reserved Gaussian primitives based on the discarding probability of the Gaussian primitives corresponding to the pixels in the input image, and updating the model parameters of the three-dimensional reconstruction model based on the reserved Gaussian primitives. According to the method, gradient noise caused by redundant Gaussian primitives is reduced, and the precision of the model is improved.

Embodiments of the present application relate to the field of time series anomaly detection, and particularly to a time series data anomaly detection method and device based on a gradient noise dynamic conditional diffusion model. A specific embodiment of the method comprises: determining a GNDC-DM model; performing trend noise detection processing on the obtained initial industrial equipment data based on a trend noise diffusion model included in the GNDC-DM model to generate trend denoising data; performing seasonal noise detection processing on the obtained initial industrial equipment data based on a seasonal noise diffusion model included in the GNDC-DM model to generate seasonal denoising data; and performing mixed anomaly detection processing on the obtained initial industrial equipment data, the trend denoising data and the seasonal denoising data based on a mixed anomaly diffusion model included in the GNDC-DM model to generate equipment denoising data. This embodiment can make the detection based on the reconstruction score more effective.

The invention relates to a distributed large model safety control system and method oriented to industrial multi-device cooperation, and belongs to the field of industrial artificial intelligence. The system comprises an industrial parameter acquisition module used for acquiring historical parameters of multiple types of equipment needing cooperative training; the local parameter encryption module is used for encrypting the historical parameters acquired by the industrial parameter acquisition module by adopting a mathematical algorithm optimization scheme of gradient noise adding synchronization points; the federated learning module is used for performing federated learning on the data encrypted by the local parameter encryption module to obtain global model parameters; the multi-modal fusion learning module is used for carrying out multi-modal fusion learning on the global model parameters and training a global model; and the reinforcement learning real-time optimization module is used for solving the conflict problem of multiple types of equipment through reinforcement learning and outputting a final task execution scheme. The method can solve the problems that in the prior art, sensitive data are prone to being leaked, the generalization ability of a single model in a complex production line is insufficient, and task allocation is unreasonable during multi-robot cooperation.

The invention belongs to the technical field of model training, and provides a differential privacy federated learning method based on noise scale distribution and related equipment. According to the embodiment of the invention, the method comprises the steps: obtaining a total noise scale sequence of a target client under a total training step number, and distributing noise scale subsequences for a plurality of local training steps of the target client in a current round of global iteration based on the total noise scale sequence; the total noise scale sequence is an incremental sequence determined by minimizing the convergence upper bound of a global model corresponding to the server under the constraint of the total privacy budget of the target client; and executing the current round of global iteration through the target client based on the progressively decreased learning rate sequence and a small-batch stochastic gradient descent algorithm, and performing gradient noise adding processing according to the noise scale subsequence. And finally, updating the current global model based on the aggregated accumulated noise-adding gradient. The method and device can be suitable for differential privacy federated learning of small-batch local training, and the model performance is improved while privacy protection is achieved.

The invention relates to a frequency domain-space domain fusion sensing method for rapid high-precision thermal prediction of a 2.5 D integrated circuit. A network comprises a frequency-space thermal encoder module, a frequency domain cross-scale interaction module, a frequency domain-space domain mixed loss function (FSL) and a model input and preprocessing network (PPNet). The frequency air-heat encoder module is combined with adaptive multi-frequency embedding and a 3D convolutional neural network to extract heat dissipation gradient features and detailed spatial features from high frequency to low frequency; the frequency domain cross-scale interaction module realizes multi-scale feature interaction through a cross-attention mechanism and solves the semantic gap problem; and the frequency domain-space domain mixed loss function is combined with the space mean square error and the frequency domain loss to suppress the high-frequency thermal gradient noise. According to the method, frequency domain-space domain double-domain sensing thermal prediction is realized for the first time, the prediction precision and speed are remarkably improved, and the method has excellent generalization ability and can be widely applied to the field of thermal management of 2.5 D integrated circuits.

The invention discloses a distributed machine learning-oriented multi-target automatic hyper-parameter configuration method and system, which are used for simultaneously optimizing model precision, training efficiency, safety and bandwidth efficiency in single training. The method comprises the steps that gradient noise scale (GNS) is used as a unified index, and a mathematical model between hyper-parameters and multi-target performance is established; training index data and GNS are monitored in real time in the training process, and hyper-parameters are dynamically adjusted; hyper-parameter optimization in single training is realized through lightweight analysis and closed optimization; a user-friendly API is provided, and the configuration process is simplified. According to the method, an efficient and safe automatic hyper-parameter configuration solution is provided, the overall performance of distributed machine learning is remarkably improved, experiments show that the training time of distributed machine learning can be remarkably shortened, and meanwhile model precision and safety are improved.

The invention discloses a semantic coding and decoding model generation method based on privacy protection, which relates to the field of semantic communication privacy protection, and comprises the following steps: acquiring semantic task requirements and shared values, and constructing a semantic coding and decoding model; generating a shared value set according to semantic coding and decoding model parameters and a local data set, and performing integer processing on the shared value set; dividing the integers of the shared value set into shared value proportions, and distributing the shared value proportions to participants; each participant inputs the shared value proportion into a deep neural network of a full connection layer, and performs iterative training of a local semantic coding model by using the held model parameter proportion and the local data proportion; parameters of the local semantic coding models are updated through a gradient noise addition iteration training process, and the trained local semantic coding models are obtained; and analyzing each local semantic coding model by using an analytic hierarchy process to obtain a corresponding weight, and carrying out weighted calculation to obtain a semantic coding and decoding model.