Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

284 results about "Cross entropy" patented technology

In information theory, the cross entropy between two probability distributions p and q over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set if a coding scheme used for the set is optimized for an estimated probability distribution q, rather than the true distribution p.

Multi-modal sentiment analysis method and system based on knowledge distillation and dynamic fusion mechanism

The invention provides a multi-modal sentiment analysis method based on knowledge distillation and a dynamic fusion mechanism. The multi-modal sentiment analysis method comprises the following steps: pre-training a single-modal teacher model; a single-mode teacher model is used for guiding the learning of a multi-mode student model, and through an interactive knowledge distillation mechanism, the middle layer probability distribution of the teacher model is used as a target to learn the correlation between modes and the cross-mode characteristics; interactive knowledge distillation comprises three loss functions: firstly, calculating the difference of output distribution of output layers of a teacher model and a student model, and defining the difference as cross-modal knowledge distillation loss; secondly, adding alignment loss based on a real label, and constraining a prediction result of the student model to be close to a real emotion label in a cross entropy form; and finally, introducing label smoothing loss to soften the real label. According to the method, the pre-trained single-mode teacher model with relatively good performance is stored and is used for guiding the learning of the multi-mode student model, meanwhile, the loss function is introduced to optimize the multi-mode student model, so that the multi-mode student model is gradually aligned with the output distribution of the teacher model, and meanwhile, the adaptive capacity of the multi-mode student model to the modal heterogeneity is enhanced. The single-mode teacher model greatly reduces the complexity of the model, reduces the calculation amount, and has better performance in the field of multi-mode sentiment analysis.
Owner:EAST CHINA UNIV OF SCI & TECH

Drawing machine operation state evaluation method and system based on deep learning

The invention discloses a wire drawing machine operation state evaluation method and system based on deep learning, and relates to the technical field of intelligent manufacturing, and the method comprises the steps: deploying a multi-source sensor at a preset part of a wire drawing machine, and forming a multi-dimensional operation data flow; the deep learning evaluation model is trained based on the standardized time series data set, model parameters are optimized through a cross entropy loss function, the model learns the characteristic difference between a normal working condition and an abnormal working condition, and the deep learning evaluation model is trained based on the standardized time series data set. And inputting a multi-dimensional operation data flow collected in real time into the trained deep learning evaluation model to generate a dynamic evaluation report. According to the wire drawing machine operation state evaluation method and system based on deep learning provided by the invention, a visual operation state score can be output, detailed fault type probability distribution is given, and intelligent support is provided for equipment maintenance decision.
Owner:HANGZHOU HARBOR TECH

Multi-modal artificial intelligence quality defect prediction method

The invention relates to the technical field of artificial intelligence manufacturing, and particularly discloses a multi-modal artificial intelligence quality defect prediction method, which comprises the following steps of: acquiring real-time multi-modal data from multiple sources, including text data, audio data, image data and sensor data, and comparing the real-time multi-modal data with historical data; performing feature extraction based on the accurate data; constructing a quality defect prediction model; realizing efficient quality defect prediction by using a deep learning technology; calculating a deviation between a predicted value and an actual value through a cross entropy loss function; according to the method, the prediction value is divided into accurate prediction and inaccurate prediction according to the preset threshold value, an early warning mechanism is established on the basis of accurate prediction, and early warning is carried out on the multi-modal data of the potential defects, so that correction measures are taken in time, and the product defect rate is reduced.
Owner:QINGDAO E LINGNET INTEGRATION CO LTD +1

Hybrid load flow calculation method and device based on cross entropy algorithm

The invention discloses a hybrid power flow calculation method and device based on a cross entropy algorithm, and belongs to the technical field of optimization algorithm and machine learning crossing. The method comprises the steps that system modeling and parameter initialization are carried out, and a state space, an action space and a target function are defined according to a specific application scene; setting and initializing row parameters; entropy algorithm-guided sample generation and elite screening: iteratively optimizing power grid parameters through probability distribution, and quickly approaching a global optimal solution; the strategy optimization target of the strategy gradient algorithm is that a dynamic strategy network is constructed based on optimized power grid parameters, and the running state of the power grid is adjusted in real time to deal with uncertainty; performing sample multiplexing and distribution fusion of importance sampling: setting a double-buffer mechanism and performing sample fusion and multiplexing; and parameter updating and convergence judgment. According to the method, global search is carried out through the cross entropy algorithm, local optimization is carried out through the strategy gradient algorithm, the sample utilization rate is improved through the importance sampling method, and cooperative processing of global optimization and local optimization is achieved.
Owner:STATE GRID ANHUI ELECTRIC POWER CO LTD ELECTRIC POWER SCI RES INST +1

Power system dominant instability mode identification method based on deep learning

The invention discloses an electric power system dominant instability mode identification method based on deep learning. The method comprises the steps of reading voltage, phase angle, active power and reactive power data in a time window of each node of an electric power system; calculating a node correlation coefficient matrix of the power system by means of a node correlation method, and calculating a time step correlation coefficient matrix by means of a spearman method; introducing the node correlation coefficient matrix into a graph convolutional neural network model, and fusing the time step correlation coefficient matrix into a bidirectional gating cycle unit model to obtain a GCN-BiGRU model capable of deeply mining spatio-temporal information; constructing a space-time integrated layer by using a multi-head attention mechanism, mining key space and time sequence features processed by GCN and Bi-GRU, and fusing space-time information in a mode of combining dimension reduction, remodeling and splicing; a cross entropy function with a weight is constructed as a loss function in training, and the problem that the number of various categories of dominant instability mode samples is unbalanced is solved; and optimizing the deep neural network model according to the final loss and carrying out dominant instability mode identification.
Owner:NORTHEAST DIANLI UNIVERSITY +1

Vector matching knowledge distillation method and system based on adaptive temperature

The invention discloses a vector matching knowledge distillation method and system based on adaptive temperature, and the method comprises the following steps: S1, inputting an image into a given teacher model and a given student model, carrying out the convolution processing of the image through the neural networks in the teacher model and the student model, and carrying out the processing through a full-link layer, a logistic regression logit output by the model is obtained, probability distribution characteristics of logits of the student model and logits of the teacher model are extracted through a softmax function, a self-adaptive temperature mechanism is introduced after the probability distribution characteristics are extracted, and temperature distillation is carried out; s2, vector features of probability distribution of the teacher model and the student model are extracted, and cosine distance loss, cross entropy loss and KL divergence loss are introduced to calculate total loss. The probability distribution characteristics of the student model and the teacher model are fully considered, the vector characteristics of the student model and the teacher model are extracted, a knowledge distillation characteristic system is expanded, the problem of non-uniform categories caused by knowledge distillation global pooling is solved, and the problem of distillation error increase caused by excessive mismatching of single data is greatly reduced.
Owner:ZHEJIANG SCI-TECH UNIV

Language model training method, device and system based on adaptive cross entropy and medium

The invention discloses a language model training method, device and system based on adaptive cross entropy and a medium, and the method comprises the steps: obtaining input data of a user, forming a training data set according to the input data and a standard reply, and inputting the training data set into a large model; the large model obtains input data in the training data set and generates a prediction reply; presetting a preset parameter, and multiplying the preset parameter by the cross entropy loss function to form an improved cross entropy loss function; calculating a loss value between the predicted reply and the standard reply using an improved cross entropy loss function; and updating large model parameters according to the loss value, and finishing large model training when a preset number of iterations is reached. The improved cross entropy loss function is used to calculate the loss value of the prediction reply and the standard reply, the model parameters are updated to train the model, the over-fitting phenomenon of the model is relieved during large model training, and the capacity of the large model is improved through fine tuning.
Owner:PING AN TECH (SHENZHEN) CO LTD

TF-IDF and cross entropy-based cue word compression method and system

The invention discloses a cue word compression method and system based on TF-IDF and cross entropy, belongs to the technical field of large model cue word compression, and aims to solve the problems that redundant information is introduced into long cue words, the model efficiency is reduced and the cost is increased. To-be-compressed content is divided into sentences at the sentence level and then converted into embedded vectors, and the Euclidean distance is calculated in combination with problem vectors so as to screen related sentences; calculating a TF-IDF value at the word level through a word frequency and an inverse document frequency to extract keywords and recombine sentences; and selecting a reference model and a basic model at the Token level, identifying the key Token based on a cross entropy loss difference value, and splicing the key Token in sequence to generate a compressed cue word. According to the method, a complex calculation structure is avoided, the inference efficiency is improved while the semantic integrity is maintained, and the resource consumption is reduced.
Owner:ARTIFICIAL INTELLIGENCE INNOVATION RES INST OF ZHEJIANG UNIV OF TECH BINJIANG DISTRICT HANGZHOU

Property collection monitoring and early warning method and system based on artificial intelligence

The invention provides a property collection monitoring and early warning method and system based on artificial intelligence. The method comprises the following steps: acquiring historical time series data of a user, preprocessing the historical time series data of the user, and extracting features from the preprocessed data; establishing a time weighted attention mechanism model based on features extracted from the divided training set; measuring the difference between the prediction probability of the time-weighted attention mechanism and the actual risk category by using a cross entropy loss function, and adjusting a time-weighted attention mechanism model; the method comprises the following steps: acquiring user time sequence feature data, inputting the user time sequence feature data into an adjusted time weighted attention mechanism model, and adopting a collection strategy according to a model prediction result. The time weighted attention mechanism and the weighted cross entropy loss function can be combined to improve the prediction accuracy of the model for the customer payment risk, so that the collection strategy is adopted in a targeted manner, the collection efficiency is improved, and the resource allocation is optimized.
Owner:HANGZHOU NEW WINDOWS INFORMATION TECH CO LTD

Magnetotelluric two-dimensional inversion method based on DeepLabV < 3 + >

The invention discloses a DeepLabV < 3 + >-based magnetotelluric two-dimensional inversion method, and belongs to the field of geophysical inversion methods, and the method comprises the steps: constructing a plurality of underground resistivity distribution models, and obtaining corresponding apparent resistivity and phase data through forward modeling calculation; designing a deep learning network based on a DeepLabV < 3 + > architecture, and enhancing the capability of capturing multi-scale geological features by using a cavity convolution and improved cavity spatial pyramid pooling (ASPP) module of the deep learning network; di ce Loss is adopted to replace a traditional cross entropy loss function, and the method is more suitable for the characteristics of anomaly and background information imbalance in geophysical observation data; according to the method, the network is trained and verified through the training sample set, and the optimal network parameters are obtained; the trained network is utilized to directly map the apparent resistivity and phase data of a test set into underground resistivity distribution, the end-to-end inversion process is realized, and the multi-scale feature extraction and semantic segmentation capabilities of the DeepLabV3 + network are utilized, so that compared with a traditional inversion method, the method has the advantages of being independent of an initial model, capable of obtaining a global optimal solution and the like.
Owner:CHINA UNIV OF GEOSCIENCES (WUHAN)

Knowledge graph knowledge expansion method

The invention discloses a knowledge graph knowledge expansion method, which comprises the following steps of: extracting a sub-graph of a target entity through a knowledge graph sub-graph extraction method based on K-hop neighborhood and link length constraint to reduce resource consumption, and then extracting neighborhood relation representation information and neighborhood relation link representation information by utilizing an attention mechanism; fusing the two kinds of relationship representation information by utilizing a relationship fusion method based on a gating mechanism, and further performing relationship prediction completion; and finally, model optimization is performed by combining comparison loss and cross entropy loss, more negative samples are increased through a subgraph structure-based negative sampling method, feedback on sparse data is enriched, the defect of the cross entropy loss in generalization performance is made up, and effective expansion of the knowledge graph is realized. According to the method, the relationship prediction accuracy of the knowledge graph is improved, the data sparsity problem of the knowledge graph is effectively relieved, the manual annotation cost is remarkably reduced, and excessive dependence on expert knowledge is reduced.
Owner:OCEAN UNIV OF CHINA

Bearing fault diagnosis method and system based on self-supervised migration of physical information

The invention relates to a bearing fault diagnosis method and system based on physical information self-supervised migration, and belongs to the technical field of bearing fault diagnosis. Vibration signals of the bearing are collected to obtain a data set, the data set is analyzed to obtain a physical label, a pseudo label and a data label, and the pseudo label is dynamically updated; obtaining data loss and physical loss according to the cross entropy of the pseudo tag and the data tag and the cross entropy of the physical tag and the data tag, and dynamically allocating weights and fusing the two to obtain a loss function; wherein the weight of the physical loss is dynamically adjusted according to the reciprocal of the variance of the data loss; inputting the target data of the to-be-detected bearing into a diagnosis model obtained by training the loss function, and outputting target features; and inputting the features into the physical model and the clustering model to obtain a target physical tag and a target pseudo tag, and aligning the target physical tag and the target pseudo tag to obtain a fault diagnosis result. According to the method, the generalization ability and the interpretability of the diagnosis model are improved, and the method does not depend on a tag.
Owner:SUZHOU UNIV

Multi-teacher model knowledge distillation method integrating semantic weight soft labels

The invention provides a multi-teacher model knowledge distillation method integrating semantic weight soft labels, which comprises the following specific steps: firstly, inputting a data set into each teacher model of a multi-teacher module, and outputting probability distribution of labels corresponding to input data by the teacher models, namely soft labels; secondly, labels output by the teacher model are converted into semantic vector representations through a pre-training language model, similarity values among the labels are calculated through the semantic vector representations of the labels, probability distribution of the soft labels is weighted through the semantic similarity values, and therefore semantic weight soft labels are obtained; finally, in the knowledge distillation process, the student model will distill knowledge from the teacher model through the two types of losses. The two types of loss comprise cross entropy loss and weighted relative entropy loss based on label semantic vector similarity. According to the method, the semantic information of the label can be fused into the probability distribution of the soft label of the multi-teacher model, and the knowledge distillation performance is improved.
Owner:ZHEJIANG UNIV OF TECH

Explanatable model robust training method based on topological regular terms

The invention relates to an interpretable model robust training method based on topological regular terms, and belongs to the field of artificial intelligence safety. The method comprises the following steps: firstly, performing semantic preserving disturbance on an original sample to obtain a disturbance sample; secondly, inputting the original sample and the disturbance sample into a target model, and respectively calculating gradient values to generate corresponding interpretation images; secondly, calculating gradient differences before and after sample disturbance based on a cosine distance and an Euclidean distance, and extracting topological features of interpretation images before and after sample disturbance by using a persistent coherence method to quantify topological differences; and finally, taking the gradient difference and the topology difference as regular terms, forming total loss by the regular terms and the cross entropy loss, and dynamically adjusting the weight of the regular terms according to the proportion of each difference in the total loss. Aiming at the problems that the anti-interference performance of an existing method is influenced by only utilizing a gradient difference feature training model and a fixed regular term weight is difficult to adapt to generalization of a multi-type disturbance reduction model, the invention proposes that the robustness of model explanation is effectively improved by utilizing topological features of an explaining image.
Owner:BEIJING INST OF TECH

A Label Noise Estimation Method Based on Manifold Regularized Transfer Matrix

A label noise estimation method based on a manifold regularization transfer matrix provided by the present invention pre-trains a first network in a second network, and after distilling a data set, inputs the obtained sub-data set into the second network to obtain the probability of the class to which the data instances in the sub-data set belong and obtain a transfer matrix related to the data instances; further calculates the cross-entropy loss of the second network according to the data instance labels, and combines an association matrix expressing the consistency of the data instances belonging to the same manifold and a penalty matrix of the data instances belonging to different manifolds to calculate the loss function of the second network; adjusts the loss function to reduce the training of the second network to obtain a trained second network, thereby completing the estimation of the class to which the data instances belong. The present invention can reduce the estimation error without affecting the approximation error of the transfer matrix, and experiments prove that the present invention can achieve excellent performance in label noise learning.
Owner:XIDIAN UNIV

Crystal oscillator circuit fault classification method based on multi-source information fusion network

The invention discloses a crystal oscillator circuit fault classification technology based on a multi-source information fusion network, and belongs to the technical field of analog circuit fault diagnosis. Firstly, a multi-source data set of different measurement points of the crystal oscillator circuit is acquired; carrying out conversion from a time domain to a frequency domain on the data set by utilizing fast Fourier transform; extracting data fault features by using a convolutional neural network, and performing a trust distribution function under each piece of source data by using a softmax classifier; fusing different trust distribution functions under the multi-source data by adopting a D-S evidence theory to obtain a final diagnosis result and diagnosis probability output, and calculating cross entropy loss; and finally, training model parameters through back propagation to obtain a final diagnosis model. According to the method, the time-frequency transformation algorithm, the deep neural network algorithm and the information fusion algorithm are combined, a multi-source information fusion network is constructed, the defects of an existing diagnosis model in crystal oscillator circuit fault classification are overcome, and the accuracy and stability of crystal oscillator circuit fault diagnosis are remarkably improved.
Owner:NORTHWESTERN POLYTECHNICAL UNIV

Information meat segmentation method and equipment based on regularization and multi-model fusion

The invention discloses an informatable meat segmentation method and device based on regularization and multi-model fusion. The method comprises the following steps: preprocessing data; credible polyp segmentation models based on PVTv2 and CNNs as encoders are constructed respectively; a loss function formed by combining cross entropy loss, KL divergence loss, Dice loss and newly defined evidence regularization is adopted for the two constructed information meat segmentation frameworks; an AdamW optimizer is adopted to train the two models at the same time; obtaining the final belief quality through a credible fusion rule; and Dice Score and mIoU are used as indexes for measuring the segmentation accuracy. According to the method, an evidence regularization item is introduced to improve the learning ability of the model on a zero evidence training sample. In addition, a credible fusion strategy is used, probability distribution and uncertainty output by multiple models are comprehensively considered, and the polyp segmentation accuracy is further improved.
Owner:NANJING UNIV OF POSTS & TELECOMM

Bearing fault diagnosis method based on causal decoupling network

ActiveCN121256610AMachine part testingBiological modelsSpurious correlationFeature extraction
The invention discloses a bearing fault diagnosis method based on a causal decoupling network, and relates to the technical field of computers. The method comprises the following steps: extracting overall characteristics of a sample bearing vibration signal; extracting cross-domain consistent causal factors of the overall features, and carrying out bearing fault classification on the causal factors; an extractor extracts non-causal factors of the overall features, performs fault classification on the non-causal factors, and performs domain classification on the non-causal factors; and determining the loss of the causal decoupling model based on the cross entropy loss of the causal branch and the non-causal branch, the cross-branch decorrelation loss, the maximum entropy loss and the branch specialization loss, and adjusting the parameters of the causal decoupling model until the loss of the causal decoupling model is minimum. The false correlation between the non-causal factors and the fault classification labels is blocked, and a trained causal decoupling model is obtained; and carrying out fault diagnosis on the to-be-classified bearing vibration signals through the trained causal decoupling model. The method can improve the diagnosis precision of the bearing fault.
Owner:HUNAN UNIV

Method and system for giving implicit thinking to large language model

The invention discloses a method for endowing a large language model with implicit thinking, and the method comprises the steps: introducing a potential token encoder which is isomorphic initialized with a basic model for a training data sample with an explicit reasoning chain of the large language model, collecting key information in the explicit reasoning chain through a special attention mask, generating a potential token, and transmitting the potential token to the basic model; limiting the potential tokens in a word list column space by using vocabulary probability superposition; discarding the potential token encoder, training a basic large language model (LLM) to autonomously generate a potential token, and adopting a combined target of KL divergence and cross entropy to stably optimize the target, so that potential reasoning collapses into an explicit answer at the reasoning tail end; and inputting a given question for the large language model LLM, generating a potential sequence in the LLM, and collapsing into a final answer to realize external hiding of intermediate reasoning content. According to the method, the efficiency, the precision, the stability and the engineering feasibility of potential reasoning are comprehensively improved.
Owner:INST OF COMPUTING TECH CHINESE ACAD OF SCI

Confrontation attack method based on reconstruction data and substitution model

The invention discloses an attack resisting method based on reconstructed data and a substitution model. The method comprises the following steps: 1) constructing a generative model: improving the performance of the generative model; 2) selecting a substitution model; 3) generating and initializing random noise and a pseudo label; 4) constructing a loss function for updating the generation model: combining the cross entropy loss function and the information entropy loss function to construct a new combined loss function; 5) constructing a distillation process loss function: combining the distillation loss function and the boundary loss function to construct a B-D combined loss function; 6) training the collaborative substitution model to generate a model; 7) performing knowledge distillation training on the substitution model by the collaborative generation model; 8) carrying out collaborative iteration updating on the generation model and the substitution model; according to the method, the quality of reconstructed data can be effectively improved, the success rate of attack resistance is better improved with low query cost, and an effective scheme is provided for the fields of attack resistance, model security evaluation and the like.
Owner:GUILIN UNIVERSITY OF TECHNOLOGY

Telecommunication fraud text classification detection method based on RoBERTa model

The invention provides a telecommunication fraud text classification detection method based on a RoBERTa model, which constructs a classification model capable of automatically learning deep semantic features of telecommunication fraud texts by adopting deep learning and natural language processing technologies and based on a deep learning framework of the RoBERTa model, and the framework of the model has the advantages that the classification model can automatically learn the deep semantic features of the telecommunication fraud texts; the method comprises three parts of multi-head attention, residual connection and full connection layer classification. In a model training process, a cross entropy loss function and an inconsistency loss function are combined so as to optimize classification accuracy and model robustness at the same time. The cross entropy loss function is helpful for improving the classification precision, and the inconsistency loss function enhances the identification capability of the model for abnormal or uncertain texts, so that the overfitting phenomenon is effectively prevented, and the adaptability and stability of the model in a complex scene are improved. By using the method, the classification accuracy and the model robustness of the telecommunication fraud text can be effectively improved.
Owner:GUILIN UNIV OF ELECTRONIC TECH +1

Face change video detection method and system based on illumination feature decoupling

The invention discloses a face-changing video detection method and system based on illumination feature decoupling. The method comprises the following steps of: framing a video and extracting an RGB (Red, Green and Blue) image of a detection area; sending the RGB image into a learnable local gravity mode extraction module, and extracting a local gravity mode image; the RGB image and the local gravity mode image are sent to a double-branch high-semantic feature extraction module, and illumination related features and illumination invariant features are extracted respectively; the illumination related features are sent to a contrast learning module based on relighting; sending the classification features into a classifier to obtain prediction probability distribution, and carrying out dichotomy supervision by using cross entropy loss; training a model and storing the model; and the model test loading model outputs a detection result of the to-be-detected video. According to the method, the illumination features are decoupled by using the domain generalization strategy, interference of domain-related illumination information is effectively suppressed, the characterization capability of the detection features is improved, and the method has a relatively good detection effect and generalization capability.
Owner:GUANGDONG POLICE COLLEGE (GUANGDONG PROVINCIAL PUBLIC SECURITY JUDICIAL MANAGEMENT CADRE COLLEGE)

Counterfeit information detection method combined with big language model knowledge distillation

The invention discloses a forged information detection method combined with big language model knowledge distillation. The method comprises the following steps: S1, preprocessing image text features; s2, predicting a large language model; s3, feature aggregation; s4, knowledge distillation; by calculating cross entropy loss of a tag predicted by a large language model and a real tag, a basis for generating the large language model is endowed with a weight, and basis features with the weight and image text features are aggregated and input into a small language model classifier, so that final classification is realized.
Owner:SHENZHEN KIM DAI INTELLIGENCE INNOVATION TECHNOLOGY CO LTD

Imbalanced sample galloping prediction method and system based on meteorological data

The invention relates to the technical field of power transmission engineering, and discloses an unbalanced sample galloping prediction method and system based on meteorological data, and the method comprises the steps: collecting the historical monitoring data of each monitoring point, and synchronously collecting the meteorological information of the corresponding monitoring point; monitoring devices and dates are taken as basic units, one piece of sample data is selected from daily data of each device, non-galloping samples in the sample data are grouped according to line numbers, and only samples corresponding to different spans are reserved in each group; according to line span characteristics, galloping high-incidence time periods and typical weather combinations, differential loss weights are given to sample data, and a weighted cross entropy loss function is constructed; and constructing a daily scale galloping prediction model, and training the model by using a weighted cross entropy loss function to realize daily scale prediction of a galloping event. By constructing a scientific sample extraction strategy, introducing a category weight mechanism and adopting a deep learning classification model, the recognition capability of the model on minority class events is improved.
Owner:STEJT GRID ELEKTRIK PAUER INZHINIRING RISERCH INSTITYUT KO LTD

Big language model black box forgetting method based on double auxiliary models

The invention relates to the technical field of large language model privacy protection, in particular to a large language model black box forgetting method based on a double-auxiliary model, which comprises the following steps of: constructing the double-auxiliary model and generating output difference characteristics, analyzing parameter criticality through a Fisher information matrix to carry out self-matching fine tuning, calculating a weighted forgetting relevance score, and carrying out self-matching fine tuning on the weighted forgetting relevance score. And a forgetting response is generated based on score matching target model output. According to the method, the forgetting demand is judged through the output difference of the double-auxiliary model, the self-matching fine-tuning optimization parameter updating strategy is utilized, the forgetting relevance is calculated in combination with the weighted cross entropy and the target model loss, the target information is accurately forgotten in the black box environment, meanwhile, the performance of the model on the non-forgetting task is maintained, and the forgetting efficiency is improved. And the practicability and efficiency of privacy protection of the large language model are improved.
Owner:BEIJING UNIV OF POSTS & TELECOMM

Robust classification method for images with noise labels based on continuous learning time-varying regularization

The invention discloses a robust classification method for images with noise labels based on continuous learning time-varying regularization, and the method comprises the steps: enabling the learning of the noise labels to be regarded as a continuous learning process which comprises a plurality of learning tasks, and enabling network parameters to be decomposed into a clean part and a noise part; the dynamic regularization technology is adopted to control the updating process of clean parameters and the updating process of noise parameters, and the method comprises the specific steps that conventional training is conducted on a data set with noise labels through cross entropy loss, the accuracy of a model in each epoch is tested through a verification data set, and a verification curve A is obtained; respectively carrying out normalization and reverse normalization processing on the verification curve to obtain an incremental curve A1 and a decline curve A2, and solving an incremental function alpha (t) and a decline function beta (t) corresponding to the curves by taking epochs as independent variables; and applying different dynamic regularization constraints to the clean parameter and the noise parameter by taking alpha (t) and beta (t) as dynamic coefficients. According to the robust classification method for the images with the noise labels, the combination of a continuous learning method and a noise label method is promoted, the setting of parameters is avoided, and the interference of the noise labels on the model is inhibited.
Owner:XINJIANG UNIVERSITY

Construction scene inter-domain difference-oriented adaptation method and system during continuous test

The invention relates to the technical field of computer vision, in particular to an adaptation method and system for continuous testing for differences between construction scene domains. According to the method, the similarity between Gram matrixes between adjacent domains is calculated, an elastic adjustment factor is set, the elastic adjustment factor is utilized, different weights are given to strong data enhancement and weak data enhancement, an elastic data enhancement strategy is provided, and an enhanced target domain image data set is input into a teacher model; updating the pseudo-tag by combining the elasticity regulation factor to obtain an elastic pseudo-tag; and inputting the target domain image data set into the student model to obtain a prediction result, constructing a global elastic symmetric cross entropy loss function based on the elastic adjustment factor, the cross entropy loss and the reverse cross entropy loss, updating student model parameters through the loss function, updating teacher model parameters, and finally obtaining a target model. According to the method, the construction scene monitoring model can adapt to complex domain changes when continuously learning test data, and the prediction result precision of the model in different environments is improved.
Owner:SUZHOU INST OF TRADE & COMMERCE +2

Streaming three-dimensional semantic occupancy prediction method based on Gaussian world model

The invention provides a streaming three-dimensional semantic occupancy prediction method based on a Gaussian world model, and the method comprises the steps: employing explicit three-dimensional Gaussian representation, and explicitly modeling three decomposition factors of scene evolution in a three-dimensional Gaussian space; through a unified evolution layer module, modeling evolution of historical Gaussian and perception of newly complemented Gaussian at the same time, interacting Gaussian representation and visual input by adopting a three-dimensional convolution operation and a deformable attention mechanism respectively, and adding time sequence characteristic attributes to model historical information of Gaussian representation; and finally, predicting scene evolution by using all optimized Gaussian representations, performing three-dimensional semantic occupancy prediction of the current scene, and constraining the model through cross entropy loss and lovasz loss of a three-dimensional semantic occupancy prediction task. Compared with a traditional time sequence fusion method, the method has the advantages that scene evolution is explicitly modeled in the three-dimensional Gaussian space, the static consistency of scene evolution and dynamic object movement can be modeled, and the efficiency and effectiveness of model three-dimensional occupancy prediction are improved.
Owner:TSINGHUA UNIVERSITY

Text classification model training method and text classification method

The invention relates to a text classification model training method and a text classification method. The training method comprises the following steps: for an original sample in a current training batch, inputting a constructed positive example data pair into a text classification model so as to obtain a feature vector pair of the positive example data pair through an encoder of the text classification model, the positive example data pair comprises data of an original sample and data of an expanded sample obtained after the original sample is expanded; determining a first cross entropy loss of the original sample, a second cross entropy loss of the expansion sample and a self-supervised contrast learning loss through the feature vector pair; and jointly determining a target loss through the first cross entropy loss, the second cross entropy loss and the self-supervised contrast learning loss, and training the text classification model through the target loss. The technical problems of poor generalization ability, model instability and the like caused by singly using cross entropy loss to train a text classification model in related technologies are solved.
Owner:湖北星纪魅族集团有限公司

Training method and device of voice large model, equipment and medium

The invention provides a large voice model training method and device, equipment and a medium, and the method comprises the steps: inputting a training voice data subset into a large voice model, and obtaining the prediction probability distribution of the training voice data subset outputted by a large language model module in the large voice model; according to the prediction probability distribution and the real probability distribution, an entropy weighted cross entropy loss function is determined, and the entropy weight of the entropy weighted cross entropy loss function is determined based on the distribution entropy of the prediction probability distribution at the current moment; and with the purpose of minimizing an entropy weighted cross entropy loss function, updating parameters of the large language model module and training a voice recognition tag of the voice data subset. In the training method of the large voice model, entropy weight is added in an entropy weighted cross entropy loss function, the problem of uncertainty of prediction probability distribution is solved to a certain extent, iteration is performed by using a self-feedback signal of distribution entropy based on prediction probability distribution, and the voice recognition effect of the large voice model is continuously improved.
Owner:NEW ORIENTAL EDUCATION & TECH GRP CO LTD