70 results about "Cross entropy" patented technology

The invention discloses a method for endowing a large language model with implicit thinking, and the method comprises the steps: introducing a potential token encoder which is isomorphic initialized with a basic model for a training data sample with an explicit reasoning chain of the large language model, collecting key information in the explicit reasoning chain through a special attention mask, generating a potential token, and transmitting the potential token to the basic model; limiting the potential tokens in a word list column space by using vocabulary probability superposition; discarding the potential token encoder, training a basic large language model (LLM) to autonomously generate a potential token, and adopting a combined target of KL divergence and cross entropy to stably optimize the target, so that potential reasoning collapses into an explicit answer at the reasoning tail end; and inputting a given question for the large language model LLM, generating a potential sequence in the LLM, and collapsing into a final answer to realize external hiding of intermediate reasoning content. According to the method, the efficiency, the precision, the stability and the engineering feasibility of potential reasoning are comprehensively improved.

The invention provides a large voice model training method and device, equipment and a medium, and the method comprises the steps: inputting a training voice data subset into a large voice model, and obtaining the prediction probability distribution of the training voice data subset outputted by a large language model module in the large voice model; according to the prediction probability distribution and the real probability distribution, an entropy weighted cross entropy loss function is determined, and the entropy weight of the entropy weighted cross entropy loss function is determined based on the distribution entropy of the prediction probability distribution at the current moment; and with the purpose of minimizing an entropy weighted cross entropy loss function, updating parameters of the large language model module and training a voice recognition tag of the voice data subset. In the training method of the large voice model, entropy weight is added in an entropy weighted cross entropy loss function, the problem of uncertainty of prediction probability distribution is solved to a certain extent, iteration is performed by using a self-feedback signal of distribution entropy based on prediction probability distribution, and the voice recognition effect of the large voice model is continuously improved.

The invention relates to the technical field of intelligent fault prediction, and provides a multi-dimensional time sequence anomaly detection method for the process industry, which aims at historical and real-time multi-source data of the process industry, performs adaptive feature extraction to obtain better features through a feature selection module fusing mRMR and a self-attention mechanism, and improves the detection accuracy. Multivariable time series data double-fusion prediction calculation is completed through a sequence prediction model, the incidence relation of data at different time points and the incidence relation of different features at the same time are mined, and after cross entropy loss and optimizer training optimization are carried out, abnormal state judgment is finally achieved in combination with a dynamic threshold value. The invention further discloses a system used for the method, the method and the system can mine mechanism information implied by equipment from historical multi-source data in the process industry, time and features are subjected to relevance fusion respectively, and therefore the method and the system can be more suitable for the unsteady state and strong coupling conditions of the process industry; and particularly, a remarkable effect is achieved in a slag grinding system.

The application discloses a kind of multi-modal large model learning method, system and storage medium based on instantaneous detection and rebalancing, method includes: S1, according to input sample construction multi-modal large model;S2, extract each single mode feature from training sample and calculate its prediction probability;S3, fusion all single mode features and obtain initial multi-modal feature and calculate its prediction probability;S4, compare the prediction probability difference of single mode and initial multi-modal, calculate rebalancing feature fusion weight;S5, based on rebalancing feature fusion weight calculation balanced multi-modal feature and calculate balanced multi-modal prediction probability;S6, calculate cross-entropy loss and update model parameter;S7, adjust the initial weight of next round feature fusion, return step S2 until all samples have participated in model training;The application can perceive and dynamically adjust the fusion weight between modes in time, to solve the mode imbalance problem in model training, to improve the accuracy and robustness of multi-modal large model.

This invention discloses an accelerated safety assessment method and system for low-altitude navigation systems based on alternating cross-entropy and importance sampling, relating to the fields of computer simulation and safety assessment technology. The method includes: constructing a probabilistic risk assessment model and initializing a proposal distribution; entering an alternating iterative loop: first, performing importance sampling assessment, calculating accident rate estimates and statistical errors; if the error does not meet accuracy requirements, initiating cross-entropy optimization, updating the proposal distribution parameters using simulation data; the new distribution is fed back to the next round of assessment. This loop continues until the results converge, and an assessment report is output. This invention, through a closed-loop feedback mechanism of alternating assessment and optimization, enables the sampling distribution to adaptively approximate the optimal distribution, intelligently focusing simulation resources on high-risk scenarios, solving the computational efficiency bottleneck of assessing extremely low probability events, achieving an order-of-magnitude improvement in assessment efficiency, and possessing high adaptability and engineering practical value.

The invention relates to a collagen extraction equipment state monitoring method and device, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring monitoring data of collagen extraction equipment; obtaining static parameters of collagen extraction equipment; the multi-sensor time sequence monitoring data are recombined into a three-dimensional tensor by calculating the adaptive slice length and the dynamic slice number; constructing an equipment state monitoring model; adopting cross entropy loss as a main loss item to calculate the difference between the equipment state probability distribution predicted by the model and a real labeling label; training the model by adopting a small-batch gradient descent strategy to obtain a trained model; newly collected time sequence monitoring data are processed to generate a recombined three-dimensional tensor, the recombined three-dimensional tensor and equipment static parameters are input into the trained model, equipment state probability distribution is output, the equipment state is judged, and early warning is carried out. According to the invention, the classification precision and robustness of the model can be improved.

The application discloses an energy model training method, a data security detection method and system. The energy model training method comprises the following steps: obtaining training sample data, sample labels and historical probability output results corresponding to historical data; inputting the training sample data into an energy model to be trained to obtain first probability output results corresponding to the training sample data; determining a target loss function based on the first probability output results, the sample labels and the historical probability output results, wherein the target loss function at least comprises a cross-entropy loss function and a contrast loss function; and adjusting model parameters of the energy model by using the target loss function to obtain a target energy model. The application solves the technical problem that the prediction result accuracy of a pre-trained natural language model used for data security detection is not high.

The application provides a few-shot bearing fault migration diagnosis method based on multi-working condition supervised contrast learning, which comprises the following steps: taking a plurality of vibration signals of bearings under different working conditions and having a large number of labels as source domain data and taking vibration signals under a working condition different from that of the source domain data and having only a small number of labels as target domain data; obtaining feature representation and prediction probability distribution of each fault category according to a pre-constructed contrast learning training classification model and the source domain data and the target domain data; then, respectively calculating a supervised contrast loss and a cross-entropy loss, and combining them into a total loss function by weighting; and finally, obtaining a trained contrast learning training classification model by updating model parameters through an adaptive learning rate optimization algorithm and back propagation according to the combined total loss function. The application extracts features by using a plurality of different source domain data through an improved supervised contrast loss function, and optimizes model parameters in combination with a cross-entropy loss function, so as to diagnose the target domain with scarce data.

The application belongs to the technical field of underwater unmanned vehicle route planning, and particularly relates to a UUV route planning method based on a cross-entropy method, which comprises the following steps: step 1), each feasible and optimized path of a UUV is divided into m connected line segments to form a parameter set; step 2), each feasible and optimized path of the UUV corresponds to a parameter set, the path cost corresponding to each feasible path is calculated, and each feasible path is sorted from large to small, 10% to 20% of the superior set is selected as an optimized set; step 3), the mean and variance of each group of parameters in the optimized set are calculated as the mean and variance of the next iteration sampling, and the updated mean and variance of the m groups of parameters are obtained to obtain the optimal solution of this time; and step 4), for the optimal solution of this time, N feasible paths are obtained again, and steps 1) to 3) are repeated, and through continuous iteration, an optimized feasible path of the UUV is obtained, and the optimized feasible path is taken as the final optimal solution.

Implementations relate to ascertaining to what extent predictions, generated using a machine learning model, can be effectively reconstructed from model updates, where the model updates are generated based on those predictions and based on applying a particular loss technique (e.g., a particular cross-entropy loss technique). Some implementations disclosed generate measures that each indicate a degree of conformity between a corresponding reconstruction, generated using a corresponding model update, and a corresponding prediction. In some of those implementations, the measures are utilized in determining whether to utilize the particular loss technique (utilized in generating the model updates) in federated learning of the machine learning model and / or of additional machine learning model(s).

The invention relates to the technical field of construction risk management of constructional engineering, in particular to a subway station deep foundation pit construction risk assessment method based on bilateral probability language, which comprises the following steps: step 1, constructing an evaluation set in a bilateral probability language term set form; 2, performing objective standardization processing on evaluation information; 3, determining a combination weight of subjective and objective combination; step 4, hierarchical information aggregation based on a weighted average operator; and step 5, risk quantification and grade determination. According to the method, a bilateral probability language term set (DPLTS) is introduced to completely describe expert group opinion distribution, and an LCM objective standardization method and a fuzzy entropy-cross entropy-BWM combined weighting model are combined to construct a risk assessment system which is complete in information, scientific in weight and robust in decision making; the defects of the prior art in the aspects of expressing complex uncertainty and fusing subjective and objective information are effectively overcome, and the reliability, the distinction degree and the practical value of a risk assessment result are remarkably improved.

The invention discloses a double-head feature separation federated learning method and system, and belongs to the technical field of federated learning and machine learning. The model uses a feature extractor to extract basic features, a feature separation module performs context enhancement on the basic features, a selection module and a gating module separate common and personalized features, a personalized head and a shared head respectively process the two features, and finally the prediction results of the two features are fused. Meanwhile, a composite loss function containing cross entropy, MMD and entropy minimization loss is adopted to guide updating; the client uploads related parameters after training, the server carries out aggregation distribution according to the weight, and a personalized model is obtained through iteration. According to the method, global and personalized information can be fully utilized, model drift is relieved, and model performance and generalization ability are improved.

The invention provides a substitution model training method and device and related equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: inputting first image data into a substitution model of a target model for countermeasure attack to generate a first countermeasure sample of the first image data, the target model being used for image processing; the first adversarial sample is processed through M teacher models of the target model, M cross entropy losses between output of the M teacher models and a first label are calculated, the M teacher models at least comprise two different types of teacher models, the first label indicates identification information of first image data, and the M cross entropy losses are calculated according to the identification information of the first image data; m is an integer greater than 1; weighting processing is carried out based on the M cross entropy losses, and target cross entropy losses are obtained; and on the basis of the target cross entropy loss, parameters of the substitution model are updated, and the substitution model is used for generating image data attacking the target model.

The application relates to the technical field of image classification, in particular to a hyperspectral open set classification method combining feature reconstruction and prototype constraint, which comprises the following steps: carrying out block division and label annotation on each original sample data in an original sample data set; inputting sample data blocks into a classification model composed of an encoder, a decoder, a prototype classification module and an unknown class detection module; constructing contrast prototype loss, reconstruction error loss and cross-entropy loss by using the encoder, the decoder and the prototype classification module; constructing a total loss function based on the contrast prototype loss, the reconstruction error loss and the cross-entropy loss, so that the classification model is iteratively trained until convergence, and a trained classification model is obtained; performing open set identification based on sample feature distribution and reconstruction error loss by using the unknown class detection module, combining prototype distance and reconstruction loss to evaluate the possibility score of the unknown class, so as to determine the unknown class. The method realizes high-precision hyperspectral open set classification.

The embodiment of the invention provides a classification model training method based on hierarchical knowledge migration. The method comprises the steps of obtaining a universal baseline model with a semantic understanding capability; according to the hierarchical classification structure of the target domain, an industry domain model containing a plurality of sub-classification output heads is constructed, and initialization is carried out by using a general baseline model parameter; constructing a hierarchical constraint loss function, and constraining the parent class prediction probability to be greater than or equal to the child class prediction probability; obtaining domain classification training samples, and respectively inputting the domain classification training samples into the general baseline model and the industry domain model to obtain first and second prediction category probability distributions; calculating domain cross entropy loss based on the second prediction distribution and the real label, calculating knowledge distillation loss based on the difference between the two distributions, and constructing a target loss function in combination with hierarchical constraint loss; and taking minimization of the target loss function as a target training industry domain model. According to the method, the model training efficiency is effectively improved, efficient migration of domain knowledge is realized, and the hierarchical classification accuracy is remarkably improved.

The invention relates to the technical field of power system reliability evaluation, in particular to a power grid reliability evaluation cross entropy important sampling method based on functional optimization. According to the method, system random variables are mapped to a standard normal space, conditional distribution is constructed, sampling estimation is carried out by using a sample re-sampling and conditional M-H sampling method, functional optimization is carried out on important sampling functions step by step by taking the conditional distribution as a target, the conditional distribution gradually approaches a theoretical optimal IS-PDF along with the increase of the number of iterations, and the optimal IS-PDF is obtained. And an important sampling function optimized by taking conditional distribution as a target gradually approaches the theoretical optimal IS-PDF, so that approximate estimation of the optimal zero-variance IS-PDF is realized. The problem that in a traditional cross entropy method, iterative optimization of IS-PDF parameters can only use fault samples, strict theoretical rules are lacked to serve as the basis for IS-PDF type selection, efficiency and performance are affected to different degrees, and serious problems occur in some occasions is solved.

The embodiment of the application provides a target recognition model training method, a target recognition method and related equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: extracting a feature vector of a sonar image sequence through a backbone network, performing manifold mapping on the feature vector to obtain normalized features; performing frequency statistics on the normalized features to obtain a category distribution; calculating a dynamic interval component of each category based on the category distribution; determining a cross-entropy loss, a dynamic interval loss and an uncertainty loss based on the dynamic interval component, and determining a total loss function based on the cross-entropy loss, the dynamic interval loss and the uncertainty loss; and performing model training based on the total loss function through a sonar image training set to obtain a trained target recognition model. The target recognition model obtained by the embodiment of the application can effectively distinguish target categories, improve the long-tail target distinguishing degree, reduce the false alarm rate of unknown obstacles, and improve the survivability in a complex environment.

This invention discloses a crystal oscillator circuit fault classification technique based on a multi-source information fusion network, belonging to the field of analog circuit fault diagnosis technology. First, multi-source datasets from different measurement points of the crystal oscillator circuit are acquired. Then, a fast Fourier transform is used to convert the datasets from the time domain to the frequency domain. Next, a convolutional neural network is used to extract fault features from the data, and a softmax classifier is used to assign trust functions to the data from each source. Then, the D-S evidence theory is employed to fuse the different trust assignment functions under the multi-source data, obtaining the final diagnostic result and diagnostic probability output, and calculating the cross-entropy loss. Finally, the model parameters are trained through backpropagation to obtain the final diagnostic model. This invention combines time-frequency transformation algorithms, deep neural network algorithms, and information fusion algorithms to construct a multi-source information fusion network, overcoming the shortcomings of existing diagnostic models in crystal oscillator circuit fault classification and significantly improving the accuracy and stability of crystal oscillator circuit fault diagnosis.

A SAR image road segmentation method based on semi-supervised prototype representation learning and pseudo label balancing includes the following steps: step 1, dividing the SAR road data set into labeled data and unlabeled data; step 2, training the labeled data to calculate the cross-entropy loss and output the segmentation result; the segmentation network is optimized in the back propagation; step 3, obtaining feature prediction and feature mapping; step 4, dynamically adjusting the feature prediction based on the class frequency in the pseudo label; step 5, calculating the cross-entropy loss of the adjusted feature prediction; step 6, calculating the prototype loss of the unlabeled data; step 7, calculating the cross-supervision loss; step 8, performing prototype consistency regularization on the feature mapping and the unlabeled prototype; step 9, updating the prototype by exponential moving average; step 10, segmenting and predicting the road image. The present application utilizes labeled data and unlabeled data to improve the segmentation accuracy in the case of fewer labeled samples.

The invention provides a next interest point recommendation method for neighbor aggregation flashback of a space-time diagram, which comprises the following steps of: 1, dividing a user historical sign-in sequence into a plurality of subsequences with the same length according to the number of hidden states of an aggregation layer, 2, mapping a user and a POI (Point of Interest) from a high-dimensional space to a low-dimensional space, learning dense vector representations of the user and the POI, and a foundation is laid for later layer information processing. The method comprises the following steps of: (1) carrying out clustering on information between users and between places, (2) aggregating information between the users and between the places, and enriching the representation of the places, (4) re-weighting a spatial distance and a time distance of a historical state by an RNN with a flash-back mechanism, and utilizing abundant spatio-temporal context information, and (5) inputting the output of an aggregation layer and user embedding, and outputting a predicted score. And carrying out global optimization by using a cross entropy function. According to the method, the user access records and the POI geographic information are utilized to directly model the space time information, the space time information is directly integrated into the graph neural network, only the calculation cost of pre-calculation is increased, and the model performance is improved on the premise that the model is kept relatively simple.

The invention discloses a training method of a knock signal defect recognition model, relates to the technical field of intelligent nondestructive testing and signal mode recognition, and can at least partially solve the problem of weak minority class recognition capability caused by strong subjectivity of manual interpretation, insufficient robustness of manual features and unbalanced class samples in the prior art. The method comprises the steps that knocking signal samples are acquired and subjected to category labeling, and a sample set is constructed; event detection, peak value alignment, fixed-length segmentation and preprocessing are carried out on the knocking signal sample; dividing a training verification test set and constructing small-batch training data streams; inputting the preprocessed sample into a one-dimensional deep convolutional neural network for multi-layer feature extraction to obtain a feature vector; inputting the feature vectors into a classification head and outputting probability distribution through a Softmax classifier; a cross entropy loss function is combined with an optimization algorithm for training until convergence; and outputting the deployable model. According to the invention, end-to-end automatic feature learning and multi-class defect discrimination are realized.

The application discloses a kind of training methods of knock signal defect identification model, it is related to intelligent nondestructive testing and signal mode identification field, it can at least partially solve the problems that the subjectivity of artificial interpretation is strong, manual feature robustness is insufficient in prior art, class sample imbalance leads to the weak recognition ability of minority class.The present application comprises: obtaining knock signal sample and carrying out class labeling, construct sample set;Knock signal sample is detected, peak alignment, fixed-length segmentation and pre-processing are carried out;Divide training verification test set and construct small batch training data stream;The pre-processed sample is input into one-dimensional deep convolutional neural network to extract multi-layer features, and a feature vector is obtained;The feature vector is input into the classification head and the probability distribution is output by the Softmax classifier;Cross-entropy loss function is used in combination with optimization algorithm for training until convergence;Output deployable model.The present application realizes end-to-end automatic feature learning and multi-class defect discrimination.

The invention discloses an engine time sequence monitoring data expansion method, and relates to the field of artificial intelligence and machine learning. The method comprises the following steps: constructing an overall monotonic function; establishing an average cross entropy used for calculating the similarity between the sample real data and the expansion generation data; the number of samples is smaller than a preset threshold; determining the overall monotonicity of the health state time sequence sample by using an overall monotonic function according to the preprocessed health state time sequence sample; selecting the dimension data of which the overall monotonicity is greater than or equal to a preset monotonicity threshold as a to-be-expanded sample; according to the to-be-expanded sample and the overall monotonic function, taking the average cross entropy as a loss function, training the generative model, and obtaining a trained generative model; and inputting the health state time sequence data, obtained in real time, of the engine into the trained generative model, and outputting expanded generation data of the health state of the engine. According to the invention, high-quality and diversified time sequence expansion data can be generated.

This invention discloses a method, apparatus, equipment, and medium for constructing a three-dimensional geological model integrating multivariate data, applicable to the field of geological exploration. The method includes acquiring two-dimensional profile detection data of a target geological area and expanding it into three-dimensional training data; extracting spatial geological constraints; constructing a multivariate clustering pattern library based on the three-dimensional training data, velocity model, and density model; constructing a candidate pattern library based on a pre-defined simulation path and spatial geological constraints; calculating a probability matrix based on the candidate pattern library; selecting patterns based on the probability matrix and cross-entropy to obtain an initial three-dimensional geological model; and performing multi-scale iterative optimization on the initial three-dimensional geological model to obtain an optimized three-dimensional geological model. This invention effectively improves the accuracy and reliability of three-dimensional geological models, better handles data sparsity and geological complexity issues, and reduces the uncertainty of simulation results.

The invention belongs to the technical field of image recognition, and particularly relates to a luggage case re-recognition method and system based on deep feature learning, and the method comprises the steps: inputting a to-be-re-recognized luggage case image into an initial re-recognition model, and obtaining the probability distribution of the to-be-re-recognized luggage case image belonging to each identity category; constructing a joint loss function according to the cross entropy classification loss based on the probability distribution and the loss based on the feature vector distance; based on the joint loss function, training the initial re-recognition model through a luggage case image data set containing an identity tag to obtain a target re-recognition model; and re-identifying a to-be-re-identified image based on the target re-identification model. According to the invention, high-precision and high-robustness re-identification of the luggage case in a complex scene is realized.

The invention discloses a text sensitive information recognition and processing method and system based on deep learning, and the method comprises the steps: building a sensitive information recognition model containing a feature extraction layer and a classification layer based on a pre-training language model, and endowing a minority class with a higher weight through a weighted cross entropy loss function to relieve class imbalance; a semantic perception adversarial sample generation technology is adopted, synonyms are screened in combination with BERT semantic similarity to replace samples, and model robustness is enhanced; the dynamic threshold adjustment mechanism optimizes the classification threshold in real time according to the misjudgment rate and the omission ratio, and the scene adaptability is improved; high-priority sensitive content is blocked in real time through the grading processing strategy, low-priority content is marked to be audited, and a closed-loop continuous optimization model is fed back through a log. The method is suitable for the fields of Internet content auditing, financial risk control, public opinion monitoring and the like, the sensitive information identification accuracy is remarkably improved, the false report and missing report rate is reduced, and self-adaptive iteration of model performance is achieved.

The embodiment of the invention provides a cross-language few-sample classification method based on instance collaboration. The method is applied to the technical field of natural language processing. Generating a pseudo tag for each sample in the target language data set through a pre-trained source language model and an initialized target language model; determining the reliability weight of each sample pseudo tag in the target language unlabeled data set, and constructing a weighted cross entropy loss function according to the reliability weight; iteratively updating the parameters of the target language model according to the weighted cross entropy loss, and stopping iteration until the performance index of the target language model on the target language verification set meets a preset condition or reaches the maximum number of iterations to obtain an optimized target language model; and inputting the to-be-classified text into the optimized target language model for analysis processing to obtain a classification result. According to the method, the generalization ability and the classification accuracy of low-resource languages are improved.

The invention relates to a model training method and system based on a cross entropy triple loss function. The method comprises the following steps: constructing a feature extraction network and a classification prediction network; fusing the feature extraction network and the classification prediction network to obtain a convolutional neural network classification model; constructing a cross entropy loss function and a triple loss function; constructing a cross entropy triple loss function based on the cross entropy loss function and the triple loss function; and updating model parameters of the convolutional neural network classification model based on the cross entropy triple loss function, so as to solve the problem that the model learns the characteristic of'effective classification but insufficient discrimination 'due to the fact that the convolutional neural network model is updated through the cross entropy loss function in the prior art. And the generalization ability and the classification reliability of the model in complex scenes (such as subdivided categories, sample noise and few samples) are influenced.

The invention discloses a crack quantitative classification method based on deep learning and a multi-view strategy, and belongs to the technical field of composite material damage detection and quantitative analysis. The method comprises the following steps: 1, adding a convolution block attention module to improve a U-Net model; 2, further improving the model by adopting a balanced cross entropy loss function; 3, outputting and slicing the CT data along YZ, XY and XZ planes; 4, respectively selecting a plurality of representative slices to make three groups of training sets; step 5, respectively training the three improved U-Net models; 6, inferring complete crack information along the YZ plane, the XY plane and the XZ plane; and 7, projecting crack classification results with high accuracy in the YZ plane and the XZ plane to the XY plane, and merging the classification results with the cracks with high accuracy in the corresponding results of the XY plane slice by slice to obtain the optimal classification result of the accuracy of each crack type. According to the method, the accuracy of crack detection can be effectively improved, and the method can be used for quantitatively analyzing the crack evolution behavior of the 3DWC under the out-of-plane shear load.

The invention discloses a loss function determination method, a defect identification method and related equipment, and relates to the field of industrial defect detection.The method comprises the steps that a transfer cost matrix is constructed according to the attention of a target user on different defect categories; obtaining a cross entropy loss function; and determining a target loss function based on the transfer cost matrix and the cross entropy loss function.