18 results about "Training performance" patented technology

This invention belongs to the field of artificial intelligence hardware architecture design and discloses a method, system, device, and medium for cross-layer optimization of multi-core packaging and optical interconnect clusters for large language model training. The method includes: constructing a design space, including a core hardware architecture layer, an optical interconnect network layer, and a training parallel strategy layer; performing an outer-layer search to search for the architecture parameters of the multi-core modules within the core hardware architecture layer; for each outer-layer search sampling point, performing an inner-layer search to collaboratively optimize the optical interconnect network topology and training parallel strategy within the optical interconnect network layer and the training parallel strategy layer; and outputting the Pareto optimal design point on the training performance and cluster cost plane. The technical solution described in this invention can guide the design of related training clusters, thereby fully leveraging the advantages and potential of core technology and optical interconnect technology in large language model training.

The present disclosure relates to communication methods, communication devices, a communication system, a storage medium and a program product, and belongs to the technical field of communications. A method comprises: a first device receives second data corresponding to first data transmitted by a second device, wherein the first data and the second data form training data pairs, and the training data pairs are used for training a first model. In the method provided in the present disclosure, while transmitted data of a transmitting end is unknown, the transmitted data of the transmitting end and received data of a receiving end are combined as training data pairs on the basis of the received data, so as to train a model, thereby improving the model training performance and accuracy.

This invention discloses a load-balanced automatic parallel modeling method, device, and storage medium. First, it analyzes the key factors affecting the execution performance of operators and models (operator in-degree, tensor shape, and tensor data type), and proposes a method for constructing a performance evaluation model based on operator characteristics to assess the computational, communication, and overall costs of operators, as well as the training performance costs of the model. Then, with the goal of load balancing across devices, a layer-by-layer partitioning method based on topological sorting is used to quickly divide the neural network model into multiple sub-models with balanced overall costs, achieving coarse-grained partitioning. Finally, based on the model's training performance evaluation model, and according to the communication characteristics between operators, a fine-grained model partitioning and scheduling method based on communication optimization is used to fine-grainedly adjust the coarse-grained sub-models, reducing the amount of cross-device communication tensor transmission to achieve optimal global model scheduling.

Provided is a method for training a quantized CNN by applying trend-based progressive scaling optimized for hardware accelerators. The method for training a quantized CNN according to an embodiment of the present invention determines a next suitable scale by identifying a previous trend, and trains the quantized CNN by applying a scale factor considering clipping loss sensitivity and progressively adjusting the scale factor on the basis of previous overflow / underflow. Accordingly, training based on hardware accelerators can be performed quickly and with minimal performance degradation, and training performance can be improved by facilitating higher precision utilization than before.

The application discloses a kind of man-machine collaborative multi-arm teleoperation control system, method and robot based on multimodal large language model, it is related to robot teleoperation technical field, the system is equipped with four big modules of visual perception, LLM driven task planning, motion primitive execution, safety supervision of human-in-the-loop, decouples high-dimensional control space with "main pilot-co-pilot" paradigm, human remote control main arm makes fine operation, intelligent agent independently controls auxiliary arm to execute coarse-grained task, voice intervention instruction has absolute priority.This application breaks through the bottleneck of cognition and operation of single-person control multi-arm, realizes single operator to complete four-arm strongly coupled collaborative task, eliminates the communication delay of multi-person cooperation, greatly reduces demonstration trajectory variance, improves the quality of teaching data, optimizes downstream model training performance, while significantly reducing the operation threshold, realizes high-level physical safety through low-delay voice supervision, can be widely applied to industrial assembly, logistics handling and other scenes, and has strong industrial practicability.

The invention belongs to the technical field of large-scale deep learning model training, and particularly relates to a video memory scheduling system based on dynamic sequence length assembly line parallel training. The system comprises a dynamic recalculation module, a video memory arrangement and prefetching module and a mapping reconstruction module, the system collects the length and load information of each micro-batch before each round of iteration, establishes an optimization model with the goal of minimizing the iteration time, dynamically generates a re-calculation plan, and realizes inter-stage video memory load balancing; by introducing a collating stream and an asynchronous scheduler outside a main computing stream, overlapping execution of video memory collating and computing prefetching is realized, and the influence of fragment collating on training performance is reduced; a dynamic physical block mapping strategy is designed based on a CUDA virtual memory management mechanism, the physical block granularity is adaptively adjusted according to the idle state of the video memory, and the API calling and context switching overhead is reduced. Experimental results show that the video memory utilization rate and the system stability are remarkably improved, and an efficient video memory scheduling scheme is provided for large model training.

This application relates to a federated learning method, apparatus, device, storage medium, and program product. The method includes: training a first artificial intelligence model locally on a client using a training dataset to obtain the classifier gradient of the first artificial intelligence model; the sample data for each category in the training dataset is imbalanced, and the sample data for the first category does not meet the data balance condition; determining the gradient adjustment value corresponding to the first category based on the global gradient issued by the federated learning center; adjusting the classifier gradient according to the gradient adjustment value to obtain the adjusted classifier gradient; updating the first artificial intelligence model based on the adjusted classifier gradient; and upon reaching a convergence condition, sending the classifier gradient obtained in each iteration to the federated learning center so that the federated learning center can update a second artificial intelligence model based on the received classifier gradient, resulting in a model for processing recommendation tasks. This method can improve model training performance.

This invention provides a data processing method, apparatus, device, computer program product, and computer-readable storage medium. The method considers that the preference data used in DPO training may have noisy labels that negatively impact training performance. It proposes a noise-aware metric to indicate the probability that the preference label of a pair of preference data is a noisy label, and incorporates this metric into the DPO training objective function. This incorporates the quality of the preference data into the DPO training process, thereby identifying and mitigating the impact of noisy preference data on DPO training performance. This method effectively identifies noisy data in the preference data used in DPO training and effectively reduces the impact of noisy data on training performance by introducing a noise-aware metric into the DPO objective function, thus improving DPO training performance. Furthermore, this method is widely applicable to various large models, including LLM and DM, to improve the DPO training performance of these large models.

A model training method and device, a cluster and related equipment, relate to the technical field of artificial intelligence. In the case that part of the computing nodes in the cluster fail, a plurality of candidate deployment strategies are determined for a plurality of computing nodes in the cluster that have not failed, each candidate deployment strategy being used to indicate a manner in which an AI model is deployed on the plurality of computing nodes based on a plurality of parallel manners; the model training performance corresponding to each candidate deployment strategy is evaluated; and the AI model is continued to be trained on the plurality of computing nodes according to a target deployment strategy, the target deployment strategy being a candidate deployment strategy in the plurality of candidate deployment strategies that is used to cause the model training performance that can be generated by the training of the AI model to satisfy a performance condition. In this way, by determining the target deployment strategy and continuing to train the AI model based on the target deployment strategy, the training efficiency of the AI model can reach a high level after the training for the AI model is resumed, so that the performance of the cluster in training the AI model can be improved.

The application provides a neural network model training method and device. The neural network model training method comprises the following steps: each neural network layer in part or all of the neural network layers is divided into S parts, S accelerators included in a training system train a part of the corresponding neural network layer respectively, and each accelerator only saves a part of the training output result of each neural network layer locally. The application effectively optimizes the video memory occupied by the training output result by distributing the training output result, improves the computing power of the neural network model, and further improves the cluster training performance.

This invention discloses a method and apparatus for wireless communication. A first node receives a first signaling, wherein the first signaling indicates that training associated with a first dataset is divided into M subtasks and a first subtask among the M subtasks; the first subtask is executed; wherein M is a positive integer greater than 1. This application can improve training performance or reduce the training overhead of each node.

The application discloses a kind of digital multi-modal muscle isometric training detection method and system.The detection method includes: collecting the skeletal key point data and muscle tissue hardness data when patient executes muscle isometric training;Judge whether the training posture of patient and effort is up to standard;Record the training data of patient in up to standard state;Training data is transmitted to cloud platform, based on pre-trained GAMLSS model, corresponding percentile standard curve of the attribute of patient is matched, to convert the muscle tissue hardness data of patient in up to standard state into percentile score;Based on the training performance of patient's percentile score evaluation, and provide training feedback.Using the present application can automatically monitor patient muscle isometric rehabilitation training and provide training feedback in home scene, save the time cost of patient.And, the present application can adopt different detection standards for different users, to adapt to the training situation of different users.

A device may maintain, by a governance module of an intelligent dosing platform, a governance library comprising an operation rule and a criterion regarding appropriate use of an intelligent injection device. A device may create, by a digital twin module of the intelligent dosing platform, a digital twin embodiment. A device may generate, by an intelligence analytics module of the intelligent dosing platform based on the governance library operation rule and the criterion and the digital twin embodiment, a personnel training protocol tailored to a specific personnel role. A device may deliver, through a training system of the intelligent dosing platform, an interactive training experience. A device may monitor, by the intelligence analytics module, personnel training performance. A device may calculate, by an artificial intelligence module of the intelligent dosing platform, a training competency score based on conformance of trainee performance to the governance library operation rule and the criterion.

The application discloses a deep learning training system based on a GPU cluster and a related method. The system comprises a host and a GPU cluster composed of multiple GPU nodes; the multiple GPU nodes form a topological ring. The host is used for distributing multiple training batches of a same pre-trained deep learning model to multiple corresponding GPU nodes respectively, broadcasting initial values of parameters of the model to each GPU node, saving parameter values of the model, receiving the single training batch and the initial values of the parameters of the model distributed by the host, executing a training process of the model, realizing gradient data sharing on the multiple GPU nodes through multiple loop transmissions, determining updated parameters of the model, and sending the parameter values of the model to the host. The training system disclosed by the application improves the training performance and the training efficiency of the model.

The invention belongs to the technical field of information security, and particularly relates to a block chain federal learning method and system for resisting poisoning and convenient vehicle taking attacks. According to the method, decentralization and tamper-resistant characteristics of a block chain are fused, node registration, committee and aggregation node election and reputation value dynamic management are realized by means of an intelligent contract, and low heterogeneous nodes are screened based on a data set heterogeneous degree to complete cross validation so as to block poisoning attacks. A gradient and reputation value-based two-dimensional feature space is constructed in a credible execution environment, credible gradients are screened through an improved DBSCAN clustering algorithm so as to resist a convenient vehicle attack, an aggregation weight is calculated in combination with node training accuracy and a historical reputation value so as to complete weighted aggregation, and finally, on the premise that the model training performance is not reduced, the reliability of the model is improved. The single-point fault risk is eliminated, the verification misjudgment rate is reduced, and the safety, stability and expandability of the federal learning system are improved.

Embodiments evaluate performance by receiving a plurality of training performance reviews. Embodiments extract from the training performance reviews, using a first machine learning model, a plurality of features comprising a training aspect, a training sentiment, and a corresponding training evidence. Embodiments use the extracted plurality of features to train a second machine learning model. Embodiments receive a first performance review and extract from the first performance review one or more first aspects, one or more corresponding first evidences, and one or more corresponding first sentiments. Embodiments, using the trained second machine learning model, predict first sentiment scores for each of the first aspects.

This application provides a communication method and a communication device, applicable to the field of communication technology. The method includes: a second communication device sending model synchronization information and first indication information to at least one first communication device; the model synchronization information instructing the synchronization of parameters of a first model, and the first indication information instructing adjustments to the discarding mechanism and / or batch normalization mechanism of the first model; and at least one first communication device training the first model based on the model synchronization information and the first indication information. Implementing this application embodiment can help improve the training performance of the model.

This invention provides a method, apparatus, device, and storage medium for managing multi-source heterogeneous data in security operations and maintenance. It determines the collection priority of target data sources by using the information entropy gain index of semi-structured log data, achieving adaptive data collection with dynamic scheduling based on information value, thus solving the problem of low collection efficiency. By calculating the heterogeneous confusion entropy of the original data and constructing an adaptive temperature adjustment model, it performs variable-temperature semantic mapping based on dynamically calculated mapping temperature parameters according to the confusion entropy. This ensures that high-confusion-entropy fields corresponding to low temperatures undergo rigid feature preservation, while low-confusion-entropy fields corresponding to high temperatures undergo flexible normalization, solving the problem of mismapping of high-confusion fields caused by fixed mapping strategies. By calculating the causal contribution of candidate training samples to the target security model and selecting high-value samples for inclusion in the database, it solves the problem of low-value samples affecting model training performance.