194 results about "Gradient estimation" patented technology

Systems and techniques are described herein for training a machine learning (ML) model. For instance, a process can include obtaining, from a teacher ML model, a first prediction based on an input. The process can further include obtaining, from a student ML model, a second prediction based on the input. The process can include determining a loss based on a difference between the second prediction from the first prediction. For instance, the loss can include a variance reduced total variation distance (TVD) loss based on an unbiased gradient estimate of the loss. The process can further include backpropagating the loss through the student ML model to train the student ML model.

The invention belongs to the technical field of image processing, and particularly relates to a road segmentation and gradient estimation method based on multi-sensor fusion. Comprising the following steps: S1, registering a camera and a laser radar; s2, generating a road segmentation result based on output data of the camera and the laser radar after registration; and S3, obtaining a three-dimensional point cloud image of the road surface based on the road segmentation result, and realizing slope estimation of the road surface based on the three-dimensional point cloud image. According to the method, the error rate of road surface segmentation and the error of slope estimation are effectively reduced through complementation of multiple sensors, and stable output is still kept under complex backgrounds such as vibration, rain and fog and weak light.

The invention discloses an automatic driving test scene set optimization method, apparatus and device, and a storage medium. The method comprises the steps of generating a simulation scene file in an OpenSCENARIO format through preprocessing data; analyzing the simulation scene file through a teacher model, outputting a risk description text, receiving a scene feature vector and the risk description text through a student model, and outputting a failure probability; determining a comprehensive value index according to the failure probability, dynamically updating a scene library of automatic driving test scenes, collecting failure data in an AUT test, performing incremental fine tuning on the student model, and obtaining an optimized target test scene set; the problem of gradient estimation variance explosion caused by sparseness disasters can be effectively solved, the stability of model training is improved, the recognition capability of a rare long-tail failure scene is enhanced, the accuracy of failure probability prediction is improved, the judgment accuracy of the model to a boundary scene is improved, and the accuracy and comprehensiveness of scene value quantification are improved.

The invention relates to the technical field of road repair state monitoring, in particular to a road repair state monitoring method and system based on edge calculation. The method comprises the following steps: acquiring planning information of a repaired road, deploying 3D laser scanning vibration meters at a plurality of repairing points, collecting pavement vibration signals in different time periods, and performing disordered distribution time domain feature analysis to obtain a vibration signal disordered distribution time domain graph; then, structure weakening behavior simulation is carried out based on the graph, periodic structure weakening data is quantified, and a pavement water damage cumulative gradient is obtained through water damage cumulative gradient estimation; and finally, in combination with the periodic structure weakening data and the water damage cumulative gradient, carrying out service life evaluation on the restoration state, and sending an evaluation result to the terminal. According to the invention, the road repair state monitoring technology is optimized, so that the road repair state monitoring technology is more perfect.

The invention discloses a fine tuning system of a large-scale pre-training model in a federated learning environment and application thereof. The system comprises a local disturbance gradient estimation module, a differential privacy protection module and a global model aggregation and update module. The local disturbance gradient estimation module is used for calculating a global model loss value by combining forward propagation with a zero-order optimization method so as to estimate a gradient and realize all-parameter fine tuning; the differential privacy protection module performs differential privacy protection processing on the estimated disturbance gradient to prevent gradient information from leaking user sensitive data; and the global model aggregation and update module reconstructs a disturbance vector and completes global model update based on a random seed and a scalar gradient uploaded by a client. Compared with the prior art, on the premise of not depending on back propagation, all-parameter fine tuning of a large-scale pre-training model is achieved, data privacy is guaranteed, meanwhile, calculation and memory expenses are remarkably reduced, and the method is suitable for a resource-limited distributed calculation environment.

Quantum-classical gradient estimation is described for use in determining a value of at least one optimizable parameter of an objective function. An optimization method can be performed by a digital processor coupled to an analog processor and can include, until the objective function converges: obtaining a set of samples generated by the analog processor; estimating a gradient of the objective function based on a current value of the at least one optimizable parameter and the set of samples; determining first and second order moments based on the estimated gradient, and, updating the optimizable parameters based on the moments. A method to train a machine learning data can include: receiving a training data set; generating a training model having an objective function; performing acts of the optimization method to optimize parameters of the training model; and, returning the optimized training model to the machine learning model.

The invention provides a multi-agent question-answer interaction efficiency and performance optimization method and device based on a large model, and relates to the technical field of multi-agent interaction. The method comprises the steps of constructing a first communication graph and a first graph weight model according to a training question and answer task based on multiple agents; based on a first preset round, according to the training data and the first communication graph, performing first-stage optimization training on the first graph weight model by using an unbiased strategy gradient estimation method to obtain a second graph weight model and a second communication graph; based on a second preset round, using the training data and the second communication graph, and using an unbiased strategy gradient estimation method to carry out second-stage optimization training on the second graph weight model to obtain a third graph weight model; and according to the actual question and answer task, performing question and answer reasoning by using the third graph weight model to obtain an actual question and answer reasoning result. The multi-agent question and answer interaction efficiency and performance optimization method based on the multi-agent role is high in efficiency and good in robustness.

The invention discloses an automatic thinking chain prompt generation method based on black box optimization and vulnerability quantification. The method comprises the following steps: acquiring an unlabeled training data set; screening a high-difficulty problem subset through a difficulty judgment module, and generating an annotation data set; generating a plurality of reasoning chains for each question in the annotation data set by using a large language model, and reserving the reasoning chains consistent with correct answers to form an example library; a variance reduction strategy gradient estimator is adopted to optimize an inference chain selection strategy, and a common style prompt is generated; and performing style diversification processing on the common style prompt through a novel format prompt module to generate an anti-vulnerability automatic thinking chain prompt.

The embodiment of the invention provides a robust network optimization method, a related device and a medium. The method comprises the following steps: acquiring network sample data of a target network and antenna parameters of each cell in the target network; performing gradient estimation on the second optimization objective function through a zero-order estimation algorithm based on network sample data and antenna parameters to obtain an estimated gradient value; updating antenna parameters through gradient descent according to the estimated gradient value; respectively calculating a first value of a first optimization objective function and a second value of a second optimization objective function according to the updated antenna parameters; jumping to perform gradient estimation on the second optimization objective function through a zero-order estimation algorithm based on the network sample data and the antenna parameters to obtain an estimated gradient value until a preset condition is met so as to obtain a plurality of first values; and taking the antenna parameter corresponding to the minimum first value in the plurality of first values as a target antenna parameter. According to the embodiment of the invention, the efficiency and robustness of network optimization can be improved.

The invention discloses a parameter determination method and device of an analog circuit, a medium and a product, and relates to the technical field of integrated circuit design. Local performance feedback and global topology perception of performance index parameters are used as input parameters, and pertinence and generalization ability of parameter adjustment are remarkably improved. Through a strategy function, a parameter candidate set of a target analog circuit is obtained, through multi-group sampling processing, variance of strategy gradient estimation is reduced, and training stability and convergence speed are improved. Depending on a value network is not needed, and training resource overhead is reduced. And applying the parameter candidate set output by the strategy function to the simulation processing scene to realize the accuracy of simulation processing evaluation. The target performance indexes are extracted from the simulation result, optimization of the performance indexes is achieved, the multiple target performance indexes are processed according to the multi-target reward function, balance selection and rejection are achieved in the multiple performance indexes, the parameter combination corresponding to the highest advantage value is obtained, and the reliability of the analog circuit parameter determination process is improved.

The invention discloses a multi-robot fixed time cooperative hunting method and system based on a distributed time-varying optimization algorithm. The method comprises the following steps: establishing a dynamic model and a communication topological graph of a multi-robot system; the method comprises the following steps: converting a fixed time distributed hunting problem of a multi-robot system into a fixed time distributed time-varying optimization problem, and constructing a global objective function; designing a fixed time distributed gradient estimator, and enabling each robot to estimate gradient information of a system global objective function within fixed time in a distributed mode; designing a self-adaptive zero-order neural network which is used for approximating an inverse matrix of a Hessian matrix; and designing a fixed-time distributed time-varying optimization algorithm for the multi-robot system by adopting the gradient information of the global objective function estimated by the distributed gradient estimator in the step 3 and the inverse matrix of the Hessian matrix approximated by the adaptive zero-order neural network in the step 4, so that each robot encircles the dynamic target in the fixed time in a distributed mode. According to the invention, the multi-robot system is ensured to surround the moving target in a formation form within a fixed time.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network by estimating the objective function curvature based on current and previous gradients. In one aspect, a method comprises: sampling a batch of training data; and for each neural network parameter: determining, based on the current batch of training data, a respective current gradient of the objective function at the current iteration with respect to the current neural network parameter; estimating an objective function curvature with respect to the current neural network parameter based on (i) the current gradient of the objective function at the current iteration, and (ii) a respective previous gradient of the objective function at each of a plurality of previous iterations; and updating a current value of the neural network parameter based on the estimate of the curvature of the objective function.

The invention discloses a control method of a flat wire motor. The method comprises the following steps: constructing an optimal current instruction table taking a rotating speed and a torque as indexes; if the motor is in the dynamic working condition, directly looking up a table to obtain a current optimal current instruction; if in a steady-state working condition, performing online search by taking a table look-up result as an initial point: generating two-dimensional Bernoulli random disturbance, and sequentially applying positive and negative disturbances to the initial point; respectively measuring the input power under the two disturbances; according to the ratio of the positive and negative input power difference to the corresponding disturbance component, synchronously calculating gradient estimation vectors of the system input power to d-axis and q-axis current; according to a steepest descent method, the current instruction is updated in the reverse direction of the gradient, an optimal instruction enabling the input power to be minimized is obtained, and an inverter is driven to control a motor to operate. According to the method, through a hybrid control strategy combining dynamic table look-up and steady-state online search, the total loss minimization of the flat wire motor under all working conditions in consideration of alternating current copper loss is realized, and the rapidity of dynamic response is ensured.

The invention discloses a slope estimation method and device, equipment and a storage medium. The method comprises the steps that the current corresponding driving working condition of a vehicle is obtained; target limiting parameters matched with the driving working conditions are determined; performing limiting processing on the original acceleration difference value by using the target limiting parameter to obtain a target acceleration difference value; the original acceleration difference value is the difference value between the detection acceleration and the motion acceleration of the vehicle, the detection acceleration is obtained through detection of an acceleration sensor arranged on the vehicle, and the motion acceleration is determined based on the vehicle speed; and estimating the gradient of the road where the vehicle is currently located by using the target acceleration difference value. In this way, the accuracy of road slope estimation can be improved.

The embodiment of the invention provides a large language model optimization method based on variance reduction and momentum acceleration. The method comprises the following steps: a gradient estimation stage of a large language model: initializing a seed list and a projection list; executing multiple independent query iterations, calling a disturbance subprogram, generating a random seed for the large language model, and storing a table; in the disturbance subprogram, the determined gradient projection value is stored in a projection list, and iteration is carried out for next query; after executing multiple independent query iterations, storing a plurality of random seeds and a plurality of gradient projection values corresponding to the random seeds; in the weight updating stage of the large language model, a gradient norm subprogram is called for each layer of the large language model, and a random seed is obtained to reset a random number generator; and determining variance-reduced gradient estimation according to the gradient projection value extracted from the projection list and the reproduced disturbance vector. According to the embodiment of the invention, gradient information queried for multiple times is aggregated to generate low-noise gradient estimation, and fine tuning of a large language model is completed.

The invention relates to the field of generative artificial intelligence image processing, and provides an initial noise distribution optimization method, device, equipment, medium and program, and the method comprises the steps: taking a denoising process as a fixed mapping relation, creating a trainable distribution parameter, and constructing a distribution parameter updating formula based on the fixed mapping relation; constructing a dynamic reward calibration module, calculating a difference value between a reward value of a current initial noise distribution generated image and a reward value of an original standard normal distribution generated image after the diffusion model outputs the generated image every time, and taking the difference value as a relative reward value; constraining the updating process of the distribution parameters by adopting a proportional clipping algorithm; and calculating a parameter updating step length based on a gradient estimation result, and carrying out proportional clipping on the updating step length. The method is used for improving the consistency of content and prompt semantics in a text-to-image generation task by optimizing the initial distribution parameters of the diffusion model, and meanwhile, the generation quality and the calculation efficiency are kept.

The invention discloses a robot group detection scheduling method and system, and relates to disaster monitoring: scene reconstruction is carried out according to collected real-time sensing information, a three-dimensional digital model of a disaster scene is generated, and a discrete monitoring grid based on sampling points is established; constructing a disaster risk assessment model on the discrete monitoring points, and for gas and fire, calculating the risk value of each monitoring point by using actual measurement data to form a discrete risk distribution map; calculating the space change trend of the risk degree by adopting an adjacent point difference method, and calculating a directed gradient through the risk degree difference value and the distance of adjacent monitoring points; when the risk degrees of any two disasters are increased at the same time, marking the risk points as potential coupling risk points; identifying a propagation path of the risk degree based on connectivity analysis of the monitoring network; and scheduling the robot group by adopting a risk avoiding path planning algorithm. According to the method, the spatial distribution and evolution trend of the disaster field are accurately reconstructed through local gradient estimation under the sparse sampling condition.

The invention discloses a data matching method and device. Based on a gradient estimation principle, the method is used for adjusting the proportion of training data of different sources in a fine adjustment stage of a large model in a professional field. And matching the field training data and the general training data according to a first proportion related to the requirements of the professional field to obtain mixed training data. The method is used for fine tuning of the large model, and the accuracy of the fine-tuned large model is calculated. And adjusting the first proportion, and obtaining two reference accuracy rates based on the adjusted first proportion. According to comparison between the reference accuracy rate and the accuracy rate and between the two reference accuracy rates, the accuracy rate change trend of the large model after the proportion is adjusted is analyzed, the gradient direction is estimated, and the target proportion is obtained. And through scientific simulation and gradient change analysis, an ideal proportion is efficiently determined. It is ensured that the large model shows the best accuracy in the general and professional fields, and the high-level performance of the general task is maintained while the performance of the professional field is enhanced.

The invention belongs to the technical field of neural network model training optimization, and discloses a low-rank matrix gradient estimation method and system for large-scale neural network training, and the method comprises the steps: sampling a low-rank random subspace meeting the equidistant or isotropic constraint based on a preset sampling rule; a matrix gradient estimation algorithm is embedded into the low-rank random subspace to be executed, and accumulation and updating are carried out on low-dimensional auxiliary variables; when the low-dimensional auxiliary variable is accumulated to a preset step number, performing inertia updating on parameters of a matrix gradient estimation algorithm; and calculating a weighting matrix based on all parameters of the matrix gradient estimation algorithm after inert updating, and optimizing a preset sampling rule of a next round of low-rank random subspace according to spectrum information of the weighting matrix so as to realize gradient estimation of the low-rank matrix. According to the method, a scheme including estimation stage low-rank, projection distribution optimization and inertia updating is provided, the video memory and step time threshold is remarkably reduced in engineering, and considerable cost performance is embodied in large model fine tuning.

Systems and techniques are described herein for training a machine learning (ML) model. For instance, a process can include obtaining, from a teacher ML model, a first prediction based on an input. The process can further include obtaining, from a student ML model, a second prediction based on the input. The process can include determining a loss based on a difference between the second prediction from the first prediction. For instance, the loss can include a variance reduced total variation distance (TVD) loss based on an unbiased gradient estimate of the loss. The process can further include backpropagating the loss through the student ML model to train the student ML model.

According to one aspect of the present disclosure, a method of image processing is provided. The method may include performing gradient-estimation of a first color image and a first depth image using the gradient-estimation network to generate first depth-edge information. The method may include inputting a second depth image and the first depth-edge information into a depth-upsampling network. The method may include performing depth upsampling of the second depth image using the first depth-edge information to generate second depth-edge information. The method may include inputting the first depth-edge information and the second depth-edge information into a fusion network, which fuses the first depth-edge information and second depth-edge information to generate a residual map. The method may include combining the first depth image and the residual map to generate a third depth image.

The invention discloses a fine tuning system of a large-scale pre-training model in a federated learning environment and application thereof. The system comprises a local disturbance gradient estimation module, a differential privacy protection module and a global model aggregation and update module. The local disturbance gradient estimation module is used for calculating a global model loss value by combining forward propagation with a zero-order optimization method so as to estimate a gradient and realize all-parameter fine tuning; the differential privacy protection module performs differential privacy protection processing on the estimated disturbance gradient to prevent gradient information from leaking user sensitive data; and the global model aggregation and update module reconstructs a disturbance vector and completes global model update based on a random seed and a scalar gradient uploaded by a client. Compared with the prior art, on the premise of not depending on back propagation, all-parameter fine tuning of a large-scale pre-training model is achieved, data privacy is guaranteed, meanwhile, calculation and memory expenses are remarkably reduced, and the method is suitable for a resource-limited distributed calculation environment.

An apparatus for estimating a parking space gradient includes a parking space detection module that recognizes a type of a parking space based on an image captured by an image capturing device and recognizes a location of the parking space by detecting a plurality of key points corresponding to the parking space and a gradient estimation module that obtains angle information formed by the key points in the type of the parking space, estimates a rotation matrix based on estimation conditions previously set according to the angle information and the type of the parking space, and estimates a gradient of the parking space using the estimated rotation matrix and a predetermined reference rotation matrix.

The invention provides a gradient determination method and a vehicle, and the method comprises the steps: calculating a prediction vector of a current moment through employing an extended Kalman filtering algorithm according to a first calibration vector corresponding to a previous moment of the current moment; the first calibration vector includes a first calibrated longitudinal vehicle speed, a first calibrated gradient, and a first calibrated gradient change rate. According to the prediction vector at the current moment and the observation vector at the current moment, calculating a calibration gradient at the current moment by adopting an extended Kalman filtering algorithm; wherein the observation vector comprises kinematically estimated longitudinal vehicle speed and pitch angle speed. In the process of calibrating the prediction vector, the pitch angular velocity is fused, a gradient can be obtained through pitch angular velocity integration, that is, the gradient obtained through the pitch angular velocity and the predicted gradient in the prediction vector are mutually verified, and the precision and robustness of gradient estimation can be effectively improved. And the calculation rate of the pitch angle speed to obtain the gradient is higher, so that the tracking effect of gradient estimation can be improved.

The invention provides a filtering-based Monte Carlo gradient path tracking rendering image reconstruction method and device. The method comprises the steps of obtaining an initial rendering image obtained by Monte Carlo gradient path tracking rendering and an auxiliary information graph in a rendering process; solving an optimized gradient through a gradient optimization objective function according to the initial rendering image and the auxiliary information graph, and generating a multi-resolution hierarchy; based on the initial rendering image, the optimized gradient and the auxiliary information graph, performing filtering calculation layer by layer from the lowest resolution hierarchy to the highest resolution hierarchy according to the multi-resolution hierarchy to obtain a target filtering result; wherein the target filtering result is the filtering result output by the highest resolution level, and the filtering result output by the highest resolution level is the reconstruction result of the initial rendering image. According to the method, gradient domain information and an auxiliary information graph in the rendering process are fully utilized, filtering kernel coefficients are solved in a resolution-by-resolution hierarchy mode, a high-quality and consistent noise reduction image (reconstruction result) is generated at a low sampling rate, the influence of noise in gradient estimation on the reconstruction result is reduced, and the reconstruction efficiency is improved. And the method also has a series of good characteristics such as consistency (progressive convergence to a reference image) and the like, and meets high-efficiency and high-quality rendered image reconstruction requirements.

The invention discloses a data-free federated distillation method and system based on zero-order gradient estimation, and the method comprises the steps: initializing a global model and a synthetic image generator through a central server, generating a synthetic image sample in each communication round, transmitting the synthetic image sample to a client, and enabling the client to update a local model through local privacy data, and uploading a prediction result. And the central server calculates the gradient of the generator by using a zero-order gradient estimation technology, and updates generator parameters and global model parameters. And by introducing fidelity loss, adversarial loss, diversity loss and negative information entropy loss, the performance of the generator is optimized. And the gradient of the generator is calculated through zero-order gradient estimation, so that the requirement of accessing a local model of a client is avoided, and privacy is effectively protected. The global model is updated by minimizing knowledge distillation loss, and at the same time, the client further optimizes the local model by receiving an integrated prediction result. The method has the advantage that the communication bandwidth requirement is reduced by reducing the access to the private data of the client.

A control device 50 as a road gradient estimation device estimates the road gradient of a road on which a vehicle 10 is traveling on the basis of an inertial acceleration derived on the basis of an inertial force in a traveling direction of the vehicle 10 and an actual acceleration derived on the basis of the amount of change in the vehicle speed of the vehicle 10. The control device 50 includes a forward / backward movement determination unit M11 for modifying a forward / backward movement determination indicating whether the vehicle 10 is moving backward. The control device 50 includes an acceleration correction unit M12 that acquires the inertial acceleration as a first acceleration and the actual acceleration as a second acceleration, and, when the forward / backward movement determination is modified, performs correction to invert the sign of the first acceleration or the second acceleration. The control device 50 includes a gradient estimation unit for estimating the road gradient on the basis of the difference between the first acceleration and the second acceleration.

The invention relates to the field of artificial intelligence security, and provides an adversarial sample optimization method and device, equipment and a program product. The method comprises the following steps: acquiring an audio sample; according to the audio sample, obtaining a candidate sample by using a preset genetic algorithm; and according to the candidate sample, performing sparse gradient estimation in combination with the target model, and optimizing the candidate sample in combination with a sparse gradient estimation result. According to the confrontation sample optimization method provided by the invention, the acquired audio sample is globally searched by using the preset genetic algorithm to find the closest candidate sample and avoid falling into a locally optimal solution, and sparse gradient estimation is combined to more accurately capture the key decision boundary of the target model, so that a more effective confrontation sample can be generated, and the accuracy of the confrontation sample optimization is improved. Meanwhile, unnecessary disturbance is avoided, the optimization pertinence is improved, the real-time requirement is met, the calculation overhead is reduced, and the optimization efficiency is improved.

The invention discloses an all-working-condition road surface gradient estimation method and device and a storage medium. The method comprises the steps that vehicle measurement data, a vehicle longitudinal kinematics model and a vehicle longitudinal dynamics model are obtained; constructing a kinematic system state equation and a kinematic system observation equation; constructing a dynamic system state equation and a dynamic system observation equation; performing abnormal point detection processing on the vehicle measurement data by using a box graph method to obtain target vehicle data; calculating a first road slope value under the vehicle longitudinal kinematics model by using a Kalman filtering method; calculating a second road surface slope value under the vehicle longitudinal dynamic model by using a recursive least square method; and carrying out data fusion by utilizing an interactive multi-model method to obtain a full-working-condition road surface slope value which is used for adjusting vehicle driving parameters. According to the invention, all-condition road slope estimation is realized, and the accuracy is improved. The method can be widely applied to the technical field of vehicle engineering.

The embodiment of the invention discloses a ship trajectory tracking event trigger control method and system with a performance game mechanism. The method comprises the following steps: S1, determining a position error and an attitude error corresponding to a ship; s2, designing a first performance index and a first value function; s3, designing a negative gradient estimated value, an optimal virtual control law and an online learning law of the first value function; s4, obtaining a dynamic reference signal and a dynamic control error; s5, defining a second performance index and a corresponding second value function; s6, designing a negative gradient estimated value, an optimal control input and an optimal trigger error corresponding to the second value function; s7, judging whether a triggering rule is met or not; and S8, changing the state of the ship to continuously judge whether the navigation task is completed or not, and if not, re-confirming the current state of the ship and returning to S1. The method solves the problems that in the prior art, it is difficult for an operator to obtain the best triggering effect and control precision, and the robustness of a ship trajectory tracking controller in marine environment interference is poor.