72results about How to "Reduce inference latency" patented technology

The invention belongs to the technical field of model training, and particularly relates to a large model increment training method and system based on dynamic sparsification. The method comprises the following steps: S1, evaluating the parameter importance of a large model by adopting a multi-dimensional evaluation method, and reflecting the contribution of parameters to the performance of the large model; s2, performing dynamic sparsification control on the large model structure by adopting a self-adaptive structure adjustment method; s3, performing incremental training management on the large model by constructing a capacity-adjustable empirical playback buffer pool and proposing a multi-objective loss function; and S4, carrying out model compression optimization on the large model by adopting a mixed precision quantification strategy. The method has the characteristics that the network structure can be dynamically adjusted by evaluating the importance of model parameters in real time, so that the efficient training is realized while the model performance is ensured.

The invention discloses a large model online reasoning service implementation method based on no-service calculation, and relates to the technical field of model reasoning, and the implementation method comprises the following operations: constructing a large model resource sniffing interface for realizing an API Server mode of sniffing server resources first and then transmitting reasoning data JSON; the model preloading module is constructed and used for rapidly loading a large model by using a model fragmentation technology; a shared memory module is constructed and used for planning and managing GPU and CPU memories and dynamically deducing the utilization rate of the simulated memories; gPU resource sharing is used when model reasoning and concurrent elastic capacity expansion and contraction are realized based on a service-free computing principle. According to the method, the problems of relatively high occupation of online reasoning resources, slow preheating of the resources and the like of a large model are solved, resource waste can be avoided, the utilization rate of the resources is improved, online reasoning service with quick response is realized, and the user experience is remarkably improved.

The invention provides a data set keyword generation and screening method based on a large language model, and belongs to the field of natural language processing and artificial intelligence. The method comprises the following steps: collecting comment text data and known keywords thereof; dividing the text data into blocks; generating preliminary keywords by the large language model; performing preliminary screening on the preliminary keywords in the preliminary keyword list in combination with a TF-IDF algorithm and a TextRank algorithm; and performing semantic and emotion correlation screening on the preliminarily screened keywords through a large language model to obtain final keywords. According to the method, the keyword generation and screening technology is remarkably improved, and the method is particularly suitable for large-scale text analysis tasks needing high precision and high efficiency.

The invention discloses a method for acquiring an optimal grabbing pose of a plug seedling. The method depends on a two-finger clamping jaw of graded transplanting equipment. The two-finger clamping jaw is positioned above the hole tray; the method comprises the following steps: 1, representing a grabbing pose: # imgabs0 # 2, constructing a data set: shooting a hole tray image from a vertically downward visual angle; the shot images are segmented and zoomed according to plug grids, and pixel-level capture pose labeling is carried out on each image; 3, training a neural network: inputting a required original image into the real-time grabbing detection network, and outputting the grabbing confidence degree, the grabbing angle and the grabbing width of each pixel point in the image by the real-time grabbing detection network; according to the method, the optimal grabbing pose of each seedling in the plug is obtained by fusing machine vision and deep learning, the seedlings are grabbed based on the optimal grabbing poses, leaves and stalks of the seedlings can be effectively avoided, the damage rate in the plug seedling grabbing process is reduced, and the survival rate of the transplanted seedlings is increased.

The invention discloses a training method of an image generation model, and an image processing method and device. Relates to the field of artificial intelligence, and the method comprises the steps: obtaining a training sample image set which comprises a plurality of spliced feature maps; the plurality of spliced feature maps are input into an initial student model to obtain a first prediction result output by the initial student model, the plurality of spliced feature maps are input into a teacher model to obtain a second prediction result output by the teacher model, and the initial student model is obtained by initializing parameters of the teacher model; the teacher model is a pre-trained text graph generative model; and constructing a target loss function based on the first prediction result and the second prediction result, and training parameters of the initial student model based on the target loss function to obtain a target image generation model. The technical problem of low image generation efficiency caused by high reasoning delay due to more reasoning step number and parameter quantity of an image generation model in the prior art is solved.

The invention belongs to the field of artificial intelligence, and particularly relates to an unloading method and device for a sparse hybrid expert large language model reasoning task. The method comprises the following steps: predicting selected experts and corresponding active neurons of the sparse hybrid expert large language model in a reasoning task; the active neurons of the selected expert are loaded at a computing device to perform the reasoning task. According to the method and the device, the reasoning delay when the sparse hybrid expert large language model is unloaded on the consumer-level GPU when executing the reasoning task is reduced.

The invention belongs to the technical field of software engineering, particularly relates to an automatic driving system performance enhancement method based on an expert hybrid architecture, and aims to solve the core problems that an end-to-end automatic driving system is confronted with semantic fuzziness to cause unreliable decision, multi-task interference hinders optimization planning, too long reasoning delay increases driving risks and the like. According to the method, an ExpertAD framework is provided, task key features are amplified through a perception adapter (PA), and the relevance of scene context understanding is guaranteed; related driving tasks are dynamically activated through a sparse expert mixture (MoSE), and task interference is minimized; and in combination with a customized training loss function, collaborative optimization of planning effectiveness and reasoning efficiency is realized. Experiments show that compared with an existing method, the method has the advantages that the average collision rate is reduced by 20%, the reasoning delay is reduced by 25%, higher multi-skill planning capacity is achieved in rare scenes (such as accident handling and first-aid vehicle avoiding), and good generalization is achieved for unseen urban environments.

The invention relates to a multi-agent cooperation-based language model reasoning resource scheduling method and system, and the method achieves the intelligent simulation of multi-view iterative thinking in a complex reasoning task through the construction of a multi-agent system which is clear in division of labor and is provided with a special knowledge base. View limitation and decision deviation of a single model in long-sequence and multi-step reasoning are effectively overcome; depending on the fusion of the general capability of the large language model and the task special knowledge base, the professionality and accuracy of the reasoning result are improved. Meanwhile, the problems of resource waste, redundant calculation and unstable convergence caused by cognitive overload in the cooperation process are solved by combining multi-round dynamic cooperation with a convergence mechanism of cognitive load perception and a dynamic token number limitation and low-confidence branch pruning strategy; therefore, on the premise that the reasoning quality is guaranteed, the utilization efficiency of computing resources is remarkably optimized, peak value occupation is reduced, the overall reasoning time delay is shortened, and reliable technical support is provided for efficient and stable deployment of a large language model in a complex task.

The invention discloses a large model reasoning hardware accelerator based on data flow execution, and belongs to the technical field of large model reasoning hardware acceleration. Each node comprises a linear calculation kernel used for sequentially completing query / key / value projection and attention output projection in multi-head attention calculation, dimension raising projection and dimension reduction projection calculation in a feedforward neural network layer, and inverse quantization operation after each projection is output; the multi-head attention calculation kernel is used for completing loading and quantization of query / key / value vectors, multi-head division and cache management, attention score calculation, Softmax normalization, context vector generation and KV cache updating; the RLA kernel is used for completing residual addition, layer normalization, quantization / inverse quantization and nonlinear activation calculation of each layer; the three cores in the same node are located in the same SLR area, and data transmission is carried out through a hardware queue and an HLS blocking mechanism. According to the invention, high hardware utilization rate and low-delay reasoning can be realized.

The invention provides a neural network multi-objective optimization and FPGA hardware acceleration collaborative design method, and belongs to the field of deep learning model compression and hardware collaborative design. The method comprises the following steps: constructing a joint optimization space containing a neural network compression parameter and an FPGA hardware design parameter; a multi-target Bayesian optimization search strategy is adopted, iterative search is carried out in the joint optimization space, model precision, FPGA resource occupation and reasoning delay are synchronously optimized, and optimal candidate configuration is obtained; matching the compressed network structure with the FPGA parallel architecture by using a hardware-perceived pruning and quantification strategy; a multi-task performance prediction model is adopted to quickly predict the precision, resource occupation and delay of the optimal candidate configuration so as to accelerate the search process; according to the optimal configuration, a hardware accelerator code facing the target FPGA is automatically generated, and integration and implementation are completed. According to the method, collaborative optimization of neural network compression and hardware design is achieved, FPGA resource occupation can be remarkably reduced, the reasoning speed can be increased, and meanwhile the model precision is kept.

The invention provides an update processing method and device, equipment, a readable storage medium and a product, and relates to the technical field of communication. The method is executed by first equipment and comprises the following steps: acquiring a personal data set of a target user; determining a general data set according to the personal data set and a target model; wherein a first parameter is added to an important layer of the target model; and updating the first parameter according to the personal data set and the general data set. According to the scheme of the application, the problem of leakage risk existing in an end-cloud collaborative optimization scheme adopted by the target model is solved.

This invention relates to a segmented hybrid inference method for large language models based on uncertainty-driven approaches. The method includes acquiring current text data and historical state features to estimate the uncertainty index of the current segment; minimizing a unified scheduling objective function based on the uncertainty index to obtain a target inference pattern; performing inference calculations based on the target inference pattern to generate information contribution values ​​corresponding to key-value pairs; calculating the corresponding dynamic merging control probabilities based on the uncertainty index and information contribution values, and performing weighted merging or pruning on the key-value pairs to be merged to obtain compressed key-value pairs; defining a deviation metric and limiting the deviation metric to not exceed a preset upper bound determined by the dynamic merging control probability set and the uncertainty index; triggering a rollback process when the deviation exceeds this limit; otherwise, feeding back the compressed key-value pair state to the next segment for iterative iteration until the inference of all segments is completed; thereby reducing memory usage and inference latency while ensuring accuracy.

The invention relates to the technical field of computer vision and coal mine intelligent mining, and discloses a lightweight working face three-dimensional reconstruction method and device fusing geometric prior constraints, and the method comprises the steps: firstly screening a dynamic effective region based on a projection relation and texture features, and removing static redundant scenes; performing sparse visual reasoning and depth regression only for the effective area to generate an initial depth map; carrying out physical structure correction on the depth map by utilizing geometric priori such as vertical upright posts and parallel top plates of the hydraulic support; and finally, constructing an optimization objective function containing deviation constraints of the vertical column perpendicularity and the top plate parallelism, and completing high-precision fusion of the incremental point cloud. According to the method, inference delay is reduced by reducing invalid calculation, structural distortion and registration drift caused by an underground severe environment are effectively inhibited by using a geometric law, and real-time and high-fidelity reconstruction of a working face scene is realized.

The application discloses a kind of machine learning model inference and forget request collaborative processing method, machine learning online service provider is updated model in real time in background based on forget request, and robustness detection is carried out to user inference request, service scheduling is carried out according to detection result, to realize faster response speed.The method comprises the following steps:1) sub-model training: split data set and train multiple sub-models with independent inference ability and online service;2) forget request processing: corresponding sub-model is updated in real time based on forget request received during service;3) inference request response: high robustness inference request is responded preferentially, and low robustness request is responded based on new model after model is completed updating.The application first proposes the collaborative processing method of multiple requests in the online service scene of machine learning, while guaranteeing privacy and correctness, higher response speed is realized, which is superior to existing method, with the advantages of strong universality, easy to deploy and the like.

The application provides an edge device-oriented lightweight multi-modal early gastric cancer collaborative auxiliary diagnosis system, which comprises a lightweight multi-modal classification module, a decision-level fusion unit, a first feature extraction network and a second feature extraction network; the decision-level fusion unit is used for evidence theory fusion of first classification probability and second classification probability, and outputs a multi-modal classification result; a lightweight segmentation module is constructed based on an encoding-decoding structure, an encoder of the lightweight segmentation module comprises a plurality of stacked lightweight bottleneck modules, a feature enhancement module is arranged in a skip connection of the encoder, and the feature enhancement module is used for pixel-level lesion segmentation of a white light endoscopy image; a gating cooperation unit is connected with the lightweight multi-modal classification module and the lightweight segmentation module respectively, and is used for generating a gating signal according to the multi-modal classification result; when the multi-modal classification result is positive, the lightweight segmentation module is activated to perform a segmentation task; and when the multi-modal classification result is negative, a subsequent segmentation process is terminated.

The application provides a video stream multi-label classification method and system, and belongs to the field of computer vision. The application inputs a current frame image into a shared backbone network and a selective spatial scanning module to obtain a global feature vector; a cosine similarity of a current frame feature and a historical memory feature is calculated and compared with a preset threshold value, if greater than the threshold value, a classification result of a last effective frame is directly reused; otherwise, complete reasoning is performed: a low-rank adapter is activated according to a label co-occurrence semantic cluster mapping relationship of a candidate label set, and an exact weight is obtained by accumulating and normalizing the absolute value of the label score to complete parameter modulation; a heat map is generated by using gradient weighted class activation mapping to guide dynamic local area sampling; finally, global and local predictions are fused and the historical memory is incrementally updated. The application significantly reduces the power redundancy and energy consumption, and improves the classification accuracy, long-tail recognition and time sequence stability.

The present application relates to the field of agricultural environmental information technology, and discloses a remote control method for nitrogen and phosphorus loss amount of paddy field facing environmental data. The method comprises the following steps: obtaining historical nitrogen and phosphorus loss data, real-time meteorological data and water and fertilizer operation information of the paddy field; constructing a mechanism constraint sub-model combining nitrogen migration and transformation dynamics and phosphorus adsorption and desorption balance; performing multi-source feature extraction on the meteorological data to generate a standardized environmental vector; inputting the light residual convolutional neural network to output an initial prediction value; combining the mechanism model to generate a corrected prediction value conforming to the conservation of mass through the Lagrange multiplier method; when the prediction value exceeds the threshold value, generating a drainage gate opening degree and a fertilizer plan adjustment instruction, and delivering the instruction to an edge terminal for execution through a narrowband Internet of Things.

The embodiments of the present application relate to the field of deep learning technology, and in particular to a method, system, electronic device and storage medium for inference optimization of a model, the method comprising: obtaining operator information of the model and hardware information of a central processing unit CPU; determining a code generation strategy according to the operator information of the model, the hardware information of the CPU and a preset strategy generation algorithm; wherein the code generation strategy comprises a scheduling code generation strategy and a calculation code generation strategy; according to the code generation strategy, a scheduling code and a calculation code are generated; the scheduling code and the calculation code are compiled and linked to obtain a compiled model file. The inference optimization method of the model provided in the embodiments of the present application can make full use of the computing power of the CPU, greatly improve the speed and efficiency of model inference, and realize targeted optimization of the model.

The application discloses a behavior tree automatic generation method and device based on a large language model and electronic equipment, relates to the technical field of artificial intelligence and automatic control, and comprises the following steps: acquiring task description information, constructing a node variable context object, injecting a structured prompt word, and forming a resource constraint-aware generation context; according to a semantic vector of the task description, searching for high-quality cases that meet a quality threshold, screening dynamic few-sample examples through a selection algorithm, and embedding generation prompt words; calling a large language model deployed locally to generate a behavior tree, performing syntax checking and logic defect detection on the generation result through an evaluation operator, converting a detection failure result into structured feedback information, driving the model to perform minimum iteration refinement until the checking is passed or the maximum number of iterations is reached. The application solves the problems of low availability, poor stability and high deployment cost of the large language model in generating the behavior tree, and enables a lightweight large language model to achieve a generation success rate of about 90% on consumer-grade hardware.

The application provides a multi-scene safety inspection method and system based on a UAV, and belongs to the technical field of UAV safety inspection, and comprises the following steps: constructing a multi-scene safety sample library; training a multi-task computer vision model based on the multi-scene safety sample library to obtain a safety inspection algorithm model integrating multiple recognition capabilities; the multi-task computer vision model comprises a shared feature extraction backbone network and multiple independent task head networks; a training set is input into the shared feature extraction backbone network to extract general feature maps; the general feature maps are respectively input into the multiple independent task head networks for parallel training to obtain the safety inspection algorithm model; and the safety inspection algorithm model is loaded on a UAV flight platform to perform parallel inference analysis on inspection images collected by the UAV in real time, and to identify preset safety hidden danger targets.

The invention relates to a large language model segmentation hybrid reasoning method based on uncertainty driving, which comprises the following steps of: acquiring current text data and historical state characteristics to pre-estimate an uncertainty index of a current segment; according to the uncertainty index, minimizing the unified scheduling target function to obtain a target reasoning mode; performing reasoning calculation according to the target reasoning mode, and generating information contribution degrees corresponding to the key values; according to the uncertainty index and the information contribution degree, calculating a corresponding dynamic merging control probability, and performing weighted merging or pruning on the to-be-merged key value pairs to obtain compressed key value pairs; defining a deviation metric, and limiting the deviation metric not to exceed a preset upper bound determined by the dynamic merging control probability set and the uncertainty index; if so, triggering fallback processing; otherwise, the compressed key value pair state is fed back to the next segment for loop iteration until reasoning of all segments is completed; therefore, the video memory occupation and the reasoning delay are reduced on the premise of ensuring the precision.

The invention discloses a large model dialogue control method and system, a storage medium and computer equipment. The large model dialogue method comprises the steps of converting user input data and dialogue historical data into state perception vectors; selecting a plurality of expert models matched with the state sensing vector from a preset expert model pool; respectively inputting the user input data and the dialogue historical data into the selected expert models to obtain a plurality of candidate replies; and calculating a value score of each candidate reply, and outputting the candidate reply with the highest value score as a final dialogue reply. Through the above method, dialogue context semantics and user core intentions can be accurately captured, dynamic sparse activation and integration of knowledge of a multi-expert model can be realized, computing resource consumption and reasoning delay can be significantly reduced, dialogue reply accuracy and reliability and cross-style and cross-field adaptivity can be improved, and dialogue reply experience can be improved. And the process has relatively high interpretability and controllability, so that debugging and optimization of a large model are facilitated.

The application belongs to the technical field of intelligent robot control, and particularly relates to a self-adaptive step length control method for a live broadcast interactive robot; the method comprises the following steps: robot control instructions are recognized from a barrage text information, and the robot control instructions and a live broadcast scene image are input into a vision-language-action model for processing to obtain a candidate action sequence under current iteration; window action deviations and convergence slopes of action sites in the candidate action sequence are calculated in sequence; if the window action deviations and the convergence slopes of the first m action sites all satisfy a stable condition, and the (m+1)th action site does not satisfy the stable condition, the first m action sites are output; otherwise, the candidate action sequence is output again; the live broadcast interactive robot executes the first m action sites, and continues to process remaining action sites in the candidate action sequence until the entire robot control instruction is executed; the application can realize high-frequency and low-delay interaction while ensuring control stability, and is more suitable for a cost-sensitive and high-concurrency live broadcast environment.

The present disclosure relates to a layered parallel inference acceleration system, method and computer program product. The system comprises a text content analysis module for analyzing and processing user input text content to obtain a vector sequence; a candidate token sequence generation module for determining a corresponding set of to-be-predicted positions according to the current position of each of the plurality of vectors, and determining the feature information corresponding to each to-be-predicted position in the set of to-be-predicted positions; and predicting at least one candidate token according to the feature information, and combining each corresponding at least one candidate token in the set of to-be-predicted positions to generate a plurality of candidate token sequences; and a token sequence verification module for screening a candidate token sequence with the most number of candidate tokens that are continuously verified from the start position from the plurality of candidate token sequences as a target token sequence. In this way, the diversity and quality of the generated content can be ensured while reducing the model inference delay.

The application discloses a speech recognition fine-tuning method based on subspace decomposition and reorganization. The low-rank space of the LoRA module is equivalently decomposed into multiple subspaces, and learnable weights are added to different subspaces, so that the reorganization of all subspaces is realized. The reorganized subspaces can be transformed into an efficient calculation mode during training and an inference mode during inference, which significantly improves the inference and storage efficiency, and solves the problem of high inference cost during high concurrency request while ensuring efficient fine-tuning of parameters.

The application discloses an empty chip integrated with a physical information graph neural network acceleration core and a potential field regulation method thereof, and belongs to the technical field of quantum semiconductor integrated circuits and space field energy regulation. The empty chip comprises an electrode array layer, a quantum bit layer, a potential source topology mapping module, a GNN hardware acceleration core and a controller. The GNN hardware acceleration core is completely integrated in the chip and comprises a graph convolution operation unit, a physical constraint storage unit and a voltage mapping unit, and is used for hardwareizing GNN operation logic. The potential field regulation method comprises the following steps: the potential source topology mapping module constructs a potential source space topology graph; the GNN hardware acceleration core calls rules in the physical constraint storage unit, iteratively solves a global optimal energy distribution through hardware-level graph convolution operation, and converts the optimal energy value into a driving voltage loaded to the electrode array layer through the voltage mapping unit, so as to construct a dynamic adjustable nonlinear potential well. The application solidifies the neural network algorithm into a hardware IP core, avoids the risk of rejection of the intellectual activity rules of the Patent Law, realizes real-time self-adaptive closed-loop regulation of the space potential field, solves the energy instability problem caused by the space potential fluctuation, and is suitable for space passive energy acquisition and new propulsion scenarios.

The application relates to a cloud service workload prediction method and system based on a convolution enhanced Transformer, which comprises the following steps: collecting cloud server workload data and preprocessing the cloud server workload data to decompose the cloud server workload data into a trend component and a residual component; constructing a workload prediction model, training the workload prediction model by using the trend component and the residual component, and obtaining a trained workload prediction model; predicting the cloud server workload by using the trained workload prediction model to obtain a workload prediction value; predicting the trend component by using a trend information capturing module to obtain a prediction result of the trend component; predicting the residual component by using a convolution enhanced Transformer encoder module to obtain a prediction result of the residual component; and fusing the prediction result of the trend component and the prediction result of the residual component by using a feature fusion module to obtain a final workload prediction value.

This invention discloses a point cloud processing method and system based on spatial locality-preserving serialization and adaptive mesh reshaping. The method includes: first, acquiring raw 3D point cloud data and quantizing and sorting it into a one-dimensional locality-preserving sequence using spatial filling curves such as Morton codes; second, introducing an adaptive mesh reshaping strategy, dynamically configuring a 2D target canvas according to the number of input point clouds, using lossless rectangular mapping for small-scale point clouds, and employing a spatial continuous truncation mechanism to extract dense topological sub-blocks for large-scale scene point clouds; finally, inputting the generated 2D pseudo-image into a 2D attention network to extract features and outputting the processing result. This invention completely eliminates zero-filling artifacts in cross-modal mapping without relying on expensive 3D domain query operators, significantly improving the computational efficiency and feature representation accuracy of 3D point cloud data processing.

This invention discloses a decoder-free multi-branch attention table large-scale model and its reinforcement learning training method. Addressing the problems of redundant decoder structures, high computational overhead, and mismatch between training objectives and evaluation metrics in traditional supervised learning models, this invention proposes a multi-branch attention network based on an encoder-only architecture. This method removes the decoder portion from the traditional Transformer architecture, directly extracting feature interaction information using a multi-branch encoder. During the training phase, the Group Relative Policy Optimization (GRPO) algorithm is introduced for reinforcement learning training, abandoning the Critic network in traditional RL and directly calculating the relative advantage by sampling a set of outputs from the same input. This invention significantly reduces the number of model parameters and memory usage, while improving the accuracy and inference speed of table data processing by directly optimizing sequence-level rewards.

The invention relates to an electroplated metal quality real-time monitoring method based on deep learning, and belongs to the technical field of industrial automation and intelligent detection. Comprising the following steps: step 1, image acquisition and preprocessing; step 2, deep learning of a feature extraction and detection framework; step 3, carrying out integrated detection without NMS post-treatment; and step 4, defect classification and real-time feedback. According to the invention, through multi-scale feature fusion and large kernel convolution, the capturing capability of tiny defects is significantly improved, high detection precision can be maintained in complex illumination and strong reflection environments, and the method has the characteristics of high precision, high efficiency and high adaptability, and is suitable for various industrial scenes.