60results about How to "Reduce inference latency" patented technology

The invention belongs to the technical field of software engineering, particularly relates to an automatic driving system performance enhancement method based on an expert hybrid architecture, and aims to solve the core problems that an end-to-end automatic driving system is confronted with semantic fuzziness to cause unreliable decision, multi-task interference hinders optimization planning, too long reasoning delay increases driving risks and the like. According to the method, an ExpertAD framework is provided, task key features are amplified through a perception adapter (PA), and the relevance of scene context understanding is guaranteed; related driving tasks are dynamically activated through a sparse expert mixture (MoSE), and task interference is minimized; and in combination with a customized training loss function, collaborative optimization of planning effectiveness and reasoning efficiency is realized. Experiments show that compared with an existing method, the method has the advantages that the average collision rate is reduced by 20%, the reasoning delay is reduced by 25%, higher multi-skill planning capacity is achieved in rare scenes (such as accident handling and first-aid vehicle avoiding), and good generalization is achieved for unseen urban environments.

The invention relates to a multi-agent cooperation-based language model reasoning resource scheduling method and system, and the method achieves the intelligent simulation of multi-view iterative thinking in a complex reasoning task through the construction of a multi-agent system which is clear in division of labor and is provided with a special knowledge base. View limitation and decision deviation of a single model in long-sequence and multi-step reasoning are effectively overcome; depending on the fusion of the general capability of the large language model and the task special knowledge base, the professionality and accuracy of the reasoning result are improved. Meanwhile, the problems of resource waste, redundant calculation and unstable convergence caused by cognitive overload in the cooperation process are solved by combining multi-round dynamic cooperation with a convergence mechanism of cognitive load perception and a dynamic token number limitation and low-confidence branch pruning strategy; therefore, on the premise that the reasoning quality is guaranteed, the utilization efficiency of computing resources is remarkably optimized, peak value occupation is reduced, the overall reasoning time delay is shortened, and reliable technical support is provided for efficient and stable deployment of a large language model in a complex task.

The invention discloses a large model reasoning hardware accelerator based on data flow execution, and belongs to the technical field of large model reasoning hardware acceleration. Each node comprises a linear calculation kernel used for sequentially completing query / key / value projection and attention output projection in multi-head attention calculation, dimension raising projection and dimension reduction projection calculation in a feedforward neural network layer, and inverse quantization operation after each projection is output; the multi-head attention calculation kernel is used for completing loading and quantization of query / key / value vectors, multi-head division and cache management, attention score calculation, Softmax normalization, context vector generation and KV cache updating; the RLA kernel is used for completing residual addition, layer normalization, quantization / inverse quantization and nonlinear activation calculation of each layer; the three cores in the same node are located in the same SLR area, and data transmission is carried out through a hardware queue and an HLS blocking mechanism. According to the invention, high hardware utilization rate and low-delay reasoning can be realized.

The invention provides a neural network multi-objective optimization and FPGA hardware acceleration collaborative design method, and belongs to the field of deep learning model compression and hardware collaborative design. The method comprises the following steps: constructing a joint optimization space containing a neural network compression parameter and an FPGA hardware design parameter; a multi-target Bayesian optimization search strategy is adopted, iterative search is carried out in the joint optimization space, model precision, FPGA resource occupation and reasoning delay are synchronously optimized, and optimal candidate configuration is obtained; matching the compressed network structure with the FPGA parallel architecture by using a hardware-perceived pruning and quantification strategy; a multi-task performance prediction model is adopted to quickly predict the precision, resource occupation and delay of the optimal candidate configuration so as to accelerate the search process; according to the optimal configuration, a hardware accelerator code facing the target FPGA is automatically generated, and integration and implementation are completed. According to the method, collaborative optimization of neural network compression and hardware design is achieved, FPGA resource occupation can be remarkably reduced, the reasoning speed can be increased, and meanwhile the model precision is kept.

This invention relates to a segmented hybrid inference method for large language models based on uncertainty-driven approaches. The method includes acquiring current text data and historical state features to estimate the uncertainty index of the current segment; minimizing a unified scheduling objective function based on the uncertainty index to obtain a target inference pattern; performing inference calculations based on the target inference pattern to generate information contribution values ​​corresponding to key-value pairs; calculating the corresponding dynamic merging control probabilities based on the uncertainty index and information contribution values, and performing weighted merging or pruning on the key-value pairs to be merged to obtain compressed key-value pairs; defining a deviation metric and limiting the deviation metric to not exceed a preset upper bound determined by the dynamic merging control probability set and the uncertainty index; triggering a rollback process when the deviation exceeds this limit; otherwise, feeding back the compressed key-value pair state to the next segment for iterative iteration until the inference of all segments is completed; thereby reducing memory usage and inference latency while ensuring accuracy.

The invention relates to the technical field of computer vision and coal mine intelligent mining, and discloses a lightweight working face three-dimensional reconstruction method and device fusing geometric prior constraints, and the method comprises the steps: firstly screening a dynamic effective region based on a projection relation and texture features, and removing static redundant scenes; performing sparse visual reasoning and depth regression only for the effective area to generate an initial depth map; carrying out physical structure correction on the depth map by utilizing geometric priori such as vertical upright posts and parallel top plates of the hydraulic support; and finally, constructing an optimization objective function containing deviation constraints of the vertical column perpendicularity and the top plate parallelism, and completing high-precision fusion of the incremental point cloud. According to the method, inference delay is reduced by reducing invalid calculation, structural distortion and registration drift caused by an underground severe environment are effectively inhibited by using a geometric law, and real-time and high-fidelity reconstruction of a working face scene is realized.

The application discloses a kind of machine learning model inference and forget request collaborative processing method, machine learning online service provider is updated model in real time in background based on forget request, and robustness detection is carried out to user inference request, service scheduling is carried out according to detection result, to realize faster response speed.The method comprises the following steps:1) sub-model training: split data set and train multiple sub-models with independent inference ability and online service;2) forget request processing: corresponding sub-model is updated in real time based on forget request received during service;3) inference request response: high robustness inference request is responded preferentially, and low robustness request is responded based on new model after model is completed updating.The application first proposes the collaborative processing method of multiple requests in the online service scene of machine learning, while guaranteeing privacy and correctness, higher response speed is realized, which is superior to existing method, with the advantages of strong universality, easy to deploy and the like.

The application provides an edge device-oriented lightweight multi-modal early gastric cancer collaborative auxiliary diagnosis system, which comprises a lightweight multi-modal classification module, a decision-level fusion unit, a first feature extraction network and a second feature extraction network; the decision-level fusion unit is used for evidence theory fusion of first classification probability and second classification probability, and outputs a multi-modal classification result; a lightweight segmentation module is constructed based on an encoding-decoding structure, an encoder of the lightweight segmentation module comprises a plurality of stacked lightweight bottleneck modules, a feature enhancement module is arranged in a skip connection of the encoder, and the feature enhancement module is used for pixel-level lesion segmentation of a white light endoscopy image; a gating cooperation unit is connected with the lightweight multi-modal classification module and the lightweight segmentation module respectively, and is used for generating a gating signal according to the multi-modal classification result; when the multi-modal classification result is positive, the lightweight segmentation module is activated to perform a segmentation task; and when the multi-modal classification result is negative, a subsequent segmentation process is terminated.

The application provides a video stream multi-label classification method and system, and belongs to the field of computer vision. The application inputs a current frame image into a shared backbone network and a selective spatial scanning module to obtain a global feature vector; a cosine similarity of a current frame feature and a historical memory feature is calculated and compared with a preset threshold value, if greater than the threshold value, a classification result of a last effective frame is directly reused; otherwise, complete reasoning is performed: a low-rank adapter is activated according to a label co-occurrence semantic cluster mapping relationship of a candidate label set, and an exact weight is obtained by accumulating and normalizing the absolute value of the label score to complete parameter modulation; a heat map is generated by using gradient weighted class activation mapping to guide dynamic local area sampling; finally, global and local predictions are fused and the historical memory is incrementally updated. The application significantly reduces the power redundancy and energy consumption, and improves the classification accuracy, long-tail recognition and time sequence stability.

The present application relates to the field of agricultural environmental information technology, and discloses a remote control method for nitrogen and phosphorus loss amount of paddy field facing environmental data. The method comprises the following steps: obtaining historical nitrogen and phosphorus loss data, real-time meteorological data and water and fertilizer operation information of the paddy field; constructing a mechanism constraint sub-model combining nitrogen migration and transformation dynamics and phosphorus adsorption and desorption balance; performing multi-source feature extraction on the meteorological data to generate a standardized environmental vector; inputting the light residual convolutional neural network to output an initial prediction value; combining the mechanism model to generate a corrected prediction value conforming to the conservation of mass through the Lagrange multiplier method; when the prediction value exceeds the threshold value, generating a drainage gate opening degree and a fertilizer plan adjustment instruction, and delivering the instruction to an edge terminal for execution through a narrowband Internet of Things.

The application discloses a behavior tree automatic generation method and device based on a large language model and electronic equipment, relates to the technical field of artificial intelligence and automatic control, and comprises the following steps: acquiring task description information, constructing a node variable context object, injecting a structured prompt word, and forming a resource constraint-aware generation context; according to a semantic vector of the task description, searching for high-quality cases that meet a quality threshold, screening dynamic few-sample examples through a selection algorithm, and embedding generation prompt words; calling a large language model deployed locally to generate a behavior tree, performing syntax checking and logic defect detection on the generation result through an evaluation operator, converting a detection failure result into structured feedback information, driving the model to perform minimum iteration refinement until the checking is passed or the maximum number of iterations is reached. The application solves the problems of low availability, poor stability and high deployment cost of the large language model in generating the behavior tree, and enables a lightweight large language model to achieve a generation success rate of about 90% on consumer-grade hardware.

The application provides a multi-scene safety inspection method and system based on a UAV, and belongs to the technical field of UAV safety inspection, and comprises the following steps: constructing a multi-scene safety sample library; training a multi-task computer vision model based on the multi-scene safety sample library to obtain a safety inspection algorithm model integrating multiple recognition capabilities; the multi-task computer vision model comprises a shared feature extraction backbone network and multiple independent task head networks; a training set is input into the shared feature extraction backbone network to extract general feature maps; the general feature maps are respectively input into the multiple independent task head networks for parallel training to obtain the safety inspection algorithm model; and the safety inspection algorithm model is loaded on a UAV flight platform to perform parallel inference analysis on inspection images collected by the UAV in real time, and to identify preset safety hidden danger targets.

The invention relates to a large language model segmentation hybrid reasoning method based on uncertainty driving, which comprises the following steps of: acquiring current text data and historical state characteristics to pre-estimate an uncertainty index of a current segment; according to the uncertainty index, minimizing the unified scheduling target function to obtain a target reasoning mode; performing reasoning calculation according to the target reasoning mode, and generating information contribution degrees corresponding to the key values; according to the uncertainty index and the information contribution degree, calculating a corresponding dynamic merging control probability, and performing weighted merging or pruning on the to-be-merged key value pairs to obtain compressed key value pairs; defining a deviation metric, and limiting the deviation metric not to exceed a preset upper bound determined by the dynamic merging control probability set and the uncertainty index; if so, triggering fallback processing; otherwise, the compressed key value pair state is fed back to the next segment for loop iteration until reasoning of all segments is completed; therefore, the video memory occupation and the reasoning delay are reduced on the premise of ensuring the precision.

The invention discloses a large model dialogue control method and system, a storage medium and computer equipment. The large model dialogue method comprises the steps of converting user input data and dialogue historical data into state perception vectors; selecting a plurality of expert models matched with the state sensing vector from a preset expert model pool; respectively inputting the user input data and the dialogue historical data into the selected expert models to obtain a plurality of candidate replies; and calculating a value score of each candidate reply, and outputting the candidate reply with the highest value score as a final dialogue reply. Through the above method, dialogue context semantics and user core intentions can be accurately captured, dynamic sparse activation and integration of knowledge of a multi-expert model can be realized, computing resource consumption and reasoning delay can be significantly reduced, dialogue reply accuracy and reliability and cross-style and cross-field adaptivity can be improved, and dialogue reply experience can be improved. And the process has relatively high interpretability and controllability, so that debugging and optimization of a large model are facilitated.

The application belongs to the technical field of intelligent robot control, and particularly relates to a self-adaptive step length control method for a live broadcast interactive robot; the method comprises the following steps: robot control instructions are recognized from a barrage text information, and the robot control instructions and a live broadcast scene image are input into a vision-language-action model for processing to obtain a candidate action sequence under current iteration; window action deviations and convergence slopes of action sites in the candidate action sequence are calculated in sequence; if the window action deviations and the convergence slopes of the first m action sites all satisfy a stable condition, and the (m+1)th action site does not satisfy the stable condition, the first m action sites are output; otherwise, the candidate action sequence is output again; the live broadcast interactive robot executes the first m action sites, and continues to process remaining action sites in the candidate action sequence until the entire robot control instruction is executed; the application can realize high-frequency and low-delay interaction while ensuring control stability, and is more suitable for a cost-sensitive and high-concurrency live broadcast environment.

The present disclosure relates to a layered parallel inference acceleration system, method and computer program product. The system comprises a text content analysis module for analyzing and processing user input text content to obtain a vector sequence; a candidate token sequence generation module for determining a corresponding set of to-be-predicted positions according to the current position of each of the plurality of vectors, and determining the feature information corresponding to each to-be-predicted position in the set of to-be-predicted positions; and predicting at least one candidate token according to the feature information, and combining each corresponding at least one candidate token in the set of to-be-predicted positions to generate a plurality of candidate token sequences; and a token sequence verification module for screening a candidate token sequence with the most number of candidate tokens that are continuously verified from the start position from the plurality of candidate token sequences as a target token sequence. In this way, the diversity and quality of the generated content can be ensured while reducing the model inference delay.

The application discloses a speech recognition fine-tuning method based on subspace decomposition and reorganization. The low-rank space of the LoRA module is equivalently decomposed into multiple subspaces, and learnable weights are added to different subspaces, so that the reorganization of all subspaces is realized. The reorganized subspaces can be transformed into an efficient calculation mode during training and an inference mode during inference, which significantly improves the inference and storage efficiency, and solves the problem of high inference cost during high concurrency request while ensuring efficient fine-tuning of parameters.

The application discloses an empty chip integrated with a physical information graph neural network acceleration core and a potential field regulation method thereof, and belongs to the technical field of quantum semiconductor integrated circuits and space field energy regulation. The empty chip comprises an electrode array layer, a quantum bit layer, a potential source topology mapping module, a GNN hardware acceleration core and a controller. The GNN hardware acceleration core is completely integrated in the chip and comprises a graph convolution operation unit, a physical constraint storage unit and a voltage mapping unit, and is used for hardwareizing GNN operation logic. The potential field regulation method comprises the following steps: the potential source topology mapping module constructs a potential source space topology graph; the GNN hardware acceleration core calls rules in the physical constraint storage unit, iteratively solves a global optimal energy distribution through hardware-level graph convolution operation, and converts the optimal energy value into a driving voltage loaded to the electrode array layer through the voltage mapping unit, so as to construct a dynamic adjustable nonlinear potential well. The application solidifies the neural network algorithm into a hardware IP core, avoids the risk of rejection of the intellectual activity rules of the Patent Law, realizes real-time self-adaptive closed-loop regulation of the space potential field, solves the energy instability problem caused by the space potential fluctuation, and is suitable for space passive energy acquisition and new propulsion scenarios.

The application relates to a cloud service workload prediction method and system based on a convolution enhanced Transformer, which comprises the following steps: collecting cloud server workload data and preprocessing the cloud server workload data to decompose the cloud server workload data into a trend component and a residual component; constructing a workload prediction model, training the workload prediction model by using the trend component and the residual component, and obtaining a trained workload prediction model; predicting the cloud server workload by using the trained workload prediction model to obtain a workload prediction value; predicting the trend component by using a trend information capturing module to obtain a prediction result of the trend component; predicting the residual component by using a convolution enhanced Transformer encoder module to obtain a prediction result of the residual component; and fusing the prediction result of the trend component and the prediction result of the residual component by using a feature fusion module to obtain a final workload prediction value.

This invention discloses a point cloud processing method and system based on spatial locality-preserving serialization and adaptive mesh reshaping. The method includes: first, acquiring raw 3D point cloud data and quantizing and sorting it into a one-dimensional locality-preserving sequence using spatial filling curves such as Morton codes; second, introducing an adaptive mesh reshaping strategy, dynamically configuring a 2D target canvas according to the number of input point clouds, using lossless rectangular mapping for small-scale point clouds, and employing a spatial continuous truncation mechanism to extract dense topological sub-blocks for large-scale scene point clouds; finally, inputting the generated 2D pseudo-image into a 2D attention network to extract features and outputting the processing result. This invention completely eliminates zero-filling artifacts in cross-modal mapping without relying on expensive 3D domain query operators, significantly improving the computational efficiency and feature representation accuracy of 3D point cloud data processing.

This invention discloses a decoder-free multi-branch attention table large-scale model and its reinforcement learning training method. Addressing the problems of redundant decoder structures, high computational overhead, and mismatch between training objectives and evaluation metrics in traditional supervised learning models, this invention proposes a multi-branch attention network based on an encoder-only architecture. This method removes the decoder portion from the traditional Transformer architecture, directly extracting feature interaction information using a multi-branch encoder. During the training phase, the Group Relative Policy Optimization (GRPO) algorithm is introduced for reinforcement learning training, abandoning the Critic network in traditional RL and directly calculating the relative advantage by sampling a set of outputs from the same input. This invention significantly reduces the number of model parameters and memory usage, while improving the accuracy and inference speed of table data processing by directly optimizing sequence-level rewards.

The invention relates to an electroplated metal quality real-time monitoring method based on deep learning, and belongs to the technical field of industrial automation and intelligent detection. Comprising the following steps: step 1, image acquisition and preprocessing; step 2, deep learning of a feature extraction and detection framework; step 3, carrying out integrated detection without NMS post-treatment; and step 4, defect classification and real-time feedback. According to the invention, through multi-scale feature fusion and large kernel convolution, the capturing capability of tiny defects is significantly improved, high detection precision can be maintained in complex illumination and strong reflection environments, and the method has the characteristics of high precision, high efficiency and high adaptability, and is suitable for various industrial scenes.

The invention relates to the technical field of data analysis, in particular to a palmprint recognition anti-fraud method based on double-branch self-supervised learning, and the method comprises the steps: constructing a double-branch self-supervised reconstruction architecture, determining a first to-be-processed image based on a high-frequency self-supervised branch, and determining a second to-be-processed image based on a chromaticity regularization self-supervised branch. Dividing the two images into patch blocks which are not overlapped with each other, independently generating random mask matrixes with different spatial distributions, determining visible areas, inputting the patch blocks of the visible areas into a shared encoder for feature extraction, outputting latent variable features, and splicing the latent variable features with learnable mask marks to obtain a patch matrix; and respectively sending to a high-frequency decoder and a chroma decoder, determining total loss, carrying out physical decoupling on illumination and material attributes, and carrying out fine adjustment on the architecture to finish convergence of the architecture. According to the method, the palm print features with discrimination are extracted through the shared encoder, and chromaticity distribution consistency constraints are introduced, so that effective decoupling of ambient light and real material attributes is realized.

The invention discloses a distributed speculation decoding system for edge equipment, and belongs to the technical field of large language model reasoning. The invention aims to solve the problems of high reasoning delay and load imbalance caused by static division and a traditional speculation scheme when a large language model is deployed on heterogeneous edge equipment in the prior art. The system comprises an off-line preprocessing module which is used for generating a model layer division strategy and an initial entropy triggering threshold value based on equipment resources; the online reasoning module is used for dynamically predicting the entropy of the middle layer of the model during decoding and determining a speculation triggering layer based on an entropy triggering threshold value dynamically adjusted according to the real-time equipment utilization rate; and transmitting a draft token on the trigger layer, and transmitting the draft token to downstream equipment for parallel verification. According to the method, the reasoning delay is effectively reduced, the utilization rate of heterogeneous equipment is improved, and the self-adaption to the dynamic load is realized.

This invention discloses a lightweight image super-resolution reconstruction method, system, and storage medium, relating to the field of image processing technology. Specifically, it includes: acquiring high-resolution and low-resolution images of the target as training sample pairs; constructing a lightweight image super-resolution reconstruction model comprising a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module performs shallow processing on the low-resolution image; the deep feature extraction module consists of multiple cascaded enhanced separable residual feature refinement blocks, extracting shallow features to obtain multiple deep features; the multi-layer feature fusion module is used to obtain fused features; and the image reconstruction module is used to output the high-resolution image. The trained model is used to reconstruct the low-resolution image to be processed. This invention has a simple network structure and achieves accurate reconstruction of image texture and details using only a low number of parameters and computational cost.

This invention provides a data scheduling method and neural network accelerator based on a reconfigurable computing array. The method includes: receiving a raw data stream and performing valid data detection on the raw data stream to generate a sparse mask sequence corresponding to the raw data stream. Valid data detection is used to remove invalid data from the raw data stream. Invalid data includes invalid padding data and / or zero values ​​or non-important data below a set threshold. Data scheduling is performed on the raw data stream based on the sparse mask sequence to obtain a valid data sequence in the raw data stream. The processing path of the valid data sequence on the reconfigurable computing array is determined based on the sparse mask sequence, and the valid data sequence is processed based on the processing path. This invention can eliminate computational power consumption for invalid data, reduce storage access, achieve zero-pause loading of irregular data streams, reduce inference latency, meet the high frame rate requirements of scenarios such as autonomous driving perception and real-time voice on the edge, and reduce power consumption.

The invention discloses a cloud edge cooperation system and method based on KubeEdge, relates to the technical field of edge computing, and realizes more accurate edge application scheduling by integrating multi-dimensional operation indexes on edge nodes. Reasoning delay and cloud cost are remarkably reduced through a cloud-side cooperative processing mechanism of a reasoning request; an enhanced edge node task management framework realizes observable task state, customizable process and failure self-healing; and a lightweight edge node group flow closed-loop mechanism is provided, and network isolation and cooperative configuration of cross-domain edge applications are simplified.

This invention discloses a drilling accident early warning method and apparatus. The method includes: acquiring multi-source drilling data; aligning the multi-source drilling data using a precise time synchronization protocol; extracting multi-modal temporal features to obtain temporal features of the multi-source drilling data; inputting the temporal features of the multi-source drilling data into a drilling accident occurrence probability generation model to output the drilling accident occurrence probability; the TLM model includes a Transformer encoder and a bidirectional long short-term memory network decoder; the Transformer encoder employs a probabilistic sparse self-attention mechanism; and triggering a drilling accident early warning based on the multi-source drilling data, the drilling accident occurrence probability, and a pre-configured physical rule base; the physical rule base includes drilling accident judgment logic, which can reduce the drilling accident false negative rate, shorten the drilling accident early warning inference delay, and improve the cross-well migration capability and accuracy of the drilling accident early warning.

The invention discloses an unmanned aerial vehicle group space structure regularity evaluation method based on brain-like calculation, and belongs to the technical field of unmanned aerial vehicle group space structure evaluation, and the method comprises the steps: calculating the optimal pose estimation of an unmanned aerial vehicle group under a global coordinate system, and extracting a global position set at a selected moment; constructing three variant matrixes of various distance deviations; calculating a correlation coefficient matrix and an autocorrelation value of each variant matrix; mapping the self-correlation value into structural regularity, and calculating confidence in combination with a consistency index; constructing a heterogeneous integrated learning framework based on a spiking neural network and a gradient boosting decision tree model, inputting the self-correlation value into the spiking neural network, inputting the self-correlation value and the structure regularity into the gradient boosting decision tree model, and executing a heterogeneous model fusion strategy to obtain an integrated evaluation result; through system performance verification and evaluation, a core output parameter set of the integrated system is generated, and high-precision global unified positioning in a denial environment is realized.