127results about How to "Improve reasoning efficiency" patented technology

The invention discloses a text data reasoning method and device based on a hybrid expert model, which are used for solving the technical problem of low reasoning efficiency caused by the existing reasoning process based on the hybrid expert model. The method comprises the steps of obtaining a pre-running text data set; inputting the pre-running text data set into a preset hybrid expert model in the multi-level heterogeneous memory space for pre-running reasoning, and determining hybrid expert module calculation transmission time and an expert correlation table; based on the expert correlation table and a preset number of prefetched hot experts, performing hot expert identification on a preset hybrid expert model, and determining a plurality of target hybrid expert modules; adopting a preset constraint perception planning inequality to calculate transmission time according to a preset number of prefetched hot experts and a hybrid expert module, and determining the batch number of input texts; and in the multi-level heterogeneous memory space, outputting a model reasoning result by adopting each target hybrid expert module according to the to-be-reasoned text data subset corresponding to the batch number of the input texts.

The invention provides a reasoning method and device, equipment and a medium. The method comprises the steps that when a reasoning request is received, model parameters of a target model deployed on each GPU are obtained, the reasoning request comprises information of a reasoning task, and the information of the reasoning task comprises input data; then, according to the structure information of the target model, segmenting the reasoning task into a plurality of groups of subtasks corresponding to the GPU according to a combination mode of a calculation task and a communication task; segmenting the input data into input sub-data corresponding to each group of sub-tasks; afterwards, generating a CUDA flow corresponding to the GPU according to a sequence dependency relationship between the calculation task and the communication task in each group of subtasks, the CUDA flow being used for realizing parallel execution of multiple groups of subtasks; then, a CUDA kernel on the GPU is scheduled according to the CUDA flow, so that the GPU executes multiple groups of subtasks in parallel according to model parameters and input subdata, and a reasoning result corresponding to the reasoning request is obtained. According to the invention, the reasoning time delay can be reduced.

The invention discloses a high-throughput large-model reasoning method, device and equipment based on a time separation type pipeline architecture and a storage medium, and relates to the technical field of large-model reasoning. The high-throughput large-model reasoning method based on the time separation type pipeline architecture comprises the steps that when a current reasoning stage is a pre-filling stage, the pre-filling stage is a pre-filling stage; performing pre-filling according to a client request, and determining the key value cache capacity of each request decision point; determining a stage switching opportunity according to the key value cache capacity of each request decision point and a preset memory capacity; switching the current reasoning stage from the pre-filling stage to a decoding stage according to the stage switching opportunity, and processing the client request according to a preset load balancing strategy to obtain a target load balancing result; and according to the target load balance result and a pipeline architecture, performing large model reasoning, and according to a reasoning result, obtaining an output text corresponding to the client request. And the reasoning efficiency of the high-throughput large model is improved.

The invention provides an edge-end large language model reasoning acceleration method and device based on core particle and near memory calculation, and the method comprises the steps: storing a model weight in Flash, and completing the operation of all matrix vector multiplication in the whole reasoning process through the cooperation of the Flash and NPU. And in the calculation part in the Flash, the NPU needs to send the input vector to the Flash, the weight matrix stored in the Flash and the input vector are used for matrix vector multiplication, and a result is obtained and sent back to the NPU. And the calculation part in the NPU is as follows: an input vector is stored in the NPU, a weight matrix needs to be read block by block from the Flash, and the operation is completed in the NPU. And the operation result is stored in the NPU. The attention operation is independently completed by the NPU. The special function calculation is independently completed by the NPU. And the NPU and the Flash respectively bear the task proportions, so that the NPU and the Flash can better collaboratively complete the reasoning task of the large language model.

A method and apparatus for wireless communication are provided. The method comprises the following steps: a first device receives a plurality of beams sent by a second device in an observation window, wherein the plurality of beams are used for determining a first beam set; the first device performs beam prediction on a second beam set in a prediction window based on the measurement value of the first beam set; the first device sends a first report to the second device; wherein the first report is used for determining one or K optimal beams in the second beam set, K is a positive integer, the first beam set is determined according to first information, and the first information comprises at least two of measurement values of the plurality of beams, capability of the first device and network configuration.

The invention discloses an avionics system anomaly detection method based on a dynamic mask diffusion model, which comprises the following steps of: 1, realizing initial reconstruction of a time sequence by adopting a self-adaptive dynamic mask strategy and utilizing information of adjacent nodes in a multivariable time sequence of an avionics system and information of abnormal nodes and adjacent nodes of the abnormal nodes; and 2, combining Transform with a denoising diffusion model, inferring an abnormal label by using a prediction error generated by the masking data in the reconstructed time sequence, and determining an abnormal detection result of the avionics system based on the abnormal label. The method is obviously superior to the most advanced method in the aspects of detection accuracy and timeliness.

The invention provides a context compression method based on query guidance. The context compression method comprises the following steps: filling query information and a context to be compressed into a dialogue template; respectively calculating first attention numerical values of a plurality of lexical elements in the context of the dialogue template; performing attention reconstruction on the context according to the first attention numerical values corresponding to the plurality of lexical elements; and compressing the context after the attention reconstruction to be within a preset target compression ratio. The invention further provides a context compression system based on query guidance, a storage medium and electronic equipment. Therefore, key information can be more efficiently reserved, a shorter context is input to the generative model, the reasoning efficiency of the generative model in a long context reasoning scene is improved, and the reasoning cost is reduced.

The invention discloses a knowledge point hybrid reasoning optimization method based on reverse deductive simulation. The method comprises the following steps: S1, obtaining a to-be-reasoned target knowledge point set; s2, generating a standardized knowledge point set; s3, constructing a causal reasoning model, and generating an initial causal network diagram; s4, bringing the potential causal relationship identified in the derivation process into a causal network diagram; s5, generating an optimized causal network diagram; s6, generating a causal network diagram with dynamically updated weights; s7, dynamically selecting an optimal reasoning path in a reasoning process, and adjusting a reasoning sequence in combination with weights and causal relationships of knowledge points; and S8, generating a final reasoning result which comprises the reasoning path of the target knowledge point set, the reasoning logic and the optimized causal network diagram, and outputting the final reasoning result for a user to use. According to the method, powerful technical support is provided for intelligent reasoning and optimization of the knowledge network.

The invention provides a task scheduling method of a large language model, a data processing system and electronic equipment. The method comprises the following steps: on a first processor, obtaining a request from a ready queue to form a first running instance and a second running instance; generating a first coroutine corresponding to the first operation instance and a second coroutine corresponding to the second operation instance, wherein the first coroutine and the second coroutine are both used for executing calculation tasks and transmission tasks of Q sub-stages; running a first coroutine to submit the 1i-th calculation task to a first workflow of a second processor and submit the 1i-th transmission task to a second workflow of the second processor, and setting the 1i-th transmission task to be executed after the 1i-th calculation task is executed; and operating the second coroutine to submit the 2i-th calculation task to the first workflow and submit the 2i-th transmission task to the second workflow, and setting the execution condition of the 2i-th transmission task to be executed after the 2i-th calculation task is executed. According to the embodiment of the invention, the reasoning efficiency of a large model can be improved.

The invention discloses an infrared image deblurring method and system for an unmanned aerial vehicle platform, and the method comprises the steps: shooting a clear image set and a motion video of a target region through an unmanned aerial vehicle, and obtaining gyroscope information during the shooting of the motion video by the unmanned aerial vehicle; according to the gyroscope information, combining with the residual error connection network structure model to obtain a pixel distribution intensity graph; obtaining scene ambiguity information according to the motion video; obtaining a first data set according to the clear image set and the scene ambiguity information; obtaining a training set and a verification set according to the first data set and the pixel distribution intensity graph; training the first model according to the training set to obtain a second model set; the first model comprises a multi-scale encoder-decoder network model used for removing image blurring; and first indexes of all second models are calculated according to the verification set and the second model set, and the second model with the maximum first index result is selected to perform unmanned aerial vehicle real-time infrared image deblurring.

The invention discloses a data reasoning method, system and device and a storage medium, and the method comprises the steps that a master node generates a query vector, a key vector and a value vector based on an input lexical element of a current layer of a Transform model, and adds the key vector and the value vector to a local key value cache of the master node; the master node broadcasts query vectors to a plurality of slave nodes through network equipment; the plurality of nodes perform attention calculation based on the query vector and the local key value caches of the plurality of nodes to obtain initial attention vectors corresponding to the plurality of nodes; the plurality of slave nodes send respective initial attention vectors to the master node through the network device; the network device performs aggregation processing on the initial attention vectors corresponding to the plurality of slave nodes to obtain a target attention vector; and the main node performs model reasoning based on the target attention vector through a Transform model to obtain an output lexical element. Therefore, the reasoning time can be reduced, and the network bandwidth requirement is reduced.

The invention relates to the technical field of computers, and discloses a model quantification method and device, a storage medium and a program product. The method comprises the following steps: in an off-line quantization stage, calculating to obtain a leveling coefficient according to an input matrix and a weight matrix of each linear layer operator, and calculating to obtain an input quantization parameter and a weight quantization parameter according to the leveling coefficient; performing 8-bit floating point quantization, grouping and 4-bit unsigned shaping asymmetric quantization on the weight matrix according to the weight quantization parameter to obtain a scaling coefficient, a zero reference value and a 4-bit unsigned shaping quantization weight; and in the online reasoning stage, performing online reasoning on the reasoning task based on the leveling coefficient, the input quantization parameter, the weight quantization parameter, the scaling coefficient, the zero reference value and the 4-bit unsigned shaping quantization weight to obtain a reasoning result. According to the scheme of the embodiment, the hardware memory occupied by the large language model can be reduced, the model precision loss can be reduced, and the model reasoning efficiency can be improved.

The invention provides a model reasoning method, system and device and electronic equipment, and is applied to the technical field of computers.The method comprises the steps that a target question is input into a pre-trained abstract tree generation model, an abstract tree which is generated by the abstract tree generation model and comprises multiple levels of abstracts is obtained, and the multiple levels of abstracts comprise multiple leaf node abstracts; the target problem and the multiple leaf node abstracts are spliced and then input into an end-side large model, and a key value KV matrix is calculated through the end-side large model; and respectively combining the target problem with each leaf node abstract in the plurality of leaf node abstracts to form a multi-section reasoning context, inputting the multi-section reasoning context into an end-side large model, and performing parallel reasoning on the multi-section reasoning context by combining the end-side large model with a model intermediate result to obtain a target reasoning result. The model reasoning method improves the model reasoning efficiency.

The invention discloses a big language model reasoning optimization method, system and device and a storage medium, which are corresponding schemes, in the scheme, a data clustering and quantification mechanism is introduced in a big language model reasoning process, when the size of a KV cache exceeds a preset threshold value, firstly, KV cache data is subjected to clustering division, and when the size of the KV cache exceeds the preset threshold value, the KV cache data is subjected to clustering division; and performing low-precision quantization processing on part of data, and replacing the other part of data with a clustering center, thereby achieving the purpose of reducing the KV cache size. Through the scheme of the invention, the video memory occupation in the reasoning process can be obviously reduced, the reasoning efficiency is improved, and the utilization rate of system resources is optimized.

The invention provides a vocoder speech synthesis processing method and device based on low resources and related equipment, and aims to solve the problem of low speech synthesis efficiency of a vocoder under the condition of low resources in the prior art. The method comprises the following steps: acquiring T frames of acoustic features, determining first target features corresponding to features including but not limited to vertical correlation between the T frames of acoustic features, performing convolution on the first target features based on respective corresponding channels, and determining features including but not limited to correlation between different channels of the acoustic features included in the first target features to obtain second target features; and determining a sampling point corresponding to the acoustic feature of each frame of voice signal according to the second target feature to obtain a target sampling point, and finally reconstructing the target sampling point into a voice waveform to obtain a target voice, so that the voice synthesis effect can be ensured, the voice synthesis efficiency of the vocoder under the condition of low resources can be improved, and the user experience can be improved. The technical scheme can be applied to the technical field of financial science and technology.

The invention relates to a data processing method and device based on a hybrid expert model, computer equipment, a computer readable storage medium and a computer program product. The method comprises: based on a target character in input data, determining a target probability corresponding to the target character, and determining a target probability difference threshold corresponding to the target character, the target probability comprising a probability estimation value of each expert network model in a preset hybrid expert model; screening the expert network models based on the probability estimation values of the expert network models and a target probability difference threshold to obtain a target expert network model; and processing the target character based on each target expert network model to obtain a data processing result. By adopting the method, the models contained in the expert model can be dynamically mixed based on the probability difference threshold, the data processing of the character dimension of the input data is realized, and the adaptation degree between each dynamically determined expert network model and the characters in the input data is ensured.

The invention relates to the technical field of image processing, and discloses a memory allocation method, device and equipment and a storage medium, the method comprises the following steps: in response to a data processing request, performing model reasoning on to-be-processed data through a neural network model, when a current operator is a static output operator, obtaining a static sub-network to which the current operator belongs, calculating an optimal memory size corresponding to the static sub-network, and allocating a memory block for the current operator according to the optimal memory size; when the current operator is the static output operator, the static sub-network to which the current operator belongs is firstly obtained, and then the static sub-network is used as the allocation object to apply for the minimum memory meeting the memory requirements of the plurality of static output operators from the memory pool; therefore, the memory can be applied for all the static output operators in the static sub-network only through one-time application, the memory allocation frequency is reduced, memory use is reduced, and the model reasoning efficiency is improved.

The invention provides a TTS system, a speech synthesis method and device, electronic equipment and a storage medium. The speech synthesis method comprises the following steps: acquiring a phoneme sequence corresponding to a target text; the condition features of the reference speech are extracted, the reference speech is processed into discretized coded representation, and the condition features comprise acoustic features and / or semantic features; inputting the phoneme sequence, the condition feature and the coded representation into a pre-trained speech synthesis model, and obtaining target speech which is output by the speech synthesis model and matched with the target text; wherein the speech synthesis model comprises a neural coding and decoding language model and an audio decoder connected with the neural coding and decoding language model, and the neural coding and decoding language model comprises an autoregression module and does not comprise a non-autoregression module.

A multi-hop reasoning method based on dynamic reasoning guidance and multistage self-feedback retrieval belongs to the field of natural language processing, and comprises the following steps: deconstructing a multi-hop reasoning process into a target-oriented sequence decision problem, carrying out dynamic reasoning guidance by using a large language model, generating a sub-problem sequence matched with a reasoning progress in real time, and carrying out multi-level self-feedback retrieval on the sub-problem sequence; target document retrieval is guided, and sub-questions are dynamically generated; according to the generated sub-questions, obtaining associated documents by adopting a three-level collaborative retrieval mechanism; and performing information refining on the associated document through a large language model, fusing the refined information into an inference chain, and performing inference to generate an answer. The invention further discloses a multi-hop reasoning system, a storage medium and a computer program product. The method aims at solving the complex multi-hop problem that multiple dispersed knowledge fragments need to be integrated, high-accuracy and high-efficiency reasoning is achieved, the retrieval requirement is dynamically generated through an explicit thinking chain guiding mechanism, and evidence obtaining is optimized and redundant information is filtered in combination with a three-level self-feedback retrieval mechanism.

The invention discloses a Tranform architecture operator hybrid optimization method. The Tranform architecture operator hybrid optimization method comprises the following steps: step 1, operator mapping analysis and optimization demand identification; 2, the operation process of the operators is disassembled, and the calculation kernel of each operator is constructed by using OpenCL; and 3, taking an OpenCL kernel as a computational graph node, and constructing a computational graph by using OpenVX. According to the method, the OpenVX provides a powerful graphic processing optimization capability, a computational graph can be optimized into an efficient execution form, and the OpenCL accelerates the execution of operators by utilizing a multi-core parallel computing capability, so that the overall reasoning efficiency is improved.

The invention discloses a self-adaptive multi-level forged voice detection method based on a belief propagation mechanism. The method comprises the following steps: performing feature extraction on voice to be detected to obtain an initial feature vector; reasoning the initial feature vector by using a lightweight model to obtain an initial confidence coefficient representing the voice forgery possibility; based on a low-confidence threshold value and a high-confidence threshold value which are determined through performance optimization, performing multi-level discrimination on the input voice and screening difficult samples; generating expert feature vectors for the difficult samples by adopting a depth feature extraction network regulated and controlled by the initial confidence coefficient; constructing a query vector based on the initial confidence, and fusing expert features through an attention mechanism; and voice authenticity determination is completed according to the fusion features. The initial confidence is used as the cross-stage control quantity, so that the self-adaptive adjustment of the detection process is realized, and the average reasoning delay is reduced while the detection accuracy is ensured.

The invention provides a model reasoning method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. Inputting the input vector into a target model to obtain a model reasoning result corresponding to the input vector output by the target model; wherein the target model comprises at least one hybrid expert module; the hybrid expert module comprises a time sequence feature extraction layer, a routing selection layer, an expert model layer and a result output layer which are connected in sequence; the expert model layer comprises a plurality of state space models; the input vector is generated based on at least one of text, speech, image, or video. According to the method and the device provided by the invention, the storage consumption of the large model is reduced, the access frequency of the large model to the storage is reduced, and the reasoning efficiency of the large model is improved.

The embodiment of the invention relates to the technical field of data processing, in particular to a model reasoning method, device and equipment based on function encryption. The user side configures public parameters according to the security parameters and the classification set, generates a master key according to the public parameters and sends the master key to the data side and the model side; the model party judges a target function of the user party after receiving the master key sent by the user party, generates a model key according to the master key and the target function, and sends the model key to the user party; after the data party receives the master key sent by the user party, the demand of the user party is judged, a ciphertext is generated according to the master key and the demand and sent to the user party, and finally the user party obtains a classification result in the classification set according to the model key and the ciphertext. Therefore, the learning model of the model party and the data of the data party are respectively encrypted and respectively sent to the user party, and the model party and the data party only need to provide the ciphertext and the secret key without participating in the execution process.

The invention discloses an index tensor vectorization calculation method and a related product. The method comprises the following steps: acquiring the length of a first index tensor of a first sparse tensor and the length of a second index tensor of a second sparse tensor; based on the length of the first index tensor, obtaining at least one first target instruction corresponding to the first index tensor from an instruction set; obtaining at least one second target instruction corresponding to the second index tensor from the instruction set based on the length of the second index tensor; and executing the at least one first target instruction and the at least one second target instruction to perform vectorization calculation on the first index tensor and the second index tensor to obtain a third index tensor. According to the invention, the computing resources of the computing units can be saved and the splicing efficiency of the sparse tensor can be improved.

The application discloses a line character fusion character string recognition method and system for ancient book optical character recognition, and belongs to the technical field of ancient book digitization. The method takes an ancient book character string as a recognition unit, adopts a transformer to model a global context, triggers a single character expert branch through a confidence degree guiding mechanism, and performs fine-grained verification on low-confidence difficult characters. A visual-semantic-character shape joint fusion strategy is combined to complete candidate reordering, and the recognition efficiency of high-frequency characters, the recognition accuracy of long-tail characters, variant characters and complex character shapes are taken into account. The system comprises visual coding, line-level recognition, single character verification and candidate fusion modules, can effectively solve the recognition bottleneck of large character sets and long-tail distribution of ancient books, improve the fidelity and reasoning efficiency of variant characters, greatly reduce the manual correction cost of ancient book digitization, and is suitable for large-scale ancient book digitization scenarios.

This invention discloses a request processing method, apparatus, device, and storage medium, comprising: when the matching result is determined to be a matching failure, inferring the user request using a synchronous direct transmission mode through the target instance to obtain a first key-value cache, and determining the storage method of the first key-value cache based on its cost; when the matching result is determined to be a match with the target key-value cache, inferring the user request using a prefetch incremental mode through the target instance to obtain a second key-value cache, and directly saving the second key-value cache to a storage center. In the synchronous direct transmission mode, the cost of the key-value cache obtained through user request inference is calculated, and key-value caches that meet the cost requirements are saved to the storage center, avoiding invalid space occupation in the storage center and improving the inference efficiency of subsequent user requests. In the prefetch incremental mode, when a key-value cache hit meets the cost requirements, it is directly used to improve inference efficiency and avoid redundant calculations and waste of computing resources.

The invention discloses a lightweight embedded implementation mechanism of a Rogowski coil inter-phase interference compensation MFNN method, and the mechanism comprises the steps: training an MFNN model, and achieving the Rogowski coil inter-phase interference compensation; floating-point number parameters of the MFNN model are lightened, and memory occupation and calculation amount of the model are reduced; reproducing an MFNN model structure by using a C language and realizing parameter loading, and reproducing an inference result of the MFNN model under the Pytorch framework; mFNN model reasoning based on the C language is optimized through parallel computing; and deploying a C language-based parallel computing MFNN model in an edge computing environment to realize the real-time compensation of the inter-phase interference of the Rogowski coil.

The embodiment of the specification provides a model training method and platform, and a text reasoning method and platform, wherein the model training method comprises the following steps: obtaining an initial sample set, a student reasoning model and a reasoning ability label of the student reasoning model, wherein the initial sample set comprises a plurality of sample thinking chains, the plurality of sample thinking chains are generated by using different reasoning methods by a teacher reasoning model, each sample thinking chain is labeled with a reasoning method label of a corresponding reasoning method, a sampling weight of the plurality of sample thinking chains is determined based on a deviation between the reasoning ability label of the student reasoning model and the reasoning method labels labeled by the plurality of sample thinking chains; a target sample set is obtained by sampling from the initial sample set based on the sampling weight; and the student reasoning model is trained based on the target sample set to obtain a target reasoning model. The target reasoning model obtained by the training method can dynamically adjust the reasoning process according to the complexity of the problem, and can ensure the reasoning accuracy while improving the reasoning efficiency.

The application discloses a scalable speaker acoustic model implementation method and device. The method comprises the following steps: in response to a plurality of target new speaker voice synthesis requests, determining the target speaker identifier of each target new speaker in the voice synthesis request; determining the target embedding vector of the target new speaker corresponding to the target speaker identifier in the preset speaker index table; determining the target adapter parameter in the target acoustic model based on the target embedding vector of the plurality of target new speakers, and recombining the target adapter parameter and the target acoustic model; and responding to the voice synthesis request of the plurality of target speakers through the combined target acoustic model. The application solves the technical problem of low inference processing efficiency caused by sequentially inferring the voice text of each speaker by calling the adaptive model parameters of each speaker when the related acoustic model performs voice synthesis on the batch input voice text of multiple speakers.

The present invention provides a computing resource allocation system for improving model reasoning efficiency, which relates to the field of electronic digital data processing, and includes a task rating module, an inference management module, an inference monitoring module and a resource allocation module. The task rating module is used to perform priority rating on tasks in the model, the inference management module is used to control and manage the execution of tasks, the inference monitoring module is used to monitor the execution process of tasks, and the resource allocation module is used to allocate computing cores and video memories to tasks. The system rates tasks in the reasoning model and allocates computing resources based on the rating results, so as to effectively control the processing speed of different tasks, thereby improving the reasoning efficiency of the entire reasoning model.