Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

6798 results about "Multiplexing" patented technology

In telecommunications and computer networks, multiplexing (sometimes contracted to muxing) is a method by which multiple analog or digital signals are combined into one signal over a shared medium. The aim is to share a scarce resource. For example, in telecommunications, several telephone calls may be carried using one wire. Multiplexing originated in telegraphy in the 1870s, and is now widely applied in communications. In telephony, George Owen Squier is credited with the development of telephone carrier multiplexing in 1910.

Large language model reasoning acceleration method and system based on dynamic video memory compression and memory isomerism

The invention discloses a big language model reasoning optimization method and system based on dynamic video memory compression and memory isomerism, and intelligent management of video memory resources is realized by integrating a dynamic compression strategy of KV Cache and a memory parallel architecture. The method comprises the following steps: 1) analyzing the spatial-temporal characteristics of the KV Cache in real time, adaptively selecting a quantization compression algorithm, a rarefaction algorithm or a low-rank decomposition algorithm, performing hierarchical storage based on attention head importance scores, keeping high precision of a core head, and implementing low-bit quantization on a secondary head; (2) the compressed inactive data are divided into a plurality of data blocks to be stored in a system memory, a parallel data channel group is established according to the number of physical channels, the compressed blocks are concurrently read through multiple channels during loading, and parallel decompression of a sparse matrix is accelerated through a GPU tensor core; and 3) constructing a KV Cache multiplexing mechanism and a parallel channel, and parallelizing a compression / decompression process and model calculation by adopting a hardware acceleration compression and asynchronous pipeline mechanism.
Owner:HANGZHOU AMTD YINGANG DIGITAL TECH CO LTD

Spatially encoded biological assays

The present invention provides assays and assay systems for use in spatially encoded biological assays. The invention provides an assay system comprising an assay capable of high levels of multiplexing where reagents are provided to a biological sample in defined spatial patterns; instrumentation capable of controlled delivery of reagents according to the spatial patterns; and a decoding scheme providing a readout that is digital in nature.
Owner:PROGNOSYS BIOSCIENCES INC

Spatially encoded biological assays

The present invention provides assays and assay systems for use in spatially encoded biological assays. The invention provides an assay system comprising an assay capable of high levels of multiplexing where reagents are provided to a biological sample in defined spatial patterns; instrumentation capable of controlled delivery of reagents according to the spatial patterns; and a decoding scheme providing a readout that is digital in nature.
Owner:PROGNOSYS BIOSCIENCES INC

Image-fused end-side cloud collaborative intelligent fire-fighting fire monitoring system

The invention discloses an end-side cloud collaborative intelligent fire-fighting fire monitoring system based on image fusion, and relates to the technical field of intelligent fire-fighting, the system is composed of a plurality of functional modules, and the system comprises a multi-modal image fusion module which generates a dynamic scanning priority map based on prior data, distinguishes a natural heat source from an abnormal fire by using a dual-light fusion algorithm, and sends an image fusion result to a cloud server; a scanning area is divided according to the thermal risk grade, and the thermal imaging resolution is dynamically adjusted; the distributed edge computing module is used for carrying out space-time synchronization on cross-modal data through a multi-modal feature alignment network, and carrying out dynamic allocation on a CUDA core and CPU resources through adaptive computing scheduling; an improved artificial bee colony algorithm is adopted, the bandwidth of the multi-sensor data flow is dynamically allocated through a time-sharing multiplexing protocol, and three-dimensional path planning is carried out; and the end-side cloud collaborative decision module constructs a federated learning driven model sharing network, and each edge node trains a lightweight YOLOv5s pruning model based on local data.
Owner:HANGZHOU ZIPENG TECH CO LTD

Spatially encoded biological assays

The present invention provides assays and assay systems for use in spatially encoded biological assays. The invention provides an assay system comprising an assay capable of high levels of multiplexing where reagents are provided to a biological sample in defined spatial patterns; instrumentation capable of controlled delivery of reagents according to the spatial patterns; and a decoding scheme providing a readout that is digital in nature.
Owner:PROGNOSYS BIOSCIENCES INC

Method and device for transmitting or receiving channel state information in communication system

An embodiment of an operating method of a first device may comprise the steps of: receiving, from a second device, a first report setting configured to perform reporting of first channel state information (CSI); receiving, from the second device, a first medium access control (MAC) control element (CE) that configures one or more multiplexing modes for the first device in at least a first time resource; generating the first CSI on the basis of the first report setting and the first MAC CE; and transmitting the first CSI to the second device, wherein the first time resource configures a first CSI reference resource, configured by the first report setting.
Owner:ELECTRONICS & TELECOMM RES INST

Long-distance communication optical cable fault automatic detection and positioning system and method

The invention discloses a long-distance communication optical cable fault automatic detection and positioning system and method, and the method comprises the steps: collecting optical signal data in an optical fiber link in real time through the deployment of a multi-optical fiber sensor or a distributed optical fiber sensing technology; denoising processing is carried out on the collected data, abnormal fluctuation is analyzed by applying a self-adaptive algorithm, and the type of a fault which is about to occur is identified; classifying the faults by using a machine learning classification algorithm; time domain reflection and wavelength multiplexing technologies are combined, a composite algorithm is adopted to accurately position a fault point, and path optimization and error correction are carried out through sensing node data comprehensive analysis; an environment compensation algorithm is introduced, influences of environment factors on the optical signals are analyzed in real time, and errors are eliminated; and based on a fault positioning and analysis result, an AI algorithm is adopted to evaluate a fault influence range, a repair scheme is automatically generated, and priorities are scheduled for repair.
Owner:GUANGZHOU JINGLEI COMMUNICATION TECHNOLOGY CO LTD

Spatially encoded biological assays

The present invention provides assays and assay systems for use in spatially encoded biological assays. The invention provides an assay system comprising an assay capable of high levels of multiplexing where reagents are provided to a biological sample in defined spatial patterns; instrumentation capable of controlled delivery of reagents according to the spatial patterns; and a decoding scheme providing a readout that is digital in nature.
Owner:PROGNOSYS BIOSCIENCES INC

Document question and answer processing method and system, electronic equipment, storage medium and computer program product

The invention discloses a document question and answer processing method and system, electronic equipment, a storage medium and a computer program product, and relates to the technical field of large model technologies and document question and answer technologies. The method comprises the following steps: according to an input question, obtaining a recalled document and a key value cache corresponding to the recalled document; generating an initial text sequence by using the input question and the recalled document; based on a mixed key value strategy, key value filling calculation is carried out on the initial text sequence, a target text sequence is obtained, the mixed key value strategy is used for representing key value confirmation mechanisms corresponding to a plurality of lexical elements in different model layers, and the key value confirmation mechanisms comprise one of a key value multiplexing mechanism based on key value cache and a key value reusing mechanism based on key value cache; a hybrid mechanism of a key value multiplexing mechanism and a key value calculation mechanism; and performing reasoning calculation by using the target text sequence to obtain a target answer. The technical problem that a model is difficult to consider the reasoning speed and the reasoning precision due to a calculation mode completely depending on key value multiplexing in related technologies is solved.
Owner:ALIBABA CLOUD FEITIAN (HANGZHOU) CLOUD COMPUTING TECH CO LTD

Electric power infrastructure field operation environment data monitoring and safety management method

The invention discloses an electric power capital construction site operation environment data monitoring and safety management method, which belongs to the field of intelligent decision technology and electric power safety management, and comprises the following steps: constructing a semantic network framework according to a construction plan; collecting and calibrating multi-source environment data to generate a trusted data set; generating a real-time risk network graph based on the semantic framework and the trusted data set; calculating a robust risk index and performing sensitivity deconstruction; generating a closed-loop intervention instruction when the risk indicator exceeds a safety threshold; and finally, collecting, feeding back, iteratively optimizing the whole system, and generating a cross-project multiplexing intelligent template library. According to the method, a comprehensive technical path of semantic modeling, causal inference and closed-loop adaptive optimization is adopted, the operation situation can be deeply analyzed, potential risks can be quantified and attributed prospectively, the optimal intervention strategy is intelligently generated, and the intelligence, precision and prospective level of safety management of the electric power capital construction site is remarkably improved.
Owner:BEIJING HUALIAN POWER ENG SUPERVISION CO +2

Deep learning training and reasoning task dynamic cooperation system based on GPU space-time resource sharing

The invention provides a deep learning training and reasoning task dynamic cooperation system based on GPU space-time resource sharing, and the system comprises a GPU resource state perceptron which is used for monitoring a GPU kernel function call sequence and a video memory distribution state of a distributed training task in real time, dynamically capturing a calculation gap and video memory fragments generated by the training task, and transmitting the calculation gap and the video memory fragments to the GPU resource state perceptron; generating a two-dimensional resource spatial-temporal characteristic spectrum, and predicting a GPU calculation idle period caused by communication synchronization based on an LSTM model; the kernel function dynamic scheduling decision maker is used for performing priority division and dynamic resource quota allocation on an online reasoning task and an offline reasoning task by adopting a self-adaptive allocation strategy on the basis of a resource spatial-temporal characteristic spectrum so as to realize spatial-temporal resource decoupling of the training task and the reasoning task; and the kernel function execution arbiter is used for dynamically controlling submission and blockage of the reasoning task kernel function according to the scheduling decision through video memory space multiplexing and a calculation instruction arbitration mechanism. According to the method, the GPU resource utilization rate is remarkably improved, on the premise that stable training task performance is guaranteed, fragmented resources are effectively utilized to support parallel execution of multiple types of reasoning tasks, and efficient resource collaboration of a deep learning task cluster is achieved.
Owner:NANJING INFORMATION HIGH-SPEED RAILWAY RES INST OF SCI AND TECH

Information retrieval intelligent knowledge service electronic equipment

The invention discloses intelligent knowledge service electronic equipment for information retrieval, and the equipment comprises a data management layer which is responsible for providing original data for a cognitive calculation layer, and carrying out the closed-loop optimization through continuously monitoring the feedback of a man-machine interaction layer; the cognitive calculation layer is responsible for converting original data into a semantic tag system and a dynamic knowledge graph, applying BERT + GNN hybrid analysis, BERT processing text context and GNN analysis entity relationship in the knowledge graph, and dynamically adjusting a reasoning path according to user field preference by combining an SWRL rule engine and reinforcement learning; the service arrangement layer is responsible for extracting data from the multi-modal data pool, solidifying and arranging a complex query stream to perform logic decoupling and efficient multiplexing, and providing a visual interface to support non-technical personnel to configure a new scene; and the man-machine interaction layer is used for constructing a natural interaction bridge through the structured answers and the visual maps output by the service arrangement layer, carrying out personalized interface adaptation based on historical dialogues, and establishing a user feedback closed loop.
Owner:ZHENGZHOU UNIVERSITY OF LIGHT INDUSTRY

Demodulation reference signal design for large sub-carrier spacing

Various aspects of the present disclosure generally relate to wireless communication. In some aspects, a user equipment may receive a demodulation reference signal (DMRS) configuration that indicates a code-division multiplexing (CDM) group configured for the UE; receive a first indication of an antenna port, included in the CDM group, assigned to the UE for reception of a first set of DMRS transmissions; receive a second indication of whether a second set of DMRS transmissions for another antenna port, included in the CDM group, is present or absent for another UE; and perform channel estimation based at least in part on the first indication of the antenna port and the second indication of whether the second set of DMRS transmissions for the other antenna port is present or absent. Numerous other aspects are provided.
Owner:QUALCOMM INC

Terminal, radio communication method, and base station

A terminal according to one aspect of the present disclosure includes a receiving section that receives a configuration of a demodulation reference signal (DMRS), and a control section that controls transmission and reception of the DMRS, based on one or more associations between a plurality of code division multiplexing (CDM) groups and a plurality of DMRS ports and the configuration. The number of the plurality of DMRS ports for DMRS configuration type 1 is larger than 8, and the number of the plurality of DMRS ports for DMRS configuration type 2 is larger than 12. The plurality of DMRS ports include a first plurality of DMRS ports and a second plurality of DMRS ports. According to one aspect of the present disclosure, an appropriate number of DMRS ports can be used.
Owner:NTT DOCOMO INC

Techniques for multi-codeword communications using CDM group-based DMRS port mapping

Methods, systems, and devices for wireless communications are described. A user equipment (UE) may receive first control signaling indicating a demodulation reference signal (DMRS) port mapping configuration for a set of antenna port field values. For each antenna port field value, the DMRS port mapping configuration may include a first mapping between a first codeword corresponding to a first transmission-reception point (TRP) and a first code-division multiplexing (CDM) group, and a second mapping between a second codeword corresponding to a second TRP and a second CDM group. The UE may receive second control signaling that indicates an antenna port field value from the set of antenna port field values. The UE may then receive, based on the antenna port field value, first and second DMRSs including the first and second codewords, respectively, via first and second sets of DMRS ports corresponding to the first and second CDM groups, respectively.
Owner:QUALCOMM INC

Non-technician-oriented script intelligent adaptation and visual arrangement method and system

The invention provides a non-technician-oriented script intelligent adaptation and visual arrangement method and a non-technician-oriented script intelligent adaptation and visual arrangement system. The method comprises the following steps: acquiring a script file uploaded by a user, extracting a dependency library through an AST analysis technology, and constructing an isolation container environment; performing intelligent identification and conversion on a file path and database connection in the script to generate a standardized API call; carrying out input and output analysis on the basis of the converted script, and carrying out DAG workflow arrangement in a visual mode; performing function-level packaging and standardization processing on the script component in the workflow to realize component multiplexing and version management; and finally, configuring a scheduling strategy and carrying out automatic operation monitoring. According to the method, a script environment depends on automatic processing and intelligent path adaptation, and visual arrangement and modular multiplexing are combined, so that non-technical personnel can independently complete the whole process from AI script generation to a production-level automatic process, the problem of'last kilometer 'in script deployment is solved, and the working efficiency is improved.
Owner:BEIJING YULORE INNOVATION TECH

Battery equalization management method and device based on port multiplexing

The embodiment of the invention discloses a battery equalization management method and device based on port multiplexing. The method comprises the following steps: calculating a state-of-charge value of each single battery according to a sampling value of an electrical basic physical quantity; calculating a difference value between the state-of-charge value of each single battery and an average state-of-charge value of the battery pack to which the single battery belongs; determining the single batteries of which the difference values exceed a preset equalization threshold value as unbalanced batteries; and formulating an equalization sequence of the unbalanced batteries, and conducting the selection switch and the multiplexing switch according to the equalization sequence, so that the unbalanced batteries are connected with a primary side port of the multi-port converter for energy adjustment. According to the embodiment of the invention, the hardware complexity and the system cost are reduced by greatly reducing the usage amount of switch components. Meanwhile, the framework also improves the equalization efficiency and flexibility of the retired battery pack.
Owner:HEFEI UNIV OF TECH

Software upgrading method and system of Internet of Things terminal

The invention relates to the technical field of software upgrading, in particular to a software upgrading method and system for an Internet of Things terminal. The method comprises the following steps: acquiring software upgrade package information and terminal operation data, performing version verification, and generating terminal version difference data so as to determine software upgrade tasks, performing channel allocation on the tasks, distinguishing independent software tasks from dependent software tasks, performing chain type sorting on the dependent tasks, and generating terminal version difference data; and the head task is upgraded based on a multiplexing scheme, and meanwhile, the subsequent task and the independent software task are preloaded for parallel upgrading, and the dependent task and the independent task are synchronously scheduled, so that efficient Internet of Things terminal software upgrading is realized. The software upgrading method for the Internet of Things terminal is more efficient and more flexible.
Owner:金华兴浩信息咨询有限公司

Method of transmitting BSM message of V2X communication device provided in vehicle in autonomous driving system

Disclosed herein is a method of transmitting a BSM message of V2X communication device provided in a vehicle in an autonomous driving system. The method of transmitting a BSM message of a V2X communication device according to an embodiment of the present invention may multiplex sensor data and GPS data received from in-vehicle sensors, respectively, and use the multiplexed sensor data to generate BSM data. As a result, it is possible to enhance stability of the V2X system and more efficiently correct errors or the like occurring between sensors.An autonomous vehicle according to the present invention may be linked with an artificial intelligence module, a drone (unmanned aerial vehicle (UAV)), a robot, an augmented reality (AR) device, a virtual reality (VR) device, devices related to 5G services and the like.
Owner:LG ELECTRONICS INC

Infection risk assessment method and system for burn nursing

The invention discloses an infection risk assessment method and system for burn nursing, and the system comprises a long-term trend analysis module and a short-term anomaly detection module, the long-term trend analysis module employs an improved Transform-XL model, and captures the chronic cumulative effect of inflammation indexes within 72 hours through a relative position coding and memory multiplexing mechanism; the short-term anomaly module is based on a bidirectional LSTM reconstruction network, combines a dynamic threshold algorithm to identify sudden physiological anomaly in a window of 6 hours in real time, generates a final infection risk index through a confidence-driven adaptive fusion algorithm and dynamic weighted dual-channel output, triggers clinical response in combination with a hierarchical early warning mechanism, and introduces closed-loop feedback optimization, so as to achieve the early warning of the physiological anomaly. The actual intervention effect feedback model is iterated, the infection recognition time is shortened to 3.2 hours, the ROC-AUC reaches 0.94, and the reasonable utilization rate of antibiotics is increased to 89%.
Owner:AFFILIATED HOSPITAL OF JIANGNAN UNIV

File fragmentation encryption transmission method and device based on B / S (Browser / Server) architecture

The invention discloses a file fragment encryption transmission method and device based on a B / S architecture, and belongs to the field of information security. The problem of systematic contradiction among file transmission efficiency, safety and reliability is solved. The method comprises the following steps: determining a fragmentation threshold value, and carrying out self-adaptive fragmentation processing on an original file; respectively performing RSA public key encryption and AES random key secondary encryption on each fragment file to generate a double-encrypted file; writing the SHA-3 hash value of each fragment file and the timestamp information into a block chain network to generate a non-tampering meta-file; a multiplexing transmission channel is established, a meta-file and an encrypted file set are downloaded at the same time, and each fragment adopts an independent transmission thread; verifying the consistency of the Hash value of the downloaded file and the blockchain evidence, and executing an exponential backoff retransmission strategy on the fragments which fail in verification; and performing streaming decryption recombination on the verified fragments to generate a temporary file with a digital watermark. The method is mainly used in data transmission field.
Owner:BEIJING SENYI INTELLIGENT TECHNOLOGY CO LTD

Charging and discharging control of energy devices in a power system

A control switch incorporating a 1:2 demultiplexer is used to control timing for a concurrent switching, break-before-make and make-before-break power multiplexing, and is configurable to link a plurality of the control switches into a control chain to perform sequential charging, sequential discharging, parallel charging, parallel discharging, and concurrent sequential charging and discharging for a plurality of batteries coupled to the control chain in a power system.
Owner:ENTRANTECH INC

Adaptive waveform selection for wireless communication

Systems, methods, and instrumentalities are described herein for adaptive waveform selection for wireless communication. In examples, the wireless transmit / receive unit (WTRU) may be configured with the orthogonal frequency-division multiplexing (OFDM) initially. If the WTRU, which may support dynamic waveform switch, receives system information blocks (SIBs) from multiple cells where at least one of them supports dynamic waveform switch, the WTRU may prioritize the initial access with the cell(s) that support dynamic waveform switch. If the WTRU supports dynamic waveform switch, the WTRU may send a physical uplink shared channel (RUSCH) in msg3 at a configured grant, which may be offset by a preconfigured number of slots with a waveform that is different from the preconfigured waveform type.
Owner:INTERDIGITAL PATENT HOLDINGS INC

Spatially encoded biological assays

The present invention provides assays and assay systems for use in spatially encoded biological assays. The invention provides an assay system comprising an assay capable of high levels of multiplexing where reagents are provided to a biological sample in defined spatial patterns; instrumentation capable of controlled delivery of reagents according to the spatial patterns; and a decoding scheme providing a readout that is digital in nature.
Owner:PROGNOSYS BIOSCIENCES INC

Pipeline element library creating method based on model training and image recognition technology

The invention relates to the technical field of image recognition data processing, in particular to a pipeline element library creating method based on a model training and image recognition technology, which comprises the following steps of: acquiring pipeline element image data with depth information through a multispectral imaging scheme, executing Retinex algorithm illumination equalization and morphological restoration through a cascade preprocessing pipeline, and establishing a pipeline element library. A standardized data set associated with the metadata is generated. A multi-task joint learning framework is adopted to integrate ResNet-50, HRNet and Mask R-CNN networks, a bottom convolution feature extraction layer is shared, oil stain and strong reflection antagonism data synthesized by a generative adversarial network is injected, model parameters are optimized in combination with a progressive training strategy, and a lightweight MobileNetV3 model is output. Model parameters and an index structure are dynamically updated, and system self-calibration is achieved in combination with a cross-device calibration protocol and data consanguinity tracking. According to the method, through automatic data labeling, multi-task feature multiplexing and retrieval feedback closed loop, the construction efficiency of the pipeline element library is effectively improved, and the recognition error rate in a complex environment is reduced.
Owner:BEIJING HKRSOFT TECH CO LTD

Parallel task scheduling method based on node dynamic load and task relevance measurement

The invention discloses a parallel task scheduling method based on node dynamic load and task relevance measurement. The parallel task scheduling method comprises the following steps: constructing a global scheduler for computing resource virtualization, sample allocation based on reinforcement learning and a local scheduler based on hybrid drive model scheduling on a server; in the preprocessing stage, computing resources are counted, and computing resource virtualization, GPU virtualization multiplexing technology, GPU acceleration and the like are carried out; a global scheduling stage oriented to test task parallel optimization: performing sample scheduling feasibility analysis containing fuzzy time, computational node selection based on an EXP3 algorithm, and sample scheduling oriented to node load; and a multilevel model parallelization stage based on domain knowledge and data hybrid driving, wherein a model cohesion coupling degree measurement mechanism, a model coupling mode knowledge base and a multilevel model parallelization method are included. According to the method, the global scheduler and the local scheduler are comprehensively utilized, the parallelism degree between samples and parallelism between models in the samples are achieved, and the method has wide application value and use prospects in all fields.
Owner:NO 8511 RES INST OF CASIC

High-temperature wireless temperature probe based on NTC (Negative Temperature Coefficient) lead time division multiplexing and manufacturing method

The invention discloses a high-temperature wireless temperature probe based on NTC lead time division multiplexing and a manufacturing method, and relates to the technical field of wireless temperature probes, and the high-temperature wireless temperature probe comprises a needle tube assembly, a ceramic insulating ring and a coaxial coupling connecting shaft which are connected in sequence; a PCBA board is arranged in the needle tube assembly, and an environment temperature sensor lead and a time division multiplexing circuit are integrated on the PCBA board and selectively connected with the radio frequency receiving and transmitting circuit or the RC charging and discharging temperature measuring circuit. At least two temperature sensors; the ceramic insulating ring is arranged at the joint part of the needle tube assembly and the coaxial coupling connecting shaft, and an inner cavity forms a closed cavity; the coaxial coupling connection shaft comprises a metal outer wall which forms a radio frequency signal ground and a metal inner conductor which forms a coaxial transmission structure with an environment temperature sensor lead. According to the high-temperature wireless temperature probe, mutual interference between the radio frequency function and the temperature sensing function of the wireless temperature probe is effectively eliminated, and the high-temperature wireless temperature probe with high-precision temperature measurement performance is provided.
Owner:SHENZHEN KUKI ELECTRONICS CO LTD

Motor-clutch time division multiplexing driving system for rope-driven dexterous hand

The invention discloses a motor-clutch time division multiplexing driving system for a rope-driven dexterous hand. The motor-clutch time division multiplexing driving system comprises a driving box, a plurality of rope-driven mechanisms and a plurality of dexterous fingers, wherein the rope-driven mechanisms and the dexterous fingers are arranged on the driving box; the rope driving mechanism comprises a power unit, a worm and gear unit and a guide unit. The power unit uses a sun gear to drive a plurality of planetary gears, and an electromagnetic clutch outputs power to the worm and gear unit; the plurality of worm and gear units are respectively used for controlling the winding and unwinding of the plurality of driving ropes; the plurality of guide units respectively guide the plurality of driving ropes to the same rope-driven rotating joints of the plurality of dexterous fingers; different rope-driven rotating joints of the dexterous fingers are controlled by different rope-driven mechanisms, and the same rope-driven rotating joint of the multiple dexterous fingers is controlled by the same rope-driven mechanism. According to the scheme, the fast switching characteristic of the electromagnetic clutch is ingeniously utilized, and driving control over multiple degrees of freedom by a single motor is achieved.
Owner:SUN YAT SEN UNIV

Multi-model time division multiplexing and parallel loading reasoning service method and system

The invention relates to a multi-model time division multiplexing and parallel loading reasoning service method and system, and the method comprises the steps: receiving a user model reasoning request, and searching a corresponding model according to the user model reasoning request; deploying part of parameter layers of the model required by reasoning into a GPU (Graphics Processing Unit); inputting the user model reasoning request text into a parameter layer deployed by the GPU for calculation; when the partial parameter layers are calculated, loading the residual parameter layers of the model to the corresponding specified GPU; after reasoning calculation is completed, the parameter layer is unloaded and returned to the CPU, and a reasoning result is returned. A GPU device cluster is used for carrying out reasoning service on the models, cooperation among device clusters of multiple models is achieved, and the reasoning time delay of each model request is reduced. And under the condition of reserving and occupying fewer system video memories, the same and even better reasoning speed can be achieved. The model deployment and scheduling process is automated, and the deployment cost of the model is reduced as much as possible under the condition that the SLO requirement of each model request is met.
Owner:BEIJING INBO DIGITAL TECH CO LTD +1