950 results about "Request distribution" patented technology

The invention provides a visitor identification and multi-dimensional user portrait routing distribution method based on private domain traffic, and relates to the technical field of data analysis, and the method comprises the steps: collecting multi-source data information of a visitor from a private domain traffic pool, constructing a multi-dimensional feature vector, generating a visitor identity fingerprint feature, and building a visitor identity identification model based on the visitor identity fingerprint feature; constructing a multi-level label system based on a recognition result of the visitor identity recognition model; performing feature extraction on historical behavior track data and historical content interaction data of visitors by adopting a transfer learning algorithm, classifying the extracted features based on the multi-level label system to generate user portrait features, dynamically updating the user portrait features in an incremental learning mode, constructing a multi-dimensional user portrait model, and displaying the multi-dimensional user portrait model. And distributing different visitor requests to the corresponding service nodes by adopting a routing distribution strategy in combination with the real-time load states of the service nodes.

The invention discloses a request allocation method and system based on a heterogeneous computing system, equipment and a medium, relates to the technical field of computers, and determines reading time required for using key value cache when a plurality of heterogeneous computing power equipment accesses a memory under the condition of preferentially considering the use condition of a key value cache mechanism. Requests are reasonably distributed by considering the performance difference of the computing power information of the heterogeneous computing power equipment, the memory information corresponding to the memory extension and the characteristics of the reasoning task information. Furthermore, the reasonability of request allocation is improved according to a comparison relationship between the preset allocation request quantity and the concurrent request quantity and different strategies for judging whether allocation conditions are met or not. Therefore, the technical problem that more computing power resources are wasted due to the fact that the allocation balance of the reasoning request is poor when a separated memory or a local memory is called can be solved, and the technical effects that the reasoning task information is reasonably allocated to the heterogeneous computing power equipment, so that the allocation balance is improved, the computing power resources are saved, and the cost is reduced are achieved.

The invention provides a cache management method, a cache management device, a computer storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps of receiving a reasoning task request and distributing the reasoning task request to a target storage page; key value cache information of the first round of reasoning task is stored in a hard disk cache, and when the second round of reasoning task is executed, key value cache information generated before the second round of reasoning task is preloaded layer by layer from the hard disk cache; when the last round of reasoning task is received, storing first target key value cache information correspondingly generated by the last round of reasoning task into the matched target physical block; and performing hybrid grouping compression on key cache information and value cache information in the first target key value cache information to obtain second target key value cache information after quantization compression. According to the invention, triple balance of video memory-calculation performance-precision can be realized.

The invention discloses a micro-service dynamic weight load balancing method and related equipment. The method comprises the following steps: asynchronously acquiring a multi-dimensional real-time load index of a micro-service instance, obtaining a load prediction value by virtue of a model, fusing the real-time index, the prediction value and a service health degree, calculating a dynamic weight by depending on service load data, distributing a request according to the weight, and feeding back an adjustment coefficient. According to the method, by asynchronously collecting the multi-dimensional real-time indexes, an accurate basis is provided for weight calculation, so that the request allocates the actual load of the matched instance, and the situation that the response of a high-load instance is slow due to too many requests is avoided; by introducing a load prediction value, load change can be pre-judged in advance, a weight strategy is adjusted in advance, overload when an instance load suddenly increases is prevented, and the fluctuation resistance of the system is improved; the weight is calculated by fusing multi-dimensional data, so that the comprehensive load of the instance can be comprehensively reflected, and evaluation deviation caused by a single index is avoided; a feedback mechanism can enable weight calculation to continuously adapt to a system state, and load unevenness caused by weight fixing or adjustment lag is avoided.

The invention relates to the technical field of computers, discloses an intelligent scheduling method and system for server cluster load balancing, and aims to solve the defects of the existing server cluster load balancing technology in response lag, non-uniform resource utilization rate, service quality guarantee, global optimization capability, fine-grained state perception and scheduling decision. The method comprises the following steps: collecting server state and request feature data, constructing a cluster state and service capability model, and predicting a load trend; and generating an optimal routing strategy by using deep reinforcement learning and multi-objective optimization, and issuing adjustment request distribution. The system comprises a data acquisition module, an application request feature acquisition module, a state sensing and modeling module, a load prediction module, an intelligent scheduling decision module and an instruction execution module. By adopting the technical scheme, the resource utilization rate can be improved, the response time can be reduced, the throughput can be improved, the system stability and elasticity can be enhanced, and the operation cost and energy consumption can be reduced.

The embodiment of the invention provides a resource scheduling method and device for large model reasoning requests, and relates to the technical field of computers, the method is applied to a scheduling server in a server cluster, the server cluster further comprises a plurality of target servers, each target server is deployed with a plurality of large models, and the large models are distributed in the scheduling server. The method comprises the following steps: determining each target reasoning request and the computing resource demand quantity of each target reasoning request; substituting the computing resource demand quantity of each target reasoning request, the current residual resource quantity of each target server and the server performance sequence of each target server into a target function to determine an allocation result of each target reasoning request; and for each target reasoning request, allocating the request to a target server represented by an allocation result, so that the target server performs resource scheduling and processing according to the corresponding computing resource demand quantity, and the purpose of optimizing the use condition of the computing resources of the servers in the server cluster is achieved.

The embodiment of the invention provides a read-write request processing method, a protocol conversion module, electronic equipment and a storage medium, and relates to the technical field of chips. The method comprises the steps that a protocol conversion module is arranged in a processor cluster, an original read-write request is converted into a first read-write request conforming to a CHI protocol through a first conversion module in the protocol conversion module, and a corresponding transmission channel is distributed to the first read-write request; and the request queue module orderly sends the received first read-write request, and the second conversion module recombines the first read-write request information to a second read-write request conforming to the CHI protocol, and allocates a corresponding channel for the second read-write request. According to the embodiment of the invention, the internal data channel (AXI or similar AXI) is directly converted into the CHI interface in the processor cluster, the internal circuit resources of the processor cluster are reused, the circuit area is saved, and the access path time delay is reduced.

Disclosed are systems and methods for effectively managing user access rights, including processing and revalidating user access requests. A computer-implemented method determines an entitlement risk score, a user risk score, and an overall risk score associated with a user access request based on criticality models and metadata. Based on the overall risk score, a criticality level is assigned to the user access request, and the computer-implemented method processes the user access request. The user access request is either automatically or manually processed based on the assigned criticality level. A plurality of user access requests requiring manual processing are processed based on an external input, and the plurality of user access requests are prioritized based on their respective assigned criticality levels and processing histories.

The invention relates to the technical field of digital marketing, in particular to a digital advertisement putting method based on big data, and the method comprises the following steps: initializing an advertisement model, importing advertisement request time sequence data, carrying out time sequence analysis, adjusting an enqueue rule and a processing rate, calculating request distribution in a differentiated time period, and generating model parameter configuration. According to the invention, through a queuing theory and time sequence data analysis, advertisement time sequence and processing efficiency optimization, time sequence analysis and parameter adjustment, advertisement timeliness is enhanced, user activity peak synchronization is ensured, user participation degree and advertisement effect are improved, and recalculation of service time and estimation of waiting time support real-time monitoring of service efficiency. Accurate matching of advertisement resources and user requirements is achieved, real-time adjustment of strategies and simulation testing rapidly adapt to market changes, the resource utilization efficiency and ROI are improved, and precision and economic benefits of advertisement putting are remarkably improved through fine management and data driving optimization.

The invention discloses an emergency fire fighting question answering method and system based on the AI large language model technology and a storage medium, and relates to the technical field of artificial intelligence, the emergency fire fighting question answering system comprises a front-end module, an AI preprocessing thread pool, a knowledge base module and a large model calculation engine module, the AI preprocessing thread pool receives a service request sent by the front-end module, and sends the service request to the AI preprocessing thread pool; different task threads are distributed, a knowledge base of a corresponding region category is automatically called according to the region category input by a service request to perform content similarity retrieval, knowledge content data obtained through retrieval and the corresponding service request are sorted and then uniformly sent to the large model calculation engine module supporting parallel processing to perform analysis and calculation, and the large model calculation engine module is used for performing analysis and calculation on the knowledge content data and the corresponding service request. By using the technical means of the invention, the user does not need to think what the keyword of the current question is, and can completely organize the question, directly ask the question, automatically query the local professional knowledge base and give professional and accurate answers according to the actual situation of the site, so that real barrier-free communication is realized.

The invention provides a TCP long connection server automatic expansion method and system. The method comprises the following steps: acquiring and monitoring various load indexes of a server in real time; receiving and analyzing the real-time load data, and judging whether resources need to be expanded or reduced currently according to a preset threshold value; based on the current load trend and historical data, performing short-term load prediction by adopting an extended decision algorithm; when expansion or reduction conditions are met, expansion or reduction operation is started, and a capacity expansion and reduction decision instruction is output; configuring a plurality of load balancing strategies, and distributing the received TCP long connection request to each service instance; and dynamically adjusting the size of a connection pool in the TCP long connection service cluster according to the system load condition and the capacity expansion and contraction decision instruction, and executing TCP connection load balancing operation after resource expansion or reduction. According to the invention, dynamic adjustment of resources is realized through technical means of load monitoring, automatic expansion, load balancing, dynamic connection management and the like, so as to meet service requirements under different load conditions.

The invention discloses a zero-copy data transmission method and device, a medium and a product, and belongs to the technical field of data transmission, and the method comprises the steps that an application program requests a memory buffer area from a kernel module; the kernel module allocates a physical memory buffer area according to the request, and returns a specially designed descriptor (including a handle corresponding to an address pointer); the application program submits a descriptor to the kernel module and requests an address pointer of the memory buffer area; after the kernel module verifies the descriptor, the physical memory buffer area is mapped to a virtual address space of the application program, and a corresponding address pointer is returned; the application program writes data needing to be sent according to the address pointer, and after writing is completed, the descriptor is submitted to the kernel module and the written data is requested to be sent; after the kernel module verifies the descriptor, a DMA controller is configured and started, and the written data is directly read from the physical memory buffer area and pushed to a network card to be sent. According to the invention, the security and certainty can be improved while the transmission performance is ensured.

The invention provides a load balancing method and device for inference service, electronic equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps: obtaining the load state information of any inference service node in an inference service node set in a process that the inference service node set processes a session request; under the condition that the load state information indicates that the node pressure of any reasoning service node is greater than a pressure threshold value, creating proxy service corresponding to any reasoning service node; creating a copy set of the proxy service according to the node pressure; according to a request distribution strategy, distributing a session request corresponding to any reasoning service node to each copy in a copy set for processing, and executing a load balancing operation between the proxy service and each copy; and under the condition that the pressure information corresponding to the copy set meets the pressure requirement, determining a first copy in the copy set, releasing copies except the first copy in the copy set, and replacing the proxy service with the first copy.

The invention relates to the technical field of reasoning optimization, in particular to a PD separation reasoning framework optimization method oriented to a large language model, which comprises the following steps: S1, reconstructing a memory structure of a KV Cache, adjusting an original discrete storage structure allocated according to a model layer into a continuous storage structure allocated according to blocks, and changing the memory structure from [layer, k / v, block id, numhead, head, block size] into [block id, layer, k / v, numhead, head size, block size]; s2, dividing a short sequence, a medium sequence and a long sequence according to the length of the cue word, combining the short sequence into a Batch group, and preferentially extruding the short sequence and then processing the long sequence; and S3, deploying a hybrid throughput node cluster, and according to the input prompt word length dynamic allocation request, allocating a short request to a low throughput TP node of the throughput, and allocating a long request to a high throughput TP node of the throughput. The method is more suitable for a PD separated system architecture, and the transmission efficiency of the KV cache and the calculation efficiency of the GPU are improved, so that the throughput of the system is improved, and the maximum resource utilization is achieved.

The invention relates to an efficient deployment method and system for a privatized large language model ecological service. The system comprises an evaluation module, an index collector, a monitoring module, a vLLM framework, a model cluster service module and a request distribution module. The evaluation module performs quantitative evaluation on the performance of the system under different hardware platforms; the index collector collects server cluster performance indexes, cluster resource states and load information in real time; the monitoring module is used for monitoring the server cluster; the vLLM framework dynamically allocates computing resources to realize management of a multi-modal server cluster, and performs unified management and scheduling through PM2 to realize multi-model efficient parallel task processing; a vLLM framework is started by the model cluster service module, and different large model instances are packaged in a Docker container; the request distribution module distributes external requests in a self-adaptive mode according to cluster resource states and load information fed back by the index collector in real time. Intelligent request distribution and dynamic load balancing are realized, and integration and efficient management of a multi-modal large language model are realized.

The invention discloses a server hardware testing terminal and method, and relates to the technical field of server hardware testing, and the method comprises the steps: obtaining server hardware configuration parameters and storage equipment specification information, collecting the number of processor cores, memory capacity and storage equipment read-write speed reference data through a system monitoring interface, building a hardware performance baseline file, and storing the baseline file in a server; obtaining a complete hardware resource list and a performance index range; the method comprises the following steps: constructing a multi-dimensional workload model according to information retrieval scene features, creating query request sets of different scales by adopting a random number generator, simulating a mixed load mode of full-text retrieval and database query, and determining resource consumption weights and execution time distribution of various query operations; and analyzing query request distribution characteristics in the workload model through a dynamic load balancing algorithm, if query requests are concentrated in a specific time period, adjusting a load distribution strategy, obtaining a uniformly distributed test load sequence, and judging the impact degree of a load peak value on hardware resources.

A method for use in a storage processor, comprising: receiving an I / O request that is associated with a logical block address; allocating a cache slot that corresponds to the logical block address, wherein allocating the cache slot includes: identifying a type of memory that is required to host the cache slot by using one or more configuration settings that are associated with the cache slot, identifying a cache slot that is both available and hosted in the identified type of memory, and assigning the identified cache slot to the logical block address; and executing the I / O request by using the allocated cache slot, wherein using the allocated cache slot includes identifying a memory address that is associated with the cache slot and executing at least one direct memory access (DMA) command that is associated with the allocated cache slot.

The invention discloses an optimization method for deploying a hybrid expert model in an edge computing environment and related equipment, and the method comprises the steps: receiving reasoning requests of all users through a pre-trained optimization model, and outputting a distribution scheme of the requests to all edge servers, and an expert hybrid quantization bit width scheme of all servers. The training process of the optimization model comprises the steps that user reasoning requests are collected, pre-training data are constructed, and the pre-training data comprise randomly-generated reasoning request distribution and corresponding expert mixed quantization bit width; pre-training a strategy network in reinforcement learning by using the data; inputting a user reasoning request into the initialization strategy network to obtain a distribution scheme and a quantization bit width scheme; according to the schemes, a user reasoning request is served to obtain service quality, the service quality is used as an award of a strategy network, and network parameters are iteratively updated until the service quality reaches a preset requirement. The invention aims to realize high-precision and low-delay edge reasoning service by dynamically adjusting the expert quantization bit width of the edge server and combining the distribution optimization of the user reasoning request.

A method for automating a process plant includes configuring field components of the process plant as a swarm of self-organizing agents where each agent communicates with adjacently physically connected agents. The method includes reacting on requests generated by requesting agents by responsible agents assigned based on a prioritization mechanism established using a topological representation of the process plant. The prioritization mechanism is established by assigning a penalty value to each request and forwarding the request via one or more paths using the topological representation by incrementing the penalty value at an agent that forwards or reacts on the request depending on the type of request and / or agent. Thereby, a penalty matrix is created indicating how much penalty would be incurred if a particular request is reacted on by a given responsible agent. The penalty matrix is used to assign requests to responsible agents to minimize a total penalty value.

The invention provides a large-scale concurrent network request management method based on a computing power host management platform, belongs to the technical field of computer networks, and aims to reduce the request pressure on a rear-end host through technologies such as merging same or similar requests, allocating priorities for the requests, maintaining request queues and controlling the request sending frequency. The utilization efficiency of resources is improved, so that the system performance and the user experience are improved.

According to one or more aspects, apparatus, systems, and methods may be provided for language agnostic full link gray deployment of microservices and middleware. According to an aspect a method may be provided for propagating and routing traffic between microservices of an application. The method includes receiving, at a plugin of a proxy corresponding to a service of the application, an outbound request from the service. The outbound request may indicate a trace identifier (ID) identifying a trace of the outbound request in the application. The method may further include assigning, by the plugin, a custom label to the outbound request. The custom label may be associated with the trace ID and indicate a version of the application towards which the outbound request is to be routed. The method may further include sending, by the plugin to a router of the proxy, the outbound request for routing.

The invention provides a multi-node dynamic switching method and a multi-node dynamic switching system for a MongoDB (MongoDB). The method comprises the following steps: acquiring configuration information of a plurality of MongoDB nodes, and establishing a data access layer based on the configuration information; through a data access layer, acquiring index data marks of the MongoDB nodes about a plurality of operation indexes according to a hierarchical frequency acquisition task configured by each MongoDB node; in combination with a node weight preset value of the MongoDB node, converting the node performance index into a weight value of the MongoDB node; based on a connection pool management mechanism and the node available state data, a request distribution strategy is determined according to the weight value, and the request distribution strategy is used for indicating the distribution proportion of the MongoDB requests distributed to the MongoDB nodes; and performing state detection on the MongoDB node through a data access layer, marking the detected MongoDB node as an unavailable state when detecting that the MongoDB node meets an abnormal state condition, and updating available state data of the node. The problems that in the prior art, manual intervention is needed, the real-time load condition of the node cannot be sensed, and the code coupling degree is high are solved.

The invention relates to the technical field of large language models, in particular to a method, a device and equipment for requesting rescheduling in large language model service and a medium. The method comprises the steps that the scheduling amount of each instance is calculated according to the queuing condition of large language model service requests in each instance and key value cache, and at least one source instance to be scheduled and a target instance corresponding to each source instance to be scheduled are determined; and when a migration instruction is received, based on the type of the large language model service request and a preset migration strategy, migrating the token and / or key value cache in each to-be-scheduled source instance to a target instance corresponding to each to-be-scheduled source instance, and feeding back a response result of the large language model service request to a user. Therefore, efficient request allocation is realized in a dynamic environment, and the problem that effective scheduling cannot be performed when a traditional scheduling strategy faces uncertainty and dynamics of large language model requests is solved, so that the resource utilization rate and the response performance of a system are improved.

The invention discloses a coroutine-based micro-service gateway dynamic scheduling method, which comprises the following steps that: 1, a coroutine pool is created to process a network request, and a gateway receives the network request and adds the network request into a request queue; 2, identifying a request type, distributing a preset static priority for the network request, and adjusting the dynamic priority in combination with features on the basis of the static priority and the configured initial dynamic priority; and 3, collecting data in coroutine operation in real time, and reducing coroutine processing delay by calculating an overtime threshold, adopting an overtime differentiation strategy, a low-priority discarding strategy and updating a data state according to a data operation condition. The invention further discloses a dynamic scheduling micro-service gateway system based on the coroutine. The method can significantly improve the gateway performance, reduce the system delay, intelligently manage the request queue under a high load condition, ensure the normal operation of key services, and can be widely applied to the field of computer data.

The invention discloses a request allocation method and device, a nonvolatile storage medium and electronic equipment. The method comprises the steps that a target first node corresponding to a user request is determined from a plurality of first nodes through a first-layer hash ring, each first node corresponds to a set of second nodes, and the first-layer hash ring comprises a hash value area corresponding to each first node; load information of each second node corresponding to the target first node is determined, the load distribution state of each second node is determined according to the load information of each second node, the load information comprises the request hit number of each second node, and the request hit number is the effective request number of each node load; and determining a target second node for processing the user request according to the load distribution state. The technical problems that the load degree is high when the number of the computational nodes is large and the whole cluster needs to be adjusted when the nodes are added and deleted due to the fact that a single-layer hash ring is adopted in the related technology are solved.

The invention relates to a resource allocation method and device, computer equipment, a storage medium and a program product. Obtaining target feature data, related to the cloud service resources, of the cloud service system in the first historical time period; determining target resource demand data for the cloud service resources in a future preset time period according to the target feature data; according to the target resource demand data and the current available resource quantity of the cloud service resources in the cloud service system, increasing or reducing the cloud service resources of the cloud service system; and distributing cloud service resources for each future service request according to the service attribute information of each future service request in the future time period. According to the scheme, the cloud service resources are dynamically regulated and controlled, and the cloud service resources are allocated to the future service requests according to the service attribute information, so that the system performance is ensured, the resource waste is avoided, the cloud service resources are pre-allocated to the future service requests, and the response speed of the future service requests can be improved.

The invention discloses a memory management method and device, a computer system and related equipment, and the memory management method comprises the steps: obtaining a first memory distribution request of a thread, the first memory distribution request being used for requesting to distribute a corresponding first memory block; when a first memory allocation request of a thread is obtained, obtaining information of a CPU running the thread; according to the information of the CPU of the running thread, obtaining a Per-CPU page frame cache of the CPU of the running thread; when the corresponding first memory block exists in the Per-CPU page frame cache, the corresponding first memory block is acquired from the Per-CPU page frame cache and allocated to the thread; when the corresponding first memory block does not exist in the Per-CPU page frame cache, performing first filling processing on the Per-CPU page frame cache; and after first filling processing is carried out on the Per-CPU page frame cache, obtaining a corresponding first memory block from the Per-CPU page frame cache and allocating the first memory block to the thread. According to the technical scheme, affinity of thread data can be achieved, and locality of memory access of the processor is kept.

A fleet-based migration system is described that can obtain requests to initiate network connections from individual client devices and divide the requests among one or more sets of endpoints with differing configurations. The endpoints initiate connection attempts based on the requests and capture data regarding the connection attempts. Using the captured data, the system can determine incompatibilities between client devices and fleet configurations and inform users of the incompatibility.

In response to receiving a reservation request at a provider network, metadata indicating that a group of network addresses of a subnet is reserved for operations of a particular category is stored. A first request for an operation, requiring assignment of an address of the reserved group is rejected if the operation does not belong to the particular category, even if the address is not in use. In response to a second request for an operation which does belong to the particular category, an address of the reserved group is assigned.

The invention provides a multi-speech synthesis model bearing method and device based on a virtual GPU, and relates to the technical field of graphics processing units, and the method comprises the steps: carrying out the virtualization processing of a physical graphics processing unit, dividing the physical graphics processing unit into a plurality of virtual processing units with independent video memories and calculation quotas, and combining a resource scheduling mechanism, and deploying the speech synthesis language model instances in a plurality of service containers, and constructing a plurality of speech synthesis model bearing units. After a voice synthesis request is accessed, the scheduling module carries out load balancing according to the request connection number of each bearing unit, the request is distributed to a target bearing unit with the minimum connection number, and a voice generation task is completed by a virtual processing unit bound with the target bearing unit. According to the invention, resource division can be carried out on the physical graphic processing unit, and efficient operation of the multi-speech synthesis model is realized.