655 results about "Request distribution" patented technology

The invention provides a cache management method, a cache management device, a computer storage medium and electronic equipment, and relates to the technical field of computers. The method comprises the steps of receiving a reasoning task request and distributing the reasoning task request to a target storage page; key value cache information of the first round of reasoning task is stored in a hard disk cache, and when the second round of reasoning task is executed, key value cache information generated before the second round of reasoning task is preloaded layer by layer from the hard disk cache; when the last round of reasoning task is received, storing first target key value cache information correspondingly generated by the last round of reasoning task into the matched target physical block; and performing hybrid grouping compression on key cache information and value cache information in the first target key value cache information to obtain second target key value cache information after quantization compression. According to the invention, triple balance of video memory-calculation performance-precision can be realized.

The invention discloses a micro-service dynamic weight load balancing method and related equipment. The method comprises the following steps: asynchronously acquiring a multi-dimensional real-time load index of a micro-service instance, obtaining a load prediction value by virtue of a model, fusing the real-time index, the prediction value and a service health degree, calculating a dynamic weight by depending on service load data, distributing a request according to the weight, and feeding back an adjustment coefficient. According to the method, by asynchronously collecting the multi-dimensional real-time indexes, an accurate basis is provided for weight calculation, so that the request allocates the actual load of the matched instance, and the situation that the response of a high-load instance is slow due to too many requests is avoided; by introducing a load prediction value, load change can be pre-judged in advance, a weight strategy is adjusted in advance, overload when an instance load suddenly increases is prevented, and the fluctuation resistance of the system is improved; the weight is calculated by fusing multi-dimensional data, so that the comprehensive load of the instance can be comprehensively reflected, and evaluation deviation caused by a single index is avoided; a feedback mechanism can enable weight calculation to continuously adapt to a system state, and load unevenness caused by weight fixing or adjustment lag is avoided.

The invention relates to the technical field of computers, discloses an intelligent scheduling method and system for server cluster load balancing, and aims to solve the defects of the existing server cluster load balancing technology in response lag, non-uniform resource utilization rate, service quality guarantee, global optimization capability, fine-grained state perception and scheduling decision. The method comprises the following steps: collecting server state and request feature data, constructing a cluster state and service capability model, and predicting a load trend; and generating an optimal routing strategy by using deep reinforcement learning and multi-objective optimization, and issuing adjustment request distribution. The system comprises a data acquisition module, an application request feature acquisition module, a state sensing and modeling module, a load prediction module, an intelligent scheduling decision module and an instruction execution module. By adopting the technical scheme, the resource utilization rate can be improved, the response time can be reduced, the throughput can be improved, the system stability and elasticity can be enhanced, and the operation cost and energy consumption can be reduced.

The embodiment of the invention provides a read-write request processing method, a protocol conversion module, electronic equipment and a storage medium, and relates to the technical field of chips. The method comprises the steps that a protocol conversion module is arranged in a processor cluster, an original read-write request is converted into a first read-write request conforming to a CHI protocol through a first conversion module in the protocol conversion module, and a corresponding transmission channel is distributed to the first read-write request; and the request queue module orderly sends the received first read-write request, and the second conversion module recombines the first read-write request information to a second read-write request conforming to the CHI protocol, and allocates a corresponding channel for the second read-write request. According to the embodiment of the invention, the internal data channel (AXI or similar AXI) is directly converted into the CHI interface in the processor cluster, the internal circuit resources of the processor cluster are reused, the circuit area is saved, and the access path time delay is reduced.

Disclosed are systems and methods for effectively managing user access rights, including processing and revalidating user access requests. A computer-implemented method determines an entitlement risk score, a user risk score, and an overall risk score associated with a user access request based on criticality models and metadata. Based on the overall risk score, a criticality level is assigned to the user access request, and the computer-implemented method processes the user access request. The user access request is either automatically or manually processed based on the assigned criticality level. A plurality of user access requests requiring manual processing are processed based on an external input, and the plurality of user access requests are prioritized based on their respective assigned criticality levels and processing histories.

The invention provides a TCP long connection server automatic expansion method and system. The method comprises the following steps: acquiring and monitoring various load indexes of a server in real time; receiving and analyzing the real-time load data, and judging whether resources need to be expanded or reduced currently according to a preset threshold value; based on the current load trend and historical data, performing short-term load prediction by adopting an extended decision algorithm; when expansion or reduction conditions are met, expansion or reduction operation is started, and a capacity expansion and reduction decision instruction is output; configuring a plurality of load balancing strategies, and distributing the received TCP long connection request to each service instance; and dynamically adjusting the size of a connection pool in the TCP long connection service cluster according to the system load condition and the capacity expansion and contraction decision instruction, and executing TCP connection load balancing operation after resource expansion or reduction. According to the invention, dynamic adjustment of resources is realized through technical means of load monitoring, automatic expansion, load balancing, dynamic connection management and the like, so as to meet service requirements under different load conditions.

The invention discloses a zero-copy data transmission method and device, a medium and a product, and belongs to the technical field of data transmission, and the method comprises the steps that an application program requests a memory buffer area from a kernel module; the kernel module allocates a physical memory buffer area according to the request, and returns a specially designed descriptor (including a handle corresponding to an address pointer); the application program submits a descriptor to the kernel module and requests an address pointer of the memory buffer area; after the kernel module verifies the descriptor, the physical memory buffer area is mapped to a virtual address space of the application program, and a corresponding address pointer is returned; the application program writes data needing to be sent according to the address pointer, and after writing is completed, the descriptor is submitted to the kernel module and the written data is requested to be sent; after the kernel module verifies the descriptor, a DMA controller is configured and started, and the written data is directly read from the physical memory buffer area and pushed to a network card to be sent. According to the invention, the security and certainty can be improved while the transmission performance is ensured.

The invention provides a load balancing method and device for inference service, electronic equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps: obtaining the load state information of any inference service node in an inference service node set in a process that the inference service node set processes a session request; under the condition that the load state information indicates that the node pressure of any reasoning service node is greater than a pressure threshold value, creating proxy service corresponding to any reasoning service node; creating a copy set of the proxy service according to the node pressure; according to a request distribution strategy, distributing a session request corresponding to any reasoning service node to each copy in a copy set for processing, and executing a load balancing operation between the proxy service and each copy; and under the condition that the pressure information corresponding to the copy set meets the pressure requirement, determining a first copy in the copy set, releasing copies except the first copy in the copy set, and replacing the proxy service with the first copy.

The invention relates to the technical field of reasoning optimization, in particular to a PD separation reasoning framework optimization method oriented to a large language model, which comprises the following steps: S1, reconstructing a memory structure of a KV Cache, adjusting an original discrete storage structure allocated according to a model layer into a continuous storage structure allocated according to blocks, and changing the memory structure from [layer, k / v, block id, numhead, head, block size] into [block id, layer, k / v, numhead, head size, block size]; s2, dividing a short sequence, a medium sequence and a long sequence according to the length of the cue word, combining the short sequence into a Batch group, and preferentially extruding the short sequence and then processing the long sequence; and S3, deploying a hybrid throughput node cluster, and according to the input prompt word length dynamic allocation request, allocating a short request to a low throughput TP node of the throughput, and allocating a long request to a high throughput TP node of the throughput. The method is more suitable for a PD separated system architecture, and the transmission efficiency of the KV cache and the calculation efficiency of the GPU are improved, so that the throughput of the system is improved, and the maximum resource utilization is achieved.

The invention discloses a server hardware testing terminal and method, and relates to the technical field of server hardware testing, and the method comprises the steps: obtaining server hardware configuration parameters and storage equipment specification information, collecting the number of processor cores, memory capacity and storage equipment read-write speed reference data through a system monitoring interface, building a hardware performance baseline file, and storing the baseline file in a server; obtaining a complete hardware resource list and a performance index range; the method comprises the following steps: constructing a multi-dimensional workload model according to information retrieval scene features, creating query request sets of different scales by adopting a random number generator, simulating a mixed load mode of full-text retrieval and database query, and determining resource consumption weights and execution time distribution of various query operations; and analyzing query request distribution characteristics in the workload model through a dynamic load balancing algorithm, if query requests are concentrated in a specific time period, adjusting a load distribution strategy, obtaining a uniformly distributed test load sequence, and judging the impact degree of a load peak value on hardware resources.

A method for use in a storage processor, comprising: receiving an I / O request that is associated with a logical block address; allocating a cache slot that corresponds to the logical block address, wherein allocating the cache slot includes: identifying a type of memory that is required to host the cache slot by using one or more configuration settings that are associated with the cache slot, identifying a cache slot that is both available and hosted in the identified type of memory, and assigning the identified cache slot to the logical block address; and executing the I / O request by using the allocated cache slot, wherein using the allocated cache slot includes identifying a memory address that is associated with the cache slot and executing at least one direct memory access (DMA) command that is associated with the allocated cache slot.

A method for automating a process plant includes configuring field components of the process plant as a swarm of self-organizing agents where each agent communicates with adjacently physically connected agents. The method includes reacting on requests generated by requesting agents by responsible agents assigned based on a prioritization mechanism established using a topological representation of the process plant. The prioritization mechanism is established by assigning a penalty value to each request and forwarding the request via one or more paths using the topological representation by incrementing the penalty value at an agent that forwards or reacts on the request depending on the type of request and / or agent. Thereby, a penalty matrix is created indicating how much penalty would be incurred if a particular request is reacted on by a given responsible agent. The penalty matrix is used to assign requests to responsible agents to minimize a total penalty value.

The invention provides a large-scale concurrent network request management method based on a computing power host management platform, belongs to the technical field of computer networks, and aims to reduce the request pressure on a rear-end host through technologies such as merging same or similar requests, allocating priorities for the requests, maintaining request queues and controlling the request sending frequency. The utilization efficiency of resources is improved, so that the system performance and the user experience are improved.

According to one or more aspects, apparatus, systems, and methods may be provided for language agnostic full link gray deployment of microservices and middleware. According to an aspect a method may be provided for propagating and routing traffic between microservices of an application. The method includes receiving, at a plugin of a proxy corresponding to a service of the application, an outbound request from the service. The outbound request may indicate a trace identifier (ID) identifying a trace of the outbound request in the application. The method may further include assigning, by the plugin, a custom label to the outbound request. The custom label may be associated with the trace ID and indicate a version of the application towards which the outbound request is to be routed. The method may further include sending, by the plugin to a router of the proxy, the outbound request for routing.

The invention provides a multi-node dynamic switching method and a multi-node dynamic switching system for a MongoDB (MongoDB). The method comprises the following steps: acquiring configuration information of a plurality of MongoDB nodes, and establishing a data access layer based on the configuration information; through a data access layer, acquiring index data marks of the MongoDB nodes about a plurality of operation indexes according to a hierarchical frequency acquisition task configured by each MongoDB node; in combination with a node weight preset value of the MongoDB node, converting the node performance index into a weight value of the MongoDB node; based on a connection pool management mechanism and the node available state data, a request distribution strategy is determined according to the weight value, and the request distribution strategy is used for indicating the distribution proportion of the MongoDB requests distributed to the MongoDB nodes; and performing state detection on the MongoDB node through a data access layer, marking the detected MongoDB node as an unavailable state when detecting that the MongoDB node meets an abnormal state condition, and updating available state data of the node. The problems that in the prior art, manual intervention is needed, the real-time load condition of the node cannot be sensed, and the code coupling degree is high are solved.

The invention discloses a request allocation method and device, a nonvolatile storage medium and electronic equipment. The method comprises the steps that a target first node corresponding to a user request is determined from a plurality of first nodes through a first-layer hash ring, each first node corresponds to a set of second nodes, and the first-layer hash ring comprises a hash value area corresponding to each first node; load information of each second node corresponding to the target first node is determined, the load distribution state of each second node is determined according to the load information of each second node, the load information comprises the request hit number of each second node, and the request hit number is the effective request number of each node load; and determining a target second node for processing the user request according to the load distribution state. The technical problems that the load degree is high when the number of the computational nodes is large and the whole cluster needs to be adjusted when the nodes are added and deleted due to the fact that a single-layer hash ring is adopted in the related technology are solved.

The invention relates to a resource allocation method and device, computer equipment, a storage medium and a program product. Obtaining target feature data, related to the cloud service resources, of the cloud service system in the first historical time period; determining target resource demand data for the cloud service resources in a future preset time period according to the target feature data; according to the target resource demand data and the current available resource quantity of the cloud service resources in the cloud service system, increasing or reducing the cloud service resources of the cloud service system; and distributing cloud service resources for each future service request according to the service attribute information of each future service request in the future time period. According to the scheme, the cloud service resources are dynamically regulated and controlled, and the cloud service resources are allocated to the future service requests according to the service attribute information, so that the system performance is ensured, the resource waste is avoided, the cloud service resources are pre-allocated to the future service requests, and the response speed of the future service requests can be improved.

A fleet-based migration system is described that can obtain requests to initiate network connections from individual client devices and divide the requests among one or more sets of endpoints with differing configurations. The endpoints initiate connection attempts based on the requests and capture data regarding the connection attempts. Using the captured data, the system can determine incompatibilities between client devices and fleet configurations and inform users of the incompatibility.

In response to receiving a reservation request at a provider network, metadata indicating that a group of network addresses of a subnet is reserved for operations of a particular category is stored. A first request for an operation, requiring assignment of an address of the reserved group is rejected if the operation does not belong to the particular category, even if the address is not in use. In response to a second request for an operation which does belong to the particular category, an address of the reserved group is assigned.

The invention provides a multi-speech synthesis model bearing method and device based on a virtual GPU, and relates to the technical field of graphics processing units, and the method comprises the steps: carrying out the virtualization processing of a physical graphics processing unit, dividing the physical graphics processing unit into a plurality of virtual processing units with independent video memories and calculation quotas, and combining a resource scheduling mechanism, and deploying the speech synthesis language model instances in a plurality of service containers, and constructing a plurality of speech synthesis model bearing units. After a voice synthesis request is accessed, the scheduling module carries out load balancing according to the request connection number of each bearing unit, the request is distributed to a target bearing unit with the minimum connection number, and a voice generation task is completed by a virtual processing unit bound with the target bearing unit. According to the invention, resource division can be carried out on the physical graphic processing unit, and efficient operation of the multi-speech synthesis model is realized.

The invention belongs to the technical field of smart contracts, and discloses a multi-version contract distribution and maintenance method for a smart contract, and the method comprises the steps: firstly achieving the initial deployment, upgrading and safety verification of the smart contract, formally releasing the smart contract on a block chain after the verification is passed, and enabling the generated multi-version smart contract to be stored in the block chain; a system divides data sharing and access requests according to time and allocates the requests to intelligent contracts of corresponding versions for management, and the allocation and maintenance method specifically comprises the following steps: 1) version control and marking of multi-version contracts; 2) automatic detection of a request timestamp and contract selection; 3) contract routing logic and request distribution; 4) a multi-version parallel execution mechanism; 5) a compatibility and rollback mechanism; and 6) contract upgrading and storage management of old-version contracts. According to the method, the contract version performance and the system load can be dynamically optimized, the performance and the request success rate of different versions of contracts are monitored in real time by introducing a dynamic weight routing mechanism based on the multi-version contracts, and the contract priority is dynamically adjusted.

The invention belongs to the technical field of computers, and relates to a network access method and device, computer equipment and a storage medium, and the method comprises the steps: receiving an access domain name input by a user in a target network environment; calling a target load balancer corresponding to the target network environment; analyzing the access domain name into a virtual I P address of the target load balancer based on a preset DNS server, and returning the virtual I P address to the user; receiving an access request sent by a user to the virtual IP address; based on the target load balancer, screening out a target server from a plurality of servers corresponding to the target network environment, and distributing the access request to the target server; based on the target server, performing identity verification on the user by using a preset identity verification strategy; and if the user passes the identity verification, performing corresponding response processing on the access request based on a preset access control strategy. According to the network access processing method provided by the invention, the security and availability of network access can be effectively improved.

The invention discloses an Internet of Things industry intelligent customer service supervision and control system based on artificial intelligence, and belongs to the technical field of customer service supervision. Comprising an omni-channel intelligent access and intention understanding module, an AI intelligent center and complex decision module, an intelligent supervision and ethical regulation and control module, a man-machine cooperation and humanistic care module, a data-driven optimization and edge intelligent module and a security privacy and controllable treatment module, and the omni-channel intelligent access and intention understanding module is used for integrating multiple channels. Unified request distribution is achieved, data input by a user are analyzed through voice recognition, NLU natural language understanding and image recognition technologies, composite intentions are accurately captured, interaction strategies are dynamically adjusted in combination with device state data, user historical behaviors and real-time positions, and pacified talking skills or manual intervention are triggered through voiceprint / text emotion analysis. On the basis of realizing customer service supervision and regulation, all-channel fusion and intention accurate analysis can be realized, and ethical safety integrated design can be realized.

The invention belongs to the technical field of middleware management, and aims to provide a middleware management platform and a working method thereof. Wherein the middleware management platform comprises a service information management center, a task scheduling engine, a load balancer and a dynamic extension module; wherein the task scheduling engine is used for receiving a task execution request sent by an external client, obtaining an available service instance list from the service information management center, and performing task allocation processing by adopting an intelligent optimization strategy to obtain a task allocation result of a specified micro-service instance in the available service instance list; task request distribution is carried out on the specified micro-service instance based on the load balancer according to the task distribution result; and the dynamic extension module is used for carrying out load prediction processing on each micro-service instance so as to execute a micro-service instance capacity expansion action based on the predicted load information of each micro-service instance. Task scheduling and resource allocation management can be efficiently carried out, and the reliability and expandability of the whole system can be improved.

Embodiments of the present disclosure provide a load balancing processing method, a device, a storage medium, and a system. The method comprises: obtaining a first number of Internet of Things devices corresponding to each of a plurality of application servers, wherein the first number of Internet of Things devices corresponding to any application server is the number of Internet of Things devices allowed to be accessed when a resource consumption of the application server reaches a set upper limit value; in response to a persistent connection establishment request of a target Internet of Things device, obtaining a second number of Internet of Things devices actually accessed in the plurality of application servers; determining load weights of the plurality of application servers on the basis of the first number of Internet of Things devices and the second number of Internet of Things devices corresponding to each of the plurality of application servers; and on the basis of the load weights of the plurality of application servers, determining a target application server to which the persistent connection establishment request is allocated, so as to establish a persistent connection between the target Internet of Things device and the target application server.

The invention discloses a DNS fault diagnosis method and device based on statistical verification, and a storage medium. The method comprises the following steps: extracting a domain name query request and response message information from a communication message; matching the suffixes of the domain names to determine an authoritative resolution server; according to the response code and the response time delay of the response message information, identifying an abnormal analysis request and respectively counting the total request quantity and the abnormal request quantity of each domain name and each authoritative analysis server; acquiring a forwarding path topology, and performing multi-layer aggregation on the total request quantity and the abnormal request quantity based on the forwarding path topology to generate abnormal request distribution feature data; and matching the abnormal request distribution feature data with the fault hypothesis model to identify a target analysis service device with a fault. According to the method, the authoritative resolution server is accurately positioned through the inverted-order dictionary tree, and the abnormal distribution characteristics are generated in combination with multi-layer aggregation statistics, so that automatic and accurate positioning and influence range quantification of the DNS fault are realized, and the accuracy and processing efficiency of fault diagnosis are remarkably improved.

The invention provides a memory operation method and device, equipment, a storage medium and a program product, and the method can comprise the steps that a memory operation request for a memory is obtained, the memory operation request is used for requesting memory allocation or requesting memory release, the memory comprises a plurality of memory blocks, and the memory blocks correspond to at least two memory granularities; determining memory block information corresponding to at least one memory granularity, wherein the memory block information comprises a memory granularity value, a memory block state and memory block use information; and processing the memory operation request according to the memory block information corresponding to the at least one memory granularity so as to allocate or release the memory. And the flexibility of operating the memory is improved.

The invention provides a server load balancing method and device, a medium and a program product, and a specific implementation mode of the application comprises the steps that after a load balancer receives a task request, attributes such as basic types (authentication, authorization and charging), priorities and protocol versions of the request are analyzed firstly; meanwhile, the resource utilization rate, response time, health state and other information of the server cluster are monitored in real time, a dynamic strategy decision tree is constructed based on the data, in addition, the load balancer sends a health check request to the server regularly, whether the server breaks down or not is judged according to the state information returned by the server, and the service life of the server is prolonged. And if a fault exists, removing the fault from the request allocation list to ensure that the request is always allocated to the available server. According to the method, the dynamic adjustment of the load balancing strategy and the update of the request allocation list are realized, and the resource utilization rate of the server is improved so as to cope with the rapidly changed request load and flow mode.

The invention provides a cache management method and device based on data small granularity. The method comprises the steps that S1, source equipment is initialized; s2, applying for a memory read request once; s3, judging whether the length of the request data is smaller than the residual space of the datram of the source equipment or not, and if yes, entering the step S4; s4, allocating a tag and a data ram write address for the memory read request, and recording a mapping relationship between the tag and the data ram write address into a mapping table of the tag and the wptr; s5, initiating a memory read request with a tag to the target device to obtain return data; s6, according to the tag of the returned data, reading the tag and the wptr mapping table to obtain a write address wptr of the corresponding data; s7, writing the returned data to a corresponding address of the datram; and S8, reading the datram to obtain the data, and processing the data. The method has the beneficial effects that an address mapping storage mode of applying for the data space based on the actual read data length is provided, the space utilization rate of source equipment data is effectively improved, and the data reading performance of the memory is improved.

The invention discloses a buffer, an instruction processing method, a processor and computer equipment, and relates to the technical field of chips. The buffer is an L2-level buffer in a processor and comprises a request distribution circuit and a buffer circuit, the request distribution circuit is connected with the cache circuit; the cache circuit comprises a cache unit and an operation unit; the operation unit is connected with the cache unit; the request distribution circuit is used for receiving the cache access request and sending the cache access request to the cache circuit; the cache unit is used for reading source data corresponding to the cache access request under the condition that the cache access request is a write request corresponding to the atomic operation, and inputting the read source data into the operation unit; and the arithmetic unit is used for executing atomic operation corresponding to the cache access request on the source data to obtain operation result data and writing the operation result data into the cache unit.