334 results about "Distributed cache" patented technology

The invention relates to the technical field of health information monitoring and management, and discloses a health information monitoring and management system based on multi-source data fusion analysis, which comprises a physiological data acquisition unit, a fusion analysis engine, an intelligent decision management module and the like. The physiological data acquisition unit acquires a multi-source heterogeneous data stream, and a multi-layer fusion topology is constructed through preprocessing; the fusion analysis engine realizes data feature association and anomaly detection through feature association and mode recognition; and the intelligent decision management module generates a health state reference strategy and dynamically allocates data source weights. The real-time calibration module calibrates a signal time domain and adapts to an analysis frequency, the data weight optimization module evaluates an optimization strategy based on credibility, and the fault-tolerant processing module completes data verification and recovery in combination with the distributed cache unit. The system realizes efficient fusion, dynamic decision and reliable management of multi-source data, improves the accuracy of health monitoring and the robustness of the system, and is suitable for intelligent health management scenes.

The invention discloses a multi-level cache optimization method and system based on data popularity self-adaptive adjustment, relates to the technical field of data storage, and aims to solve the problems that a cache mechanism in an existing desktop cloud system is static, initialization is slow, and cold and hot data are difficult to recognize. The adopted scheme comprises the steps that a three-level cache structure of a local cache L1, a distributed cache L2 and a database L3 is set; constructing a data popularity scoring model based on the access frequency, the time decay and the user priority; dividing the data into hot data, temperature data and cold data according to the popularity score, and dynamically determining migration and elimination strategies of the data in L1, L2 and L3; the hot data are loaded to the upper-level cache T in advance through a prediction mechanism, and container-level cache preheating is supported; a cache preheating strategy is integrated to a Kubernetes life cycle, and hot data loading and cold data degradation are achieved; visual strategy configuration and monitoring during operation are supported, and cache strategies are managed in real time. The method is suitable for an access optimization scene of a large-scale desktop cloud system.

The invention discloses a distributed computing task non-perception migration method and system under optical fiber network interruption. The distributed computing task non-perception migration method comprises the following steps: constructing a distributed task migration system and an output system architecture; collecting network state indexes, constructing a network topological graph, and analyzing the state of an optical fiber link for fault prediction; obtaining a calculation task operation state, establishing a sub-task mapping relation, constructing a directed dependency graph, and generating a state snapshot; analyzing resource requirements, reserving standby resources, and deploying a distributed cache system; analyzing a fault influence range, extracting an influenced calculation sub-graph, and selecting a migration target node; in a target node preloading environment, reconstructing an execution context, and redirecting a communication path; and setting a data change capture mechanism, synchronizing incremental data and executing consistency verification. According to the method, non-perception migration of the computing tasks is realized, task continuity and data consistency are guaranteed, and the reliability of the distributed computing system in an optical fiber network fault scene is improved.

The invention discloses an AI model distribution and deployment system and method oriented to cloud edge collaboration, belongs to the technical field of artificial intelligence and edge computing crossing, and aims to solve the problems of low AI model distribution efficiency and poor deployment adaptability in a cloud edge collaboration scene. The method comprises the steps of collecting multi-dimensional attribute data of edge nodes and converting the data into multi-level capability tags, and triggering tag differentiation updating according to data change amplitude; constructing a model demand hierarchical description framework by relying on capability labels, converting distribution demands into quantitative query conditions, and adjusting and screening a target edge node set through a three-level progressive matching mechanism in combination with dynamic weight; coding data blocks corresponding to redundancy are generated based on the network state of a target node, adaptive cache nodes are screened by using a distributed cache mechanism, the data blocks are pre-stored, and a cache network is established and transmitted to the target node; according to the invention, efficient distribution and accurate deployment of the AI model in the cloud edge collaborative environment are realized, and the model transmission reliability and the node adaptability are improved.

The invention provides a drug identification information efficient query method and system based on distributed cache. The method comprises the following steps: firstly, receiving and analyzing a drug identifier query request to generate a dynamic cache routing key, then positioning a target cache node, then matching a cache data block corresponding to a drug unique code in the target cache node, if matching succeeds, extracting a drug full life cycle record in the cache data block, and sending the drug full life cycle record to the target cache node; if the matching fails, initiating a cross-node retrieval request to the drug block chain database to return drug source data, splitting the drug source data into a core attribute set and an extended track set, and finally executing a dynamic combination operation on the core attribute set and the extended track set to generate a drug tracing result set and returning the drug tracing result set to the user terminal; according to the technical scheme provided by the invention, the query fluctuation defect of a single storage architecture is eliminated from the source, so that the whole system simultaneously meets three drug traceability core requirements of high concurrency, cross-mechanism and tamper-proofing.

A hotspot data elimination and maintenance method in a distributed cache system comprises the steps that the access frequency, the time locality score, the service weight and the real-time data popularity value of data are acquired, and the dynamic priority score of the data is generated; determining failure data based on the dynamic priority score; dividing the data into hot spot area data, temperature spot area data and cold spot area data, when the cache space is insufficient, preferentially eliminating the data with the lowest dynamic priority score and the retention benefit value lower than a preset threshold value in the cold spot area data, and if the cache space is still insufficient, performing data elimination on the temperature spot area data according to the latest access time of the temperature spot area data; packaging the cold point area elimination data, the warm point area elimination data and the failure data into a final transaction message; and broadcasting the final transaction message through a preset protocol, and performing distributed consistency maintenance operation on the final transaction message through the receiving node. According to the invention, intelligent and efficient cache data elimination and consistency maintenance management can be realized.

The invention relates to the technical field of high-concurrency data processing, and discloses a storage and calculation integrated data scheduling system and method for a high-concurrency scene, and the system comprises a storage and calculation fusion architecture module, a distributed cache module, a dynamic fragmentation module, a parallel calculation module, a preloading module and a resource scheduling center. The method corresponds to the system. According to the method, by deeply fusing data storage and calculation and utilizing a distributed caching technology and a data preloading mechanism, I / O bottleneck is effectively reduced, dynamic data fragmentation and parallel calculation are supported, and the method is particularly suitable for large-scale data analysis scenes.

The invention relates to the field of data processing, and discloses a cloud computing-based data processing method, which comprises the following steps of: constructing a reference count management micro-service cluster at a cloud end, receiving a reference count request carrying a count type through an API (Application Program Interface) gateway, and analyzing the reference count request into a structured operation instruction; reading a first state identifier of the referenced object and a second state identifier of the reference object from the distributed cache system according to the metadata snapshot; according to the method, through dynamic scheduling of a reference counting management micro-service cluster and a container arrangement engine, efficient allocation and scheduling of computing resources can be realized, maximization of resource utilization is ensured, resource waste is avoided, and high-speed query capability of a distributed cache system and a memory database is utilized; the state identification of the object can be retrieved at a millisecond level, the system response speed is improved, and the performance requirement under a high-concurrency scene is met.

The invention discloses a large language model reasoning method based on a distributed KV cache pool. According to the method, a platform receives requests which are sent by a large number of users and need a large language model service, the requests of the users and machines in a cluster are modeled correspondingly, and then corresponding strategies are used for processing. In addition, the use condition of machine resources in the cluster is also considered, and machines with more idle resources are preferentially considered. In this way, interference caused by resource competition is reduced to a certain extent. Meanwhile, by abstracting memories of numerous NPU cards into a distributed KV cache pool, elastic expansion and contraction are facilitated during request processing. Through the method, an efficient cluster based on a large language model can be constructed, and a corresponding platform can better understand the demands and intentions of users so as to provide more timely and personalized services.

The invention discloses a data migration method and device, computer equipment and a storage medium, and realizes a data migration processing mode oriented to an unstable network environment by combining neural network prediction, decision tree anomaly recognition and a distributed cache mechanism. Therefore, the data transmission stability and the migration task completion rate in the cross-network environment are obviously improved. Firstly, a pre-trained neural network is utilized to carry out deep prediction on a real-time network state, and a potential performance reduction trend is identified in advance, so that a migration process can actively adjust a path or a strategy before a risk occurs. And secondly, by performing decision tree identification on a data packet sending sequence, the system can position a potential interruption point based on fine-grained time sequence characteristics, and quantify the risk in an interpretable manner, so that migration control can be accurate to a specific data packet level. Finally, under the condition that the risk is unacceptable, the system automatically activates a distributed cache mechanism, and the durability and the restorability of the data are guaranteed in a redundant copy storage mode.

The invention discloses an authority management method and system for double-token decoupling and dynamic token mapping, and relates to the technical field of information security. According to the method, a user identity is verified in a unified identity management system; receiving the user identity identifier in the target subsystem and generating a local permission token based on the permission model; in the token mapping service, when it is detected that a mapping relation is not established, the user identity identifier, the target subsystem identifier and the time slice identifier are combined according to a preset sequence, a one-time random factor is inserted, and a mapping key is generated through two-time abstract calculation of different secure hash functions and key parameters; determining a unified expiration time based on a smaller value of the validity period of the double tokens, generating a mapping value by using the exclusive key of the target subsystem in an encryption manner, and storing the mapping value, the mapping key and the expiration time in a distributed cache in an association manner; and the validity of the identity token is verified and the corresponding local permission token is obtained during user access, so that cross-system security access is realized, permission data migration is reduced, and compatibility and security are improved.

The invention provides a multi-tenant multi-user login method based on enhanced OAuth2, which relates to the technical field of network security, and comprises the following steps: obtaining a key threshold through a tenant identifier, and executing OAuth2 authentication to obtain an access token; tenant fingerprint information is generated through zero-knowledge proof and homomorphic encryption, and a tenant verification token is generated through secure multi-party calculation; constructing a reinforcement learning access controller in combination with the historical access data, and generating and verifying an authority list; and finally forming a session identifier and storing the session identifier in a distributed cache. According to the invention, the authentication security is improved, cross-tenant fine-grained authority management is realized, and unauthorized access is effectively prevented.

The invention provides a distributed cache consistency system, method and device, a storage medium and a product, and is applied to a CXL switch, and a connection physical device comprises a first cache device and a CXL memory protocol-based memory extension device comprising a second cache device and a memory device. The system is composed of a virtualization aggregation layer and a cache consistency control module, the virtualization aggregation layer aggregates physical equipment as logic equipment, and the cache consistency control module intercepts operation requests, performs protocol conversion and queries directories to determine cache line states and equipment where the cache line states are located and feeds back target data. According to the scheme, the problems of delay, expansibility, communication efficiency and resource waste of the existing centralized architecture are solved. A distributed system is deployed on a CXL switch, so that processing skips and steps are reduced, and delay is reduced; a distributed architecture is adopted to avoid resource bottlenecks and improve expansibility; inter-device communication intermediate links are reduced, and the communication efficiency is improved; intermediate nodes of a data transmission path are reduced, transmission overhead is reduced, resource utilization rate is improved, and resource waste is reduced.

The invention discloses a distributed caching method and system and an acceleration method for large model reasoning service. The system comprises a confidential virtual machine cluster, a zero-trust distributed caching system and a FUSE client. The zero-trust distributed cache system comprises a data strategy service, a data engine plug-in, a distributed file system and a trusted access service. And the FUSE client is deployed at the confidential container end, and establishes secure connection with the trusted access service based on the remote proof and the identity legality confirmation proof of the corresponding confidential container so as to provide a plaintext data access service for the corresponding confidential container. According to the scheme, a distributed cache implementation scheme is provided for the confidential computing container cluster, and the problem of performance loss caused by repeatedly reading, encrypting and decrypting the same data from an external storage system is avoided while the data security is guaranteed by using the large-scale confidential container cluster; and the storage access efficiency in scenes such as model training and reasoning is improved.

The invention discloses a database access parameter real-time cooperative processing method and device based on multi-level cache, and belongs to the technical field of computer data caching. The method comprises the steps that a version number management table is configured; establishing a four-level cache architecture of a transaction-level cache layer, a process-level cache layer, a distributed cache layer and a database cache layer; receiving a transaction request, creating a transaction level cache layer and loading current version number information; comparing the version number in the transaction level cache layer with the version number in the process level cache layer; according to the version number comparison result, obtaining parameter data from the corresponding cache level according to a preset cache access priority strategy; and when the parameter change is detected, updating the corresponding version number in the version number management table, and updating the cache layer data as required. According to the method, real-time global effectiveness of parameter change is realized, parameter consistency of in-transit transaction is guaranteed, system processing performance and throughput are improved, system reliability and fault-tolerant capability are enhanced, and system resource utilization rate is optimized.

The invention provides a message processing method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to a service request, executing a distributed cache operation and generating a processing message containing a service parameter; asynchronously submitting the processing message to a message queue service; if the first submission fails, triggering an instant retry mechanism to resubmit the processing message for at least one time; if the retry mechanism fails, converting the processing message into a failure record and persistently storing the failure record in a database; and scanning the failure record in the database through a timing compensation service, regenerating a processing message and submitting the processing message to a message queue service. According to the method and the device, the requirement of manual intervention is completely eliminated, the high performance of the system is maintained while the reliability of the message is ensured through cooperation of all links, and the problems of response delay, insufficient reliability and the like existing in a traditional manual processing mode are perfectly solved.

The invention provides a request parameter value authority authentication method and system, and relates to the technical field of network security, and the method comprises the steps: creating an access control list configuration and binding a service when a gateway registers the service, building the association mapping of a consumer and the service, and setting a parameter authority list; encrypting the sensitive field by adopting an advanced encryption standard algorithm and synchronizing the sensitive field to a distributed cache; and verifying the validity of the token when the service request is received, verifying the authority of the consumer, and verifying the parameter value. According to the invention, unauthorized access can be effectively prevented, the system security is improved, and the authentication efficiency is improved through the distributed cache.

The invention relates to a server data caching and accelerating method based on edge computing, which comprises the following steps: firstly, acquiring historical access data, counting the access frequency of each type of data, and storing high-frequency to-be-accessed data to an edge node through a distributed cache; updating the data access frequency in real time, and dynamically adjusting the priority weight of the cache item based on a self-adaptive LRU algorithm; after a data access request is received, detecting whether the data is in a cache by utilizing a hash search algorithm, directly responding if the data is hit, and forwarding the data to a central server through asynchronous I / O if the data is not hit; moving the data to the head of the chain table when the data is accessed, and eliminating the tail data of the chain table when the cache reaches the capacity; preprocessing the access data; and the central server performs big data analysis in combination with Spark and a federated learning framework, stores data through an NoSQL database, and realizes data synchronization through gRPC. The data access efficiency, the cache utilization rate and the data quality are improved, the big data analysis effect is optimized, and the method is suitable for server data caching and acceleration in an edge computing scene.

The invention discloses an ICS-oriented efficient multi-source data interaction method and system, and relates to the technical field of intelligent power plant ICS data communication. According to the method, interaction variables are limited through a backboard bus agent, and the data interaction security is improved in combination with a dual-access security control mechanism; a multi-link communication architecture fusing optical fiber, 5G and industrial Ethernet is constructed, transmission tasks are dynamically allocated by using an intelligent scheduling algorithm, and a communication procedure and a coding strategy are adaptively adjusted, so that the data transmission efficiency and reliability in different network environments are improved; a distributed cache unit is deployed at each server node, and the data access delay is reduced in combination with a cache consistency regulation and an intelligent scheduling and prefetching mechanism; through an omnibearing fault perception and parallel rapid switching algorithm, rapid takeover under a fault condition is realized, and service continuity is ensured in cooperation with data consistency; the method can be widely applied to industrial control systems with high requirements on data interaction safety, real-time performance and reliability, such as intelligent power plants.

The invention provides a data stream parallel acceleration processing method and system in chip design, and relates to the technical field of chip design, and the method comprises the steps: partitioning a data stream through a preset scale, building a computing resource pool with graded computing power and a storage resource pool with graded access speed, and achieving the optimal distribution of storage resources and calculation tasks; dividing parallel processing units, and determining a data processing sequence; setting a shared cache area to establish a distributed cache synchronization network, and realizing synchronization control based on a cache identifier and a Lamport logic clock mechanism; and generating a resource competition degree matrix, and rearranging the task according to the access popularity map. According to the method, the data stream processing efficiency and the resource utilization rate of the chip are improved, and resource competition conflicts are reduced.

The invention provides an industrial chain knowledge graph dynamic updating method and system based on multi-source data, and relates to the technical field of data processing.The method comprises the steps that information of industrial classification nodes is obtained through a database query interface, multi-level sorting processing is conducted on an industrial classification list composed of the nodes, and an analysis template is initialized; when a user side selects a node, triggering a sub-tree acquisition request, querying a direct sub-industry classification list based on an industry classification list, searching a sub-industry classification node in a recursive lazy loading mode, and constructing a classification sub-tree structure; constructing a multi-level cache architecture, storing the analysis template and the classification sub-tree structure of the high-frequency access to a local memory cache, and storing the industry classification list and the classification sub-tree structure to a distributed cache; and querying and calling the template in the multi-level cache architecture, and outputting industrial classification template data, so that closed-loop synchronization from data source change to cache update to front-end interface display is realized, data consistency is guaranteed, and smooth and non-perceptual user experience is provided.

The invention relates to the technical field of information management methods, and discloses a data query management method based on dynamic cache optimization. Comprising the steps of constructing a dynamic aging adjustment engine, and automatically adjusting cache time based on data change frequency; real-time monitoring is carried out through a database trigger and log analysis; the method comprises the following steps: acquiring a historical query record, predicting query types and time possibly initiated by a user in the future through a machine learning model, before the query time is predicted, loading predicted query type data to a cache, dynamically marking hotspot data according to an access frequency, preferentially allocating cache resources to the hotspot data, and prolonging the time efficiency; constructing a multi-level cache architecture based on a local cache and a distributed cache, and acquiring data by adopting a cache degradation strategy; an enhanced Kafka message processing mechanism is introduced; the method has the advantages of improving data query efficiency and ensuring data consistency.

The invention provides an invoicing data asynchronous concurrent processing method and system based on a message queue, and the method comprises the steps: receiving invoicing application data of a business system through a tax platform, carrying out the basic format verification of the invoicing application data, asynchronously pulling an invoicing application message from a message queue through a consumption cluster composed of one or more message consumers, and transmitting the invoicing application message to a server; judging whether all the preset service rules are met or not; if all the preset business rules pass the verification, pushing the verified invoicing application data to a tax center, and initiating an automatic invoicing request; generating a final result of invoicing processing; and persistently storing a final result of invoicing processing in a database or a distributed cache, and feeding back the final result to a service system through an asynchronous notification mechanism. Complete decoupling of requests and processing is realized, the throughput and concurrency capability of the system are effectively improved, the stability and fault tolerance of the system are enhanced, the maintainability and flexibility are improved, the impact on a downstream system is reduced, and elastic expansion and automatic operation and maintenance are supported.

The invention provides a load migration method and system for balancing the energy efficiency of a server cluster, and relates to the technical field of distributed computing. According to the method, the power consumption data of the distributed cache nodes in the server cluster are acquired, and the high-power consumption nodes and the low-power consumption nodes are identified; meanwhile, access frequency data of the cache data fragments are obtained, and cross-node unified sorting is carried out to generate a global access popularity result; based on the popularity sorting and the power consumption data, determining a fragment with the highest access popularity in the high-power consumption nodes as a fragment to be migrated, and selecting a low-power consumption node with sufficient resources as a target node to form a migration decision; updating a cluster metadata mapping relation, establishing a data transmission channel, asynchronously copying data blocks in batches, and verifying the data blocks; and after completion, switching a client route, and pointing the request to a target node. Energy efficiency balance of the server cluster can be realized, the overall power consumption is effectively reduced, and the resource utilization rate is improved.

The invention discloses a data synchronization method based on edge side distributed cache, and relates to the technical field of data management. The method comprises the following steps: S1, calculating the occupational behavior entropy OBE of data, wherein the OBE is calculated through three dimensions of access frequency variance, timeliness attenuation rate and multi-source correlation Cr; according to the method, a dynamic fragmentation mechanism is constructed by introducing occupational behavior entropy OBE, the problems of low cache hit rate and back-to-source request sharp increase caused by a traditional fixed fragmentation strategy are effectively solved, and the operation principle of the method is as follows: firstly, the data activity degree is quantized based on three dimensions of access frequency variance, timeliness attenuation rate and multi-source association degree Cr; wherein the data distribution imbalance is reflected through the access frequency variance of different edge nodes of a certain type of data in a dynamic time window, the data type is combined to set an attenuation coefficient to calculate the timeliness, and the Cr represents the data association degree through knowledge graph association edge number normalization.

The embodiment of the invention provides a data caching method and a distributed caching system. The method is applied to a table extension client and comprises the following steps: dividing a column storage file into a plurality of cache data blocks according to column block data of the column storage file; respectively caching the plurality of cache data blocks into a plurality of storage blocks of a distributed cache system; and storing the mapping relationship between the plurality of cache data blocks and the plurality of storage blocks into the block metadata of the corresponding storage blocks. Through the embodiment of the invention, the problems that the existing distributed cache acceleration technology in the related technology does not support block-based access and data skip reading and the random access performance is poor can be solved, and the effect of improving the data access efficiency is achieved.

The invention discloses an Ethernet-based distributed cache coherence protocol, an implementation method, an implementation device and an implementation system. A plurality of computing nodes are connected through a packet switching network. Each computing node comprises a CPU / GPU (Central Processing Unit / Graphics Processing Unit) and a local cache thereof, and is provided with a cache agent. The far-end memory is organized in home nodes, and each home node manages a part of physical address space and is equipped with a directory controller. When the CPU of the computing node accesses a far-end memory address and does not hit in the local cache, the CA of the computing node replaces the far-end memory address and communicates with the DC managing the address through the network so as to maintain the cache consistency of the data among all the nodes. Based on a cache consistency protocol of a directory, the CXL.cache consistency of a plurality of independent computing nodes can be maintained in a low-overhead and high-reliability mode on a high-delay and lossy packet switching network, and broadcast storm caused by a monitoring protocol is avoided.

The invention discloses a graph database query acceleration method and system. The method comprises the following steps: constructing panoramic query observation data through non-intrusive log interception and structured processing; the query statement is analyzed into an abstract syntax tree, semantic normalization processing is carried out, and a unique hash value is generated; predicting a hotspot query template and high-frequency parameters thereof based on a popularity value score model, and asynchronously preloading a result to a cache; dynamically calculating adaptive survival time for each cache item according to the data change frequency and the access popularity; and constructing a multi-level cache architecture comprising a local cache, a distributed cache and a database built-in cache, and performing intelligent routing and collaborative backfilling based on a hash value. According to the method, intelligent and transparent acceleration of graph database query is realized, the query performance is remarkably improved, the database load is reduced, and the problems of redundancy, high cold start delay and difficulty in balancing cache consistency caused by grammar difference in traditional cache are solved.

The invention relates to a distributed storage cache hotspot prediction method and system based on an access mode. The method comprises the following steps: acquiring an access log to obtain original data; the gradient change of the access frequency between the adjacent data blocks is analyzed in real time to divide the hot spot range and identify continuous hot spot data blocks; constructing a time sequence based on the historical access record, and pre-judging an access hotspot in a future time period by applying a time sequence trend prediction algorithm; constructing a multi-dimensional feature vector and inputting the multi-dimensional feature vector into a machine learning prediction model to output a hotspot probability of the corresponding data object in a future time period; according to the hotspot probability and a popularity threshold value, scheduling the data predicted as the hotspot into a cache, and allocating corresponding cache levels or storage paths for the data objects with different popularity at the same time; and periodically updating the prediction model and the popularity statistics to obtain dynamic prediction and cache optimization of the hotspot data. According to the invention, the prediction of the hotspot data is realized, and the hit rate and the system performance of the distributed cache are improved.

The invention relates to a distributed cache delay optimization method and system for multiple calculation copies, and the method comprises the steps: carrying out the fragmentation processing of a to-be-processed data set, and generating a plurality of data fragments; scheduling the data fragments to different computing nodes respectively, creating a computing copy for each data fragment on the corresponding computing node, and creating a corresponding cache copy on the computing node where each computing copy is located; constructing a value evaluation function of the cache copy based on historical existence time sections of the cache copy in each computing node and a time interval between each time section and the current time point, and generating a cache weight value corresponding to each computing node according to the value evaluation function; and after the calculation copy of the target data fragment is allocated to the new calculation node, constructing a new cache copy corresponding to the target data fragment on the new calculation node. The method has the effect of improving the cache performance and the cache management efficiency.