404 results about "Access frequency" patented technology

The invention relates to the field of data query, and particularly provides a medical big data query method and system based on dynamic desensitization, a terminal and a medium, and the method comprises the steps: collecting multi-source heterogeneous medical data, carrying out semantic normalization, constructing a unique identifier of a patient, and generating standardized intermediate plaintext data; constructing an adaptive isomorphic index structure based on the data, constructing a semantic hash B + tree index for structured data, constructing a feature vector index for unstructured data, and dynamically optimizing index hierarchy and parameters according to access frequency; and after a query request is received, the permission context is analyzed, intermediate plaintext data is positioned and read by using the index, and a differential privacy protection strategy is dynamically applied according to the access permission. According to the method, the problem of balance between privacy protection and data effectiveness as well as between retrieval efficiency and storage cost in medical data processing is effectively solved, and data availability and access performance are improved while safety compliance is guaranteed.

An adaptive random access system and method with learned query optimization for compacted data files that enhances random access performance through machine learning and pattern recognition. The system incorporates a query pattern learning module that analyzes historical access patterns and user behavior to build statistical models of data usage. An adaptive estimator module improves location estimation accuracy by incorporating learned patterns rather than relying solely on mathematical calculations. A predictive boundary detector uses learned codeword patterns to more accurately identify boundaries in compacted data, reducing misalignment errors. An intelligent search engine coordinates optimization strategies including context-aware search string parsing and encoding strategy selection based on learned performance data. A dynamic codebook optimizer reorganizes sourceblock layout based on access frequencies and co-occurrence patterns to improve retrieval speed. An enhanced search cache implements predictive caching algorithms that anticipate user queries and proactively load relevant data.

The invention discloses an equipment access method, device and equipment based on address space identification. The method is applied to simplifying an equipment memory management unit and comprises the steps of establishing each single-level range table; acquiring an equipment access request, reading an address space identifier and a virtual address from the equipment access request, and matching a corresponding target range table from each single-level range table based on the address space identifier; and calculating a target physical address according to the target range table and the virtual address, and establishing equipment access based on the target physical address. By replacing a traditional multi-level page table with a single-level structure, the data structure design of address mapping is simplified, the dynamic configuration complexity is reduced, and hardware formal verification is facilitated. Through direct matching of the address space identifier, the traditional steps of analyzing the equipment identity identifier and querying the external table item are omitted, and the external memory access frequency is reduced. By calculating the target physical address and establishing access, the hardware address conversion logic is simplified, the hardware implementation cost and power consumption are reduced, and the data transmission efficiency is improved.

The invention discloses a log grading method and device, equipment and a program product, and the method comprises the steps: obtaining access parameters of a to-be-graded log, and the access parameters comprise at least two of the number of access users, the access frequency and the access timeliness; determining a comprehensive grading index of the to-be-graded log according to the access parameter; determining a target dynamic grading threshold according to historical log access data in the target first sliding time window; and determining a target grading result of the to-be-graded log according to the target dynamic grading threshold value and the comprehensive grading index. Access parameters of multiple dimensions are considered when the comprehensive grading index is determined, and the limitation that traditional grading depends on a single index is solved. Besides, the sliding time window is dynamically changed, so that dynamic grading of the to-be-graded logs is realized, and the problem that static grading cannot dynamically adapt to user access requirements is solved.

The invention relates to the technical field of distributed computing, and discloses a distributed computing modeling system design method for high-dimensional discrete data, which comprises the following steps: acquiring a feature identifier access request in a high-dimensional discrete data stream, counting an access frequency, and marking a feature identifier of high-frequency access as a high-frequency access state; constructing a local parameter copy on a non-master node physical computing unit of the distributed cluster; routing gradient update data aiming at the feature identifier to a local parameter copy closest to the network and generating a local residual tensor; calculating the direction cosine similarity of the local residual tensor and the global gradient update vector; calculating a real-time access entropy based on the access source distribution and converting the real-time access entropy into a dynamic synchronous threshold value; according to the method, through a dynamic synchronization judgment mechanism based on semantic information gain, high-frequency but invalid redundant communication is effectively inhibited, and network congestion under long-tail distribution is reduced.

The invention provides a self-adaptive cache evaluation method and device based on multi-dimensional evaluation parameters and a medium, and relates to the technical field of data storage. The method comprises the following steps: acquiring an evaluation access frequency f of to-be-cached data, evaluation semantic popularity h and evaluation space-time demand density d of the to-be-cached data by a to-be-cached terminal, and evaluation terminal performance e of the to-be-cached terminal; obtaining an access frequency evaluation weight alpha, a semantic popularity evaluation weight beta, a space-time demand density evaluation weight gamma and a terminal performance evaluation weight delta; and obtaining the cache value of the to-be-cached data in the to-be-cached terminal, wherein the cache value is equal to alpha * f + beta * h + gamma * d + delta * e. According to the method, the cache value of the to-be-cached data in the to-be-cached terminal is evaluated from the four dimensions of the access frequency, the semantic popularity, the space-time demand density and the terminal performance, deep demands such as semantic association, space-time distribution and terminal suitability are considered while high-frequency access is captured, resource mismatching is effectively avoided, and the cache content value can be evaluated more accurately.

The invention relates to the technical field of big data storage, and discloses a multi-level data storage and mixed query method and system, a storage medium and equipment, and the method comprises the steps: collecting original buried point data, carrying out real-time stream processing, storing the processed real-time data in a hot storage layer, carrying out timing offline batch processing on historical buried point data, and storing the processed real-time data in a hot storage layer; converting into an optimized storage format and storing in an object storage system; an extended metadata management system is constructed, and structure information, partition information and access frequency information of data in the object storage system are automatically extracted and managed; executing query, and reducing the data scanning amount through a query optimization strategy; automatically migrating data among different storage layers through a dynamic cold and hot data layering mechanism; a unified query interface is provided, a query request is distributed to a corresponding real-time processing system or an offline query system based on an intelligent routing strategy, and a query result is cached and returned, so that high-performance and low-cost storage and second-level query of data are realized.

The invention provides an intelligent grading and excess subscription management system and method for a GPU video memory, and relates to the technical field of GPU video memory management, and the method comprises the steps: firstly building a grading storage system comprising a first performance region and a second performance region; then receiving a video memory allocation request containing task priority and service quality requirements, and monitoring a data access mode and access frequency; the initial placement position of the data block is dynamically determined through the intelligent engine according to the task priority, the data access mode and the frequency; predicting target access probability distribution of each data block by adopting an LSTM network; then migrating data between the first performance area and the second performance area through a swap-in and swap-out mechanism based on the distribution and periodicity characteristics, and ensuring the service quality of the first performance area; finally, the global oversale proportion is dynamically adjusted through an overpurchase safety management module, an independent virtual address space is distributed for each task, and safe overpurchase can be achieved through the process.

The invention discloses a flow scheduling method and device, a medium and a program product, and relates to the technical field of heat dissipation, and the method comprises the steps: obtaining a video memory access frequency as a main parameter when a parallel computing function is executed, matching with other auxiliary parameters related to heat consumption, distributing a weight, determining a heat load scoring result, and then adjusting a cold plate micro-channel flow distribution coefficient, and determining an opening value of a flow control valve and sending a flow scheduling instruction to the liquid cooling system so as to dynamically adjust the cooling liquid flow of each cold plate micro-channel. In this way, accurate temperature control and heat dissipation of the video memory area of the computing device can be achieved, energy waste caused by unnecessary circulation of cooling liquid is avoided, the heat dissipation efficiency is remarkably improved, energy consumption is reduced, and the service life of the computing device is prolonged; in addition, the mode can autonomously complete full-flow operation such as video memory heat consumption monitoring, score calculation and flow adjustment, intelligent management of the heat dissipation process is achieved, manual operation errors and cost are reduced, and the stability and reliability of operation of computing equipment are guaranteed.

According to the method, PCIe extended storage equipment is identified by responding to a system starting signal, and initialization and address mapping control logic is executed to construct a first address space addressable by a GPU (Graphics Processing Unit); then, loading model data according to an AI task instruction, and dynamically dividing hot, warm and cold multi-level data popularity levels by monitoring memory access frequency; a migration data block is generated based on the popularity level, a migration data flow is organized and verified through priority scheduling logic, and intelligent data scheduling between extended storage and a GPU video memory is achieved; finally, the resources are released after the task is completed. Therefore, dynamic expansion of the physical video memory capacity is effectively achieved, large-scale AI model localization training is met, dependence on a high-end GPU chip is remarkably reduced, and data safety and system economy are improved.

The invention belongs to the field of computer system structures and artificial intelligence accelerators, and discloses a software and hardware collaborative heterogeneous storage computing accelerator oriented to full-chip DLRM reasoning. A control domain composed of a static random access memory and an embedded dynamic random access memory, an on-chip storage domain, an in-memory computing domain and an on-chip computing domain are integrated in the same chip, an embedded table is compressed by introducing a hierarchical clustering quantization frame, and a dynamic storage management strategy of access frequency perception is combined. The embedded access path is shortened, and the bandwidth requirement is reduced. According to the method, the reasoning process of the whole set of DLRM can be independently completed, and the memory access overhead in a traditional terminal SoC is remarkably reduced by reducing cross-domain data migration between the calculation unit and the storage unit, so that the model reasoning performance and energy efficiency are improved.

The invention relates to an Agent operation system and method oriented to a long-term task and driven by a task state, and belongs to the field of artificial intelligence. A state system is used for explicit modeling and unified management of an overall cognitive state; the context constructor is used for constructing a structured context matched with the current decision stage; the decision engine is used for outputting a structured decision intention; the execution and tool layer converts the structured decision intention output by the decision engine into executable operation; the memory management module is used for realizing memory enhancement, fading and hierarchical storage according to the access frequency of the context builder; and the verification and control module is used for performing consistency verification and process control on the decision result and the execution result. According to the method, structured management of task states, accurate control of task mainlines, controllability of context construction and interpretability and auditing of decision-making processes can be realized, and reliability and applicability of an Agent system in long-term, multi-tool and complex task scenes are improved.

The invention discloses a multi-layer memory management method and device, a storage medium and electronic equipment, and relates to the technical field of memory management, and the method comprises the steps that a prediction frequency set of storage equipment is obtained through prediction by using a reference information set corresponding to the storage equipment, and the storage equipment comprises a first memory and a second memory; the first delay of the first memory is smaller than the second delay of the second memory; determining a plurality of target sub-data contained in the target data of the storage device, and determining predicted access frequencies corresponding to the plurality of target sub-data based on the predicted frequency set; under the condition that the predicted access frequencies corresponding to the multiple pieces of target sub-data meet a preset splitting condition, splitting the target data into multiple pieces of target sub-data; and storing the target sub-data with the predicted access frequency greater than the frequency threshold in the first memory, and storing the target sub-data with the predicted access frequency less than the frequency threshold in the second memory. The technical problem that a memory management method in the related technology is low in resource utilization rate is solved.

The invention relates to a distributed storage cache hotspot prediction method and system based on an access mode. The method comprises the following steps: acquiring an access log to obtain original data; the gradient change of the access frequency between the adjacent data blocks is analyzed in real time to divide the hot spot range and identify continuous hot spot data blocks; constructing a time sequence based on the historical access record, and pre-judging an access hotspot in a future time period by applying a time sequence trend prediction algorithm; constructing a multi-dimensional feature vector and inputting the multi-dimensional feature vector into a machine learning prediction model to output a hotspot probability of the corresponding data object in a future time period; according to the hotspot probability and a popularity threshold value, scheduling the data predicted as the hotspot into a cache, and allocating corresponding cache levels or storage paths for the data objects with different popularity at the same time; and periodically updating the prediction model and the popularity statistics to obtain dynamic prediction and cache optimization of the hotspot data. According to the invention, the prediction of the hotspot data is realized, and the hit rate and the system performance of the distributed cache are improved.

Provided in the embodiments of the present disclosure are an address translation method and apparatus, and an electronic device and a storage medium. The address translation method is applied to an address translation apparatus, the apparatus comprising multiple sets of translation lookaside buffers, wherein each of the multiple sets of translation lookaside buffers is configured to store page table information for translating a virtual address into a physical address. The address translation method comprises: in response to obtaining a first virtual address translation request, determining a target translation lookaside buffer from multiple sets of translation lookaside buffers on the basis of a first mapping algorithm, wherein the first virtual address translation request comprises an object virtual address; and accessing page table information in the target translation lookaside buffer, in order to translate the object virtual address into an object physical address. The method further comprises: acquiring access frequency information for each of the multiple sets of translation lookaside buffers; on the basis of the access frequency information, determining whether the first mapping algorithm is required to be updated; and in response to the first mapping algorithm being required to be updated, updating the mapping algorithm to a second mapping algorithm.

The invention provides an intelligent caching method and device based on a self-adaptive caching strategy, and the method comprises the steps: collecting an access event flow of each caching key in real time through a sensing probe embedded in a caching client, the event flow comprising Key, operation type, timestamp, request source, hit state and response time; running a streaming processing engine at a client based on an access event stream, and calculating indexes such as instantaneous QPS, continuous miss times and short-term access frequency by adopting a sliding time window; a calculation result is input into a rule engine for online matching, a decision instruction for a single Key is generated, and the decision instruction comprises distributed mutual exclusion lock addition, local TTL dynamic adjustment and read-write mode switching; a decision instruction is directly executed in a client memory through a strategy executor, and nanosecond strategy effectiveness is achieved; and finally, collecting system feedback data after strategy execution, inputting the system feedback data into a machine learning module for online learning, and dynamically optimizing a rule threshold value or recommending a new strategy.

The invention discloses a hotspot sensing hierarchical storage and computing resource scheduling method oriented to multi-modal data, and belongs to the technical field of data center resource management. The core of the method is as follows: firstly, monitoring characteristics, access modes and computing node loads of multi-modal data in real time; secondly, a comprehensive popularity evaluation model is used for dynamically calculating the comprehensive popularity value of the data object, and the model comprehensively considers the data type, the access frequency, the data size and the task priority; then, intelligently scheduling the data to different performance levels of the hierarchical storage system according to the popularity value; and finally, when a calculation task is scheduled, synchronously sensing the heat and the position of required data, and carrying out collaborative scheduling by taking optimal data locality and load balance as targets. According to the method, storage and computing resources are managed cooperatively through the unified heat index, the problem of resource scheduling disjunction in a multi-modal data environment is effectively solved, and the system throughput rate, the response speed and the resource utilization efficiency are remarkably improved.

The invention discloses a path mark-based mandatory access control method and system during container operation. The method comprises the following steps of: generating a security mark for a container process and a file; executing dynamic access control, and dynamically starting a security model according to the file marking field; dynamic mark adjustment is implemented, the file access frequency is counted in real time through a kernel layer, and when overrun access is detected, if the process confidentiality level is higher than the lowest level, the confidentiality level is degraded; if the process is the lowest level, the BLP model verification permission of the process is temporarily forbidden; all strategy updating and audit log recording are completed locally, and it is ensured that the operation track is traceable; the system comprises a path marking module, an access control module, a dynamic regulation and control module, a strategy management module and a log auditing module. According to the method, error interception is reduced, millisecond-level real-time blocking of high-frequency abnormal access is realized, and the safety protection capability and the operation and maintenance efficiency of the container platform are remarkably improved.

The invention provides a data compression and transmission method and system based on an HTTP (Hyper Text Transport Protocol), and relates to the technical field of network data transmission, comprising the steps of analyzing semantic hierarchy, redundancy mode and access frequency characteristics of a data object, constructing a multi-granularity compression strategy matrix and a deduplication index structure, and realizing compression coding of data. According to the method, the HTTP data transmission efficiency can be effectively improved, the network bandwidth consumption is reduced, and meanwhile, the integrity and accuracy of data transmission are ensured.

The invention relates to the technical field of information integrated management, in particular to a hospital information integrated management system, which comprises the following steps of: acquiring communication time delay, bandwidth and queuing cost among service nodes, and performing comprehensive calculation and optimal path screening on multiple paths by using a Dikstra algorithm; according to the method, cost-minimized dynamic path planning is realized, a shortest cost path can be efficiently determined under a complex network topology, and failure range calculation and multicast synchronization are executed through hit judgment and a conflict key value updating algorithm in combination with a segmented transmission and block allocation strategy. In the access behavior analysis, trend prediction is performed by counting the access frequency of medical records and prescription sequences and combining a long-short-term memory network model, and the access priority and the capacity threshold are dynamically adjusted, so that the balance of task scheduling and the real-time performance of response are kept during load fluctuation, a traceable cluster recovery index is formed, and the service life of the cluster is prolonged. And stable transmission and intelligent scheduling of cross-system data can still be kept in a complex environment.

The invention relates to the technical field of artificial intelligence, in particular to a load balancing method and device, electronic equipment, a storage medium and a program product, and the load balancing method comprises the steps: obtaining traffic characteristics of communication links between different nodes in a data block migration process of a distributed storage system; determining a load distribution mode of each communication link based on the flow characteristics of each communication link; based on the flow characteristics and the load distribution mode of each communication link, determining hotspot nodes and corresponding service priorities thereof; determining a target hotspot node based on the access frequencies, the data transmission quantities, the node loads and the corresponding service priorities of all the hotspot nodes; and based on the processing performance, the network state and the storage capacity of the node, screening out a target low-load node meeting a preset condition from the distributed storage system, and migrating data of the target hotspot node to the target low-load node. According to the invention, the network flow distribution and node load balancing state can be dynamically optimized.

The invention relates to a PCDN optimization method and system based on intelligent scheduling and dynamic management, and belongs to the technical field of distributed computing and edge network scheduling. The method comprises the following steps: collecting hardware resource state data of a node, and generating a node resource scheduling signal through a content demand optimization scheduling algorithm in combination with a user geographic position and a content demand priority; constructing a multi-level cache priority system to distribute an optimal node matching and data transmission path according to the hotspot contents of different time periods and regions in combination with the content timeliness and the user access frequency; analyzing a dynamic relationship between each node load and a resource utilization rate through a path cross modeling scheduling method, and allocating an independent scheduling window; a point-to-point content distribution network authorization feature library is constructed to classify and identify node traffic, a stepped suppression strategy is executed on unauthorized traffic according to network real-time load, and traffic transmission is limited in low load and blocked in high load according to network node load conditions.

The embodiment of the invention discloses a key value indexing method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to an access operation for a specific key, and determining heat information of the specific key; the popularity information comprises the latest access frequency and the latest access timestamp of a specific range to which the specific key belongs; the specific range is obtained by dividing the whole key value space; determining a target data layer to which the specific key belongs according to the popularity information and set hierarchical parameters; the target data layer comprises a hot data layer, a temperature data layer or a cold data layer; and indexing based on an indexing strategy of the target data layer, and determining a specific value corresponding to the specific key. By applying the key value indexing method provided by the embodiment of the invention, the performance of data storage can be improved.

The invention provides a method for managing a dynamic data directory and constructing the dynamic data directory in a data knitting environment, which comprises the following steps of: carrying out classification and normalization and metadata extraction on multi-source heterogeneous original data in the data knitting environment to generate a structured original data set and an associated metadata table; constructing a dynamic data directory initial skeleton according to the structured original data set and the associated metadata table in combination with the access frequency and the association relationship of the data in the data weaving environment; on the basis of the dynamic data directory initial skeleton, real-time mapping logics are configured for different data types in the structured original data set to generate a data-directory mapping rule base; and performing real-time change monitoring on the structured original data set based on the data-directory mapping rule base, and triggering a corresponding rule in the data-directory mapping rule base to execute directory node updating so as to generate an available dynamic data directory adaptive to the data weaving environment. According to the method, invalid consumption of computing resources and time is reduced, and unnecessary resource occupation is avoided.

A method for migrating memory data, including obtaining a sampling result by sampling obtained access requests according to a sampling rule, where the access request indicates an operation performed by an application run by a processor in a computer device on a memory medium, where the processor is associated with at least two memory media of different types, the sampling rule indicates a periodic sampling mode, collecting statistics on access frequencies of pages indicated by the sampling result, where the access frequencies of the pages are associated with determining access temperatures of the pages, and triggering migration of memory data between the at least two memory media of different types based on the access frequencies.

The invention discloses a multi-modal corpus storage method and device, computer equipment and a medium, and the method comprises the steps: carrying out the compression processing of text modal data and audio and video modal data of a multi-modal corpus, and employing a suitable compression algorithm for the characteristics of different modal data; similarity calculation is carried out on the text compression features, the audio and video compression features and corresponding modal features of the newly-added multi-modal corpus, semantic redundancy of the newly-added multi-modal corpus and historical compression corpus can be recognized, a fusion compression process is triggered to eliminate cross-modal redundancy, and repeated storage of same semantic information is avoided; corpus features which do not trigger the fusion process are independently stored, the corpus features which trigger the fusion process are fused to generate multi-modal joint features, and the stored data volume is further simplified; the corpus features are migrated to the corresponding data storage layer in combination with the access frequency and the response time, so that the storage cost is reduced.

The invention discloses a service-driven log intelligent hierarchical storage method and system. The method comprises the following steps: associating multi-dimensional log data based on a global request identifier to form an associated log; determining the storage level of the associated log according to the calculated service sensitivity value and the access frequency predicted value; according to the determined storage level, the associated log is routed to a corresponding storage system to be stored; and constructing a multi-modal index for the associated logs stored at different storage levels, and carrying out distributed associated retrieval on the associated logs. According to the invention, based on the global request identifier and the multi-dimensional association log, the log association problem under complex business scenes such as cross-platform file transmission, single sign-on and cross-end jump is solved; through a multi-modal index and a distributed association retrieval mechanism, the retrieval efficiency is remarkably improved; and on the basis of a dynamic hierarchical storage and intelligent compression strategy driven by business value, on-demand distribution of storage resources is realized, and the storage cost is optimized.

The application belongs to the technical field of machine room power distribution, and provides a machine room power distribution equipment remote monitoring method and system, which comprises the following steps: collecting operation data in real time through deploying edge equipment, fusing basic operation data, transient characteristic data and historical abnormal data to construct a state scoring model, and utilizing neural network training weight coefficients to realize dynamic evaluation of equipment health degree; based on the scoring result, the sampling frequency is adaptively adjusted, combined with current mutation intensity, energy mutation amount, temperature gradient and other multi-feature threshold values, abnormal data is screened, and only abnormal period high-frequency data segments are reserved; through edge-cloud collaborative storage mechanism, after abnormal data is subjected to RAID redundancy check, the abnormal data is locally cached, normal data is uploaded to the cloud according to storage time length and access frequency, data full life cycle traceability is realized through labeling processing, abnormal detection sensitivity is improved, and storage efficiency and fault diagnosis reliability are taken into account.

The invention discloses a hot and cold data hierarchical processing method and system for an industrial internet, and relates to the technical field of industrial internet data processing, and the method comprises the steps: constructing a first-stage hot library through a memory library meeting a first preset access speed, and constructing a second-stage hot library through a time sequence library meeting a second preset access speed, the refrigeration house is constructed through the time sequence library meeting the preset capacity; splitting the industrial internet data according to a preset particle dynamic segmentation strategy to obtain particle segmentation data; judging the particle segmentation data as to-be-reheated data or to-be-cooled data according to a preset access frequency threshold value and the particle segmentation data access frequency; and according to the to-be-reheated data and the to-be-cooled data, data migration in the first-stage heat reservoir, the second-stage heat reservoir and the cold storage is carried out. The method has the beneficial effects that the dynamic cold and hot bidirectional switching of the particle segmentation data is realized, the storage position of the cold and hot data is ensured to adapt to the access requirement of a user, and meanwhile, the data processing efficiency of the industrial internet is ensured without complicated calculation.

The invention discloses a computing power center capacity expansion method based on distributed storage and mixed data archiving, and relates to the technical field of computing power capacity expansion. Data can be subjected to fragmentation management according to the difference of service loads through establishment and updating of an access frequency label, and an archiving snapshot mechanism is combined, so that the capacity expansion efficiency is improved. Rapid data migration and continuous storage and writing under the condition that the storage nodes are abnormal are realized, so that the fault-tolerant capability and the system stability in the capacity expansion process are effectively improved; meanwhile, by means of logical sequence splicing and archiving index construction of the low-access-frequency fragmentation units, the storage pressure brought by redundant data is reduced, and the efficiency of archiving retrieval and calling is improved; in addition, when the access frequency of the fragmentation units changes, an intelligent relocation strategy based on delay, load and topological distance can be realized, and quick response and balanced distribution of high access data are ensured.