710 results about "Thread pool" patented technology

The invention provides an intelligent analysis and execution method and system for database cascading operation, and relates to the technical field of database management.The intelligent analysis and execution method comprises the steps that a target table and an association table are extracted through a recursive query algorithm, field mapping analysis is carried out to generate a dependency weight matrix, cyclic dependency is recognized based on the matrix and decomposed into directed acyclic subgraphs, and the directed acyclic subgraphs are analyzed; node depth values and breadth values are calculated for topological sorting to generate an execution sequence; performing node classification on the execution sequence, organizing the execution sequence into a batch processing task group, calculating an optimal execution path through an adaptive path optimization algorithm, generating a distributed transaction control instruction, and allocating the task group to an execution thread pool for execution by using a consistent Hash algorithm; monitoring an execution process, dynamically adjusting thread pool resources according to a performance prediction value, executing cascade rollback when an exception occurs, and reconstructing a transaction execution path; according to the method, the execution efficiency and stability of the database cascade operation can be effectively improved, and the system resource consumption is reduced.

The invention relates to the technical field of electric data processing, and discloses a system and a method for hybrid processing of mass offline data and mass real-time data, and the system comprises a queue management module which receives task requests, identifies and marks task types, distributes tasks to a stream processing queue or a batch processing queue, and sorts the tasks according to priorities; the task types comprise flow tasks and batch tasks; the thread pool management module establishes a thread pool, sequentially processes tasks according to priorities and collects real-time parameters; the calculation module is responsible for calculating task priorities and calculating task jitter factors based on real-time parameters; the scheduling decision module compares the task jitter factor with a reference value, determines a weight adjustment coefficient, corrects the priority and formulates a scheduling strategy; the alarm module judges whether to trigger an alarm. According to the invention, by introducing the dynamic priority scheduling and preemptive processing mechanism driven by the jitter factor, the real-time response capability and the resource utilization efficiency of the system are improved.

The embodiment of the invention provides a thread pool operation state monitoring method and device, and the method comprises the steps: dividing the fluctuation range of the ratio of the number of active threads to the number of core threads into a high-load interval, a normal interval and a low-load interval, and calculating the optimization coefficient of the core threads based on the interval distribution characteristics; calculating the variance of the ratio of the total thread count to the maximum thread count, and extracting the adjustment coefficient of the maximum thread through the fluctuation characteristics of a variance curve; taking a difference value between the task growth rate and the processing rate as a calibration factor of the queue capacity; respectively comparing the evaluation results of the three parameters with a preset reasonable interval boundary to generate a parameter rationality score; counting the performance improvement effect of each preset adjustment strategy from a historical adjustment record; calculating a current adjustment step length based on the reference strategy; recording the adjustment suggestions and the calculation basis thereof into a version library, and setting corresponding effective conditions; according to the application, the efficiency and accuracy of monitoring the running state of the thread pool can be improved.

The invention belongs to the technical field of data transmission and processing, and discloses a zero-copy-based multi-thread high-throughput data stream channel separation method and system, and the method comprises the steps: directly writing a mixed data stream into a front-end buffer region configured as an annular structure through a data receiving module by adopting direct memory access; then, a multi-thread processing module dynamically allocates a plurality of processing threads from a thread pool to separate channel data in parallel, each thread adopts a zero copy algorithm based on pointer offset, positions the channel data in a memory, creates pointer reference and associates the channel data to a corresponding rear-end buffer area, and logic separation is achieved without physical copy; and finally, the data storage module efficiently writes the separated data into persistent storage in an asynchronous I / O mode. According to the method, zero-copy, multi-thread parallel and two-stage dynamic buffering strategies are combined, the data separation efficiency is remarkably improved, CPU occupation and memory bandwidth are greatly reduced, and the real-time performance and stability of high-throughput data processing are guaranteed.

The invention discloses an emergency fire fighting question answering method and system based on the AI large language model technology and a storage medium, and relates to the technical field of artificial intelligence, the emergency fire fighting question answering system comprises a front-end module, an AI preprocessing thread pool, a knowledge base module and a large model calculation engine module, the AI preprocessing thread pool receives a service request sent by the front-end module, and sends the service request to the AI preprocessing thread pool; different task threads are distributed, a knowledge base of a corresponding region category is automatically called according to the region category input by a service request to perform content similarity retrieval, knowledge content data obtained through retrieval and the corresponding service request are sorted and then uniformly sent to the large model calculation engine module supporting parallel processing to perform analysis and calculation, and the large model calculation engine module is used for performing analysis and calculation on the knowledge content data and the corresponding service request. By using the technical means of the invention, the user does not need to think what the keyword of the current question is, and can completely organize the question, directly ask the question, automatically query the local professional knowledge base and give professional and accurate answers according to the actual situation of the site, so that real barrier-free communication is realized.

The invention relates to the technical field of computers, can be applied to the field of science and technology finance / digital medical treatment, and discloses a method and device for dynamically adjusting threads of a thread pool, electronic equipment and a storage medium. The method comprises the steps that initialization setting is conducted on thread pool parameters through a management center of a target service system; when a thread pool parameter adjustment request is received, generating a first thread pool parameter according to the thread pool parameter adjustment request; receiving a configuration change event through a configuration change monitor, and analyzing the configuration change event to obtain a first thread pool parameter; positioning a thread pool configuration field of the distributed service instance according to a mapping relation between a program entity factory and a parameter cache; injecting the first thread pool parameter into a thread pool component instance of the distributed service instance; and calling the setting function of the distributed service instance, and resetting the corresponding configuration parameters in the thread pool component of the distributed service instance. The method improves the adaptability of the system to load change, and breaks through the system performance bottleneck.

The invention discloses a three-area three-line database automatic quality inspection evaluation method based on deep learning. The method comprises the following steps of S1, obtaining three-area three-line database data and performing preprocessing; s2, checking a directory structure and a file name of the original data set; s3, extracting spatial layer data from the hierarchical data set; s4, performing segmentation and attribute information extraction on various pattern spots in the layer data after spatial verification; s5, inputting the pattern spot relation characteristic data into an artificial fish swarm algorithm; s6, automatically comparing the pattern spot area and distribution data with a quality inspection standard preset in a three-area three-line database; and S7, performing parallel processing on an automatic comparison result through a multi-thread thread pool. According to the method, an artificial fish swarm algorithm, a relational attention network and a task allocation and thread pool management technology are combined, and three-area three-line database automatic quality inspection evaluation based on deep learning is realized.

The invention relates to a railway material purchasing asynchronous task processing method, device and system, and the method comprises the steps: constructing a six-layer message channel architecture of a purchasing business process, and executing an instruction of deploying a message agent node in an active-active mode for each channel based on the core message middleware of a Kafka cluster; the front-end gateway is instructed to package the purchase event into a binary message body with a transaction ID and a timestamp, and the binary message body is written into a persistent queue after priority labeling is carried out according to a task type label; processing the concurrent tasks by adopting a multi-stage feedback queue scheduling algorithm, and dynamically allocating computing resources based on a thread pool; a maximum retry threshold value is set according to an exponential backoff retry strategy, so that the failed task is automatically transferred into a dead message queue; and dynamically calculating an optimal message routing path based on the supplier portrait and the material classification model, and extracting the order to an exclusive processing queue of the optimal supplier according to Topic dynamic switching of the message queue. According to the invention, the coupling degree is reduced, the system performance is improved, and the fault tolerance is enhanced.

The invention relates to a lightweight asynchronous message processing method and system for a high-concurrency scene, and the method comprises the steps: receiving a message request of a client, carrying out the preprocessing of the message request, and generating a to-be-processed message; distributing the to-be-processed message to a multi-level priority queue based on a priority scheduling mechanism; dynamically adjusting the number of core threads and the maximum number of threads of the thread pool according to the resource state data, and forming a thread pool resource configuration scheme of the current scheduling period; dynamically calculating the message processing quantity of the current scheduling batch based on the message accumulation quantity of each priority queue, the thread pool resource configuration scheme, the CPU utilization rate in the resource state data and the JVM heap memory utilization rate; and based on the out-of-heap memory buffer area reference, assigning the to-be-processed message object to a corresponding processing thread to execute a service processing logic until the processing operation of the messages in all the scheduling units is completed. The method has the effect of improving the resource adaptability of the asynchronous message processing architecture.

The invention provides a method and system for dynamically exporting Excel based on asynchronous streaming block data, and belongs to the technical field of large data volume export. Comprising the following steps: receiving a user export request, and generating a unique request ID; an export task is generated based on the export request, a task ID is returned, and an asynchronous response is made; submitting the task to a thread pool queue, and obtaining block data from a database in batches through streaming query; wherein the core thread count and the maximum thread count of the thread pool are adjusted based on a dynamic resource allocation method, and load change is smoothed in combination with a sliding window algorithm; writing the block data into an Excel file batch by batch by adopting a streaming writing technology; and after the task is completed, pushing a notification to the user side. According to the method, the rapidity, the availability and the safety during data export can be ensured when large data volume export is performed.

The invention discloses a cloud hard disk data backup and recovery method and device, electronic equipment and a storage medium, and relates to the technical field of cloud computing. A cloud hard disk is segmented into a plurality of segmentation blocks and a multi-thread parallel processing mode is adopted, a backup and recovery task of a large-capacity cloud hard disk is decomposed into a plurality of sub-tasks to be processed at the same time, and the data backup and recovery efficiency is improved. The operation time is greatly shortened, the number of threads in the thread pool is determined according to the number of the segmentation blocks and the current available computing resources, the utilization rate of the computing resources can be improved, cloud hard disk segmentation strategy parameters can be flexibly configured according to service requirements and cloud hard disk characteristics, data protection requirements in different scenes are met, and the user experience is improved. Consistency verification is carried out on the transmitted data, so that the integrity and consistency of the data in the backup and recovery process are ensured, the data loss risk is reduced, and the purposes of improving the backup and recovery efficiency of the cloud hard disk and enhancing the system reliability and the computing resource utilization rate are achieved.

The invention relates to the technical field of data acquisition, in particular to an AOP-based user behavior data acquisition method. The method comprises the following steps: firstly, identifying user behavior related code features by statically analyzing Java source codes of an application program, and constructing a feature template library; during a compiling period, according to a template library identification target method, embedding a point burying instruction by utilizing a byte code enhancement technology, and constructing a monitoring section framework; and during running, dynamically adjusting an execution strategy according to the running state of the application program by virtue of a dynamic rule engine. Meanwhile, an annular buffer area is adopted to asynchronously process user behavior data streams, cache data are managed in a sub-generation mode, and asynchronous processing is triggered when a threshold value is exceeded. The method also constructs an adaptive thread pool model based on a hierarchical thread pool architecture, distributes tasks according to data value weights, monitors the task backlog rate of a high-priority sub-pool, adjusts the sampling frequency or acquisition dimension when the task backlog rate exceeds a threshold value, forms adaptive closed-loop control, and effectively acquires and processes user behavior data.

The invention provides a pathological specimen auxiliary diagnosis system based on multi-thread control and task scheduling, and the system comprises a distributed lock management module which is used for employing a distributed lock to judge the operation state of a resource needed by a task based on a task initiation request, and distributing the lock and starting task execution when the needed resource is not locked; the timed task scheduling module is used for regularly scanning the task queue and distributing periodic tasks and delayed tasks in the task queue to a thread pool based on a scanning result; the multi-thread task execution module is used for performing multi-thread task scheduling and control on the periodic tasks and the delayed tasks according to a priority mechanism on the basis of the distribution result, and performing data processing on the periodic tasks and the delayed tasks on the basis of a multi-thread task scheduling and control result; and the loop operation management module is used for releasing the lock after the execution is finished, and notifying the idle thread to continuously process the to-be-processed task in the task queue based on a release result. And the efficiency and the effect of auxiliary diagnosis of the pathological specimen are ensured.

The invention relates to the technical field of data management, and discloses a dynamic partitioning and priority scheduling method based on time sequence data, which comprises the following steps: collecting time sequence characteristics of an industrial data source through an edge gateway and carrying out structured storage in Redis, the time sequence characteristics comprising collection frequency, historical peak time period and service priority; load indexes are calculated based on time sequence characteristics, and a distributed message system is driven to execute dynamic partition adjustment of pre-judgment type capacity expansion, hot spot precise splitting and low-peak intelligent capacity reduction; and full-link data priority scheduling is realized through production end route isolation, Broker end resource inclination and consumption end thread pool isolation. According to the invention, the dynamic adaptation of the distributed message system resource and the industrial time series data load is realized, the real-time performance of high-quality data is ensured, the integrity of data transmission and the system adaptability are improved, and the problems of poor load fluctuation adaptation, high-quality data delay and easy data loss due to faults in the prior art are effectively solved.

The invention discloses an input and output request processing method and device and electronic equipment, and belongs to the technical field of electronic equipment. The method is applied to the electronic equipment, a first execution environment of the electronic equipment comprises a first working thread pool, and the first working thread pool communicates with a first application running in a second execution environment of the electronic equipment through a first task queue and a first result queue. The first working thread pool comprises a plurality of working threads used for processing IO requests in the first task queue. The method comprises the following steps: working threads in a first working thread pool respectively obtain IO requests from a first task queue; and for a first IO request obtained by the first working thread in the first working thread pool, the first working thread executes the first IO request, and writes an obtained first request result into a first result queue. Through the method, the efficiency of processing the IO request through the first execution environment by the first application in the second execution environment of the electronic equipment can be improved.

The invention provides an elastic thread pool management method and device based on load prediction and electronic equipment, and the method comprises the steps: obtaining a task submitted by a client, and routing the task to a corresponding queue according to the task feature of the task; wherein the I / O type task is routed to an asynchronous queue, and the CPU calculation type task is routed to a thread pool queue; monitoring and collecting thread index data of the thread pool queue in real time, and obtaining thread index data of a first preset time period based on a preset sliding window; inputting the thread index data of the first preset time period into a prediction model to obtain a predicted load value of a second preset time period; wherein the prediction model is a Holt-Winters model, and the prediction model is a Holt-Winters model; and performing thread adjustment based on the predicted load value and a preset load threshold value. The problems existing in an existing thread pool management scheme are solved.

The invention discloses an unmanned ship-oriented task priority scheduling and resource awareness management method, and belongs to the technical field of unmanned ship control. The method comprises the steps of obtaining a task request generated by an unmanned ship system, inputting the task request into a task classifier for task classification and priority allocation, performing real-time monitoring on resources of the unmanned ship system based on a resource perception and threshold grading model, judging an alarm interval where the resources of the current unmanned ship system are located, and sending the alarm interval to the unmanned ship system. Obtaining a target scheduling strategy; allocating each task in the task request to a corresponding thread pool for execution, dynamically adjusting the scheduling of each task according to a multi-thread pool cooperation and preemption mechanism, and dynamically optimizing the load and channel of the unmanned ship system based on a load balancing and channel selection mechanism; feedback information during execution of each task is collected, a target scheduling strategy is adjusted through an adaptive optimization mechanism, and scheduling and resource allocation of each task are updated in real time. The method improves the resource utilization rate of the unmanned ship system.

The invention relates to a multi-modal interaction-oriented automatic operation backtracking and digital person continuous talking control method and system, and the system comprises an operation snapshot module which records operation nodes through a semantic anchor point marking technology, such as an interface XPath page number and a PPT page number; the interruption detection module is used for capturing a user interruption instruction in real time; the progress prediction engine dynamically adjusts a follow-up demonstration path based on a program; and the continuous talk controller is used for coordinating RPA operation execution, digital human rendering and voice synthesis through a three-level priority thread pool. The problems of inaccurate operation flow backtracking and multi-modal thread conflict after demonstration interruption are solved, the breakpoint continuous talk error is obviously lower than that of a traditional scheme, and the method is suitable for PPT demonstration, cockpit explanation, business system training, course explanation and 3D modeling guidance scenes.

The invention provides a search task processing method, electronic equipment and a storage medium, and relates to the technical field of task processing, the method concurrently executes a core sub-task and all edge sub-tasks based on a thread pool technology, when a return result of the core sub-task is received, a time point at the moment is recorded as a key update time point, and the key update time point is updated; and copying the initial return result list at the moment as a first return result list, when the first return result is NULL, indicating that the edge sub-task corresponding to the first return result is not completed, obtaining candidate waiting duration corresponding to the identifier of the uncompleted sub-task, and taking the maximum candidate waiting duration as target waiting duration, and when the time point obtained by adding the key update time point and the target waiting duration arrives, all the initial return results in the initial return result list at the moment are integrated to obtain the final return result corresponding to the search task, so that the overall search time is shortened, and the overall search efficiency is improved.

The invention discloses an ROS2 hybrid scheduling actuator system. The system comprises a registry, two ready callback sets, two schedulers and two thread pools, the two dispatchers are respectively a real-time dispatcher and a fair dispatcher; the real-time scheduler is responsible for scheduling callback instances of the key chain; the fair scheduler is responsible for scheduling callback instances of the common chain; the two thread pools comprise a special thread and a shared thread, and the special thread specially runs a real-time scheduler; the shared thread not only can run a real-time scheduler, but also can run a fair scheduler; the shared thread comprises two execution modes: (1) real-time scheduler priority: the thread operates the real-time scheduler preferentially, and only when the priority queue of the real-time scheduler is empty, the fair scheduler operates; and (2) priority of the fair scheduler: firstly operating the fair scheduler by the thread, and operating the real-time scheduler when the vector set of the fair scheduler is empty.

The invention discloses a thread task processing method, a storage medium, electronic equipment and a program product, and relates to the technical field of computers.The method comprises the steps that in response to a to-be-processed thread task triggered by a user, current hardware resource data of a client side are obtained, and the hardware resource data comprise hardware configuration data and resource use data of the client side; identifying a target category corresponding to the to-be-processed thread task; determining a target parameter prediction model corresponding to the to-be-processed thread task from the plurality of parameter prediction models based on the target category, inputting the hardware resource data into the target parameter prediction model, and predicting target parameters needing to be configured when a thread pool in the client processes the to-be-processed thread task through the target parameter prediction model; adjusting parameters of the thread pool into target parameters, and processing the to-be-processed thread task based on the adjusted thread pool; according to the invention, the resource utilization rate of the system can be optimized.

The invention relates to a service orchestration method based on a multi-tenant micro-service architecture, in particular to the field of distributed systems, comprising the steps of creating a thread pool and defining a private final pipeline, creating a pipeline builder and appointing each stage in the pipeline, generating a pipeline execution class, and initializing a pipeline processing overall business process. The method comprises the following steps: establishing a pipeline object, packaging into different processors, creating a pipeline object, effectively processing concurrent tasks, carrying out reasonable processing under an overtime condition, avoiding waste and blockage of system resources, adding each processor into a pipeline and inserting an interceptor into the pipeline, enhancing expandability and flexibility of a system, and improving system performance. The output results of the processors are combined into a final result, exception handling and rollback operation are carried out, the steps of the whole business process are divided into a plurality of sub-tasks, parallel execution and serial execution sub-task groups are formed, the execution sequence is determined, and the response speed and the concurrent processing capacity of the system are improved.

The invention discloses an RDMA asynchronous communication optimization method and system based on a lock-free queue, a medium and a processor, and relates to the technical field of remote communication. The method comprises the following steps: constructing a lock-free annular buffer area at an edge node, registering the lock-free annular buffer area as an RDMA memory area, and configuring a queue and a thread pool; the collection thread concurrently writes data, and the RDMA submission thread asynchronously completes remote transmission based on a buffer area state; and the queue processing thread performs closed-loop linkage transmission and buffer area states through'batch polling-result analysis-state synchronization-exception processing ', and dynamically adjusts the buffer area and queue parameter optimization performance at the same time. The problems that an existing RDMA scheme is poor in thread cooperation, weak in resource adaptation, not timely in state linkage and the like are solved, transmission delay and CPU overhead are greatly reduced, communication stability and the resource utilization rate are improved, and the method is suitable for edge computing and other high-concurrency data interaction scenes.

The invention discloses a data processing method and device, electronic equipment and a nonvolatile storage medium. The method comprises the steps that a data processing task of a data file is received, and the data file is used for recording three-dimensional geographic information data; storing the data processing task into a task queue; according to the priority of the data processing task of the data file in the task queue and the load condition of the thread in the thread pool, a target thread in the thread pool is called to execute the data processing task, and data in the data file is directly mapped to the virtual address space of the process. According to the method and the device, the technical problems of low resource utilization rate and loading delay during traditional single-thread processing or common multi-thread data processing are solved.

The invention provides a multi-strategy caching and automatic data generation method and system based on a Node environment, and relates to the technical field of data processing, and the method comprises the following steps: receiving a client request, obtaining a data generation template, constructing a data dependency graph to analyze a field dependency relationship, and determining a field generation sequence by adopting a recursive generation strategy and depth-first search; data validity verification is executed, a data generation thread pool is created by adopting an adaptive load balancing mechanism based on the data dependency graph, and asynchronous processing is realized through double buffer queues, so that the data generation efficiency is improved, the data logic integrity is ensured, and the system resource consumption is reduced.

The invention provides an automatic fault-tolerant propulsion method and system for task execution abnormity and a storage medium, and relates to the technical field of distributed system fault-tolerant control, and the method comprises the steps: task dynamic arrangement: arranging tasks into a serial, parallel or dependent execution mode according to a preset dynamic rule, and realizing task trigger execution through an asynchronous thread pool; performing process monitoring, adding a monitoring annotation to an execution method of each task by using an AOP tangent, and capturing a task execution state in real time; performing abnormal grading processing, and executing a strategy according to a pre-configured three-level processing channel; spreading a dependency state, and automatically updating an associated task state based on a dependency relationship among the tasks: when an upstream task fails, pausing a downstream dependency task and marking the downstream dependency task as a waiting state, and continuing to execute a non-dependency task; and performing global state statistics, calculating a global state in real time through a task state algorithm, if all tasks are completed, marking the tasks as completed, if abnormal tasks exist, marking the tasks as abnormal, and triggering root cause analysis.

The present disclosure relates to computer-implemented methods, software, and systems for data migration between data sources by applying nested parallel copying mechanism. A request to copy a source directory to a target directory is received. Entities included in the source directory are identified. Data migration is executed from the source directory to the target directory by performing: assigning a thread pool executor that is configured to execute copy operations at at least one of multiple threads executed by the thread pool executor, where a new copy job is to be executed for migrating content of a respective entity from the source directory to the target directory; and in response to determining that there is no available-for-execution thread at the thread pool executor, copying the respective entity as a nested copy in a currently executing thread that is executing a current copy job.

The invention provides a memory management method and device, equipment, a storage medium, a program product and a chip. The method relates to the technical field of mobile terminals. Comprising the steps of starting an asynchronous compression process under the condition that a current idle memory is lower than a waterline value; and traversing the plurality of memory control groups, and allocating a thread to each memory control group through an asynchronous compression process so as to compress the anonymous page in each memory control group. According to the method provided by the invention, the compression efficiency is improved by asynchronizing the anonymous page compression part in the memory recovery process, creating the thread pool and performing parallel compression according to the memcg memory control group, and meanwhile, the asynchronous memory compression process does not influence the main recovery process.

The invention relates to a business big data processing system and method based on double timed tasks, electronic equipment, a computer readable medium and a computer program product. The method comprises the steps of obtaining business big data from a database through a first timed task; creating a thread pool according to a time window of the first timed task, wherein the thread pool comprises a plurality of threads; processing the business big data through threads in the thread pool; the processing result is updated to the database; obtaining business big data meeting a preset condition from the database through the second timed task; and processing the business big data according to the time window of the second timed task. The real-time performance of data processing and the utilization efficiency of system resources can be remarkably improved, meanwhile, load balancing is achieved through task dispersion, stable operation of the system in a high-load scene is ensured, the system safety is ensured, and the system is suitable for various service scene requirements.

The invention discloses a large model training dynamic memory management method based on a multilayer hierarchical cache mechanism, and relates to the technical field of computer system architecture and storage. The method is managed on the basis of a dynamic heterogeneous memory unloading system, and comprises the following steps of: in an initial stage of a training task, constructing a user state access path, establishing a TensorID-LBA mapping table, and performing hierarchical caching and threshold presetting; in the training task running stage, the access popularity of each tensor is evaluated in real time; when occupation of the GPU video memory cache layer exceeds a threshold value or cold data is detected, a dynamic migration strategy is triggered, and the low-frequency tensor is unloaded to a CPU memory cache layer or an SSD storage layer; when the tensor access popularity is improved, automatically migrating back to a high-performance layer through a prediction mechanism; in the process, tensor self-adaptive migration is executed by a user mode I / O thread pool. According to the method, the high-performance I / O stack is constructed in the user mode, and the adaptive tensor unloading and hierarchical caching mechanism of the multi-layer heterogeneous memory is combined, so that the universal deployment of the system is realized.