159 results about "Task parallelism" patented technology

The invention provides a resource intelligent bus and task parallel hydrogen energy data scheduling optimization method and system, relates to the technical field of energy, and aims to solve the problems of resource heterogeneity, lack of real-time performance and dynamic adjustment, insufficient energy efficiency optimization, poor adaptability to abnormal working conditions and the like in the prior art. According to the method, a resource intelligent bus architecture is constructed, and the resource intelligent bus architecture comprises a resource virtualization encapsulation layer, a resource state sensing layer and a global prediction and scheduling layer; in the resource state sensing layer, performing multi-granularity task decomposition on hydrogen energy business requirements, and establishing an inter-task dependency relationship model by adopting a directed acyclic graph; in the global prediction and scheduling layer, a multi-objective optimization model is constructed based on a dependency relationship between resource states and tasks, and resources and tasks are dynamically matched according to an optimal solution of the model; and dynamically scheduling the resources according to a matching result of the resources and the tasks. The problems in the prior art are solved, and the hydrogen energy data scheduling optimization performance is improved.

The invention relates to the technical field of artificial intelligence, and provides a model distributed training optimization method, electronic equipment and a storage medium, and the method comprises the steps: respectively constructing a calculation flow and a communication flow based on the calculation operation and the communication operation in a model; in the training iteration of the model, executing staggered pipeline scheduling for at least two micro-batches of tasks; wherein the staggered pipeline scheduling comprises the following steps: scheduling a first micro-batch of calculation tasks to a calculation flow for execution, and scheduling a second micro-batch of communication tasks to a communication flow for execution, or scheduling a first micro-batch of communication tasks to the communication flow for execution, and scheduling the second micro-batch of calculation tasks to the calculation flow for execution, the tasks of the first micro-batch and the tasks of the second micro-batch are processed in parallel. According to the method, the parallel processing of the calculation task and the communication task is realized by executing the staggered assembly line scheduling on the tasks of different micro-batches, and the hardware utilization rate and the model training efficiency are improved.

The invention provides a safety measure generation method and system based on a power distribution network multi-task parallel work ticket, and the method comprises the steps: receiving a to-be-executed work ticket containing a plurality of parallel tasks, and extracting the task information of each task; obtaining a plurality of task groups and task group associated equipment sets corresponding to the task groups; obtaining a task group minimum operation sequence set; obtaining a task group topology boundary equipment set, and performing topology search by using a topology boundary equipment set subset to obtain a safety measure set of each task group; and based on the minimum range maintenance decision model, optimizing the safety measure set of each task group to obtain a final multi-task parallel safety measure set. According to the scheme, the tasks which are electrically connected adjacently are combined into one task group, so that repeated operation on the same equipment can be avoided, unnecessary workload is reduced, and the safety measure generation efficiency of the safety measure generation method based on the multi-task parallel work ticket of the power distribution network is improved.

The invention provides a multi-task parallel processing method for user problems under an AI platform, and belongs to the technical field of digital data processing of the AI platform. Task resource requirements are accurately calculated through a video memory pre-estimation function and a memory pre-estimation function, a resource consumption mode of concurrent execution is analyzed through a multi-task resource prediction model; the load capacity of the system is evaluated based on a video memory utilization rate gain index, an optimal task segmentation strategy is determined through a data set splitting degree calculation function, multi-stage video memory sub-pools are constructed to realize differentiated resource management, and a Nash equilibrium point of resource allocation is solved by adopting a data set video memory allocation game model. Collaborative optimization allocation of resources is achieved through the video memory and memory coupling allocation equation set, the task state is monitored in real time in the multi-task concurrent execution process, the resource allocation weight is dynamically adjusted, and the technical problem that the system resource utilization rate is low during AI platform multi-task parallel execution is solved.

The invention discloses a multi-task parallel data stream processing method and device based on an FPGA (Field Programmable Gate Array), which are applied to the technical field of data processing. Carrying out normalization processing on the decoded data to generate a data stream in a unified format; performing coding and ID identification code embedding processing on the normalized data to generate to-be-stored data with an identifier; carrying out FIFO (First In First Out) buffering and write-in priority judgment on the coded data to generate priority queue data; dDR writing and BANK scheduling processing are carried out on the priority queue data, and stably stored parallel data are generated; carrying out reading and output coordination processing on DDR storage data to generate a continuous output data stream, and triggering a reading application by decoding ID identification code positioning data, reading FIFO according to depth scheduling and monitoring data volume by channel temporary storage FIFO; and performing dynamic bandwidth allocation and energy efficiency optimization on the whole data stream processing process to generate an efficient parallel processing result.

The invention relates to a task execution method and device based on a hybrid expert model, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence. The calculation utilization rate and the calculation efficiency of the equipment can be improved. The method comprises the steps that the number of lexical units distributed to each expert in equipment is determined, the number of lexical units of each expert group in the equipment is obtained according to the number of lexical units of each expert, the expert group comprises one or more experts in the equipment, and an execution sequence corresponding to each expert group is determined according to the relative size of the number of lexical units of each expert group in the equipment, different devices are configured with the same number of expert groups, each expert group comprises the same number of experts, the expert groups in the same execution sequence in the different devices have the same relative size in the respective devices, and the communication task and the calculation task of each expert group are sequentially executed according to the execution sequence. The computing task of each expert group in the device is parallel to the communication task of another expert group in the device.

The invention relates to the technical field of industrial automatic control, and discloses an industrial configuration software cross-platform cooperative control method and system based on a swan gap, which realize flexible initialization and dynamic task distribution of a control terminal by fully utilizing distributed communication and microkernel characteristics of a swan gap system. Through structured analysis and task graph construction of a configuration control model, granularity recombination and task parallel execution of a control process are supported; a task terminal recommendation mapping table is further generated in combination with the execution requirement of the control task and the terminal capability label information, and the adaptation matching degree of the control task and the execution terminal is effectively improved; and meanwhile, a distributed cooperative channel is constructed, a communication binding channel between the master control terminal and the slave control terminal is established, and a guarantee is provided for rapid scheduling and execution of tasks. Compared with the prior art, the heterogeneous compatibility, the task scheduling efficiency and the protocol adaptation flexibility can be remarkably improved, and the method is suitable for high-complexity and multi-terminal collaborative industrial automation control scenes.

The invention provides a graph card-based hybrid scheduling rendering method and system, and the method comprises the steps: dividing a frame buffer area, binding the divided frame buffer area with a task, completing the rendering of each card through a hybrid task scheduling strategy, and finally synthesizing and outputting a complete graph; a frame buffer area of a rendering graph is divided into a plurality of graph cards according to display criticality, each graph card is bound with an independent rendering task and a video memory area, a unique task division and resource allocation mode is formed, and the mode is different from a traditional overall rendering mode; and in combination with a hybrid scheduling strategy of high-priority task preemptive scheduling and dynamic card task parallel rendering, the problems of rendering task priority conflict and resource utilization are effectively solved.

The invention provides an MCM-GPU-oriented resource storage optimization method, device and equipment, and the method comprises the steps: carrying out the static analysis of an operator-level data flow diagram of a calculation task in a compiling stage, and extracting the data dependence relation and life cycle characteristics of each operator; generating a storage optimization scheme based on the data dependency relationship and life cycle characteristics of each operator; in the operation stage, task and data collaborative allocation is carried out according to a data prefetching strategy and a data replacement strategy, task and data collaborative allocation and the calculation task are executed in parallel, and a data flow diagram and an operator dependency relationship of the task are analyzed by utilizing a compiling period; in combination with the data locality characteristics of the MCM-GPU multi-level storage architecture and the concurrency requirements of different modules for data access, an appropriate GPU module is selected to carry out data optimization management, so that frequent data exchange between a GPU and a CPU is reduced, and the performance potential of the MCM-GPU architecture in a super-large-scale task is brought into full play.

This invention discloses a method and apparatus for processing multi-task parallel data streams based on FPGA, applied in the field of data processing technology. The invention constructs a multi-format raw data receiving and decoding system; normalizes the decoded data to generate a unified format data stream; encodes and embeds ID identification codes into the normalized data to generate tagged data to be stored; performs FIFO buffering and write priority determination on the encoded data to generate a priority queue data; performs DDR write and BANK scheduling on the priority queue data to generate stable parallel data; and coordinates the reading and output of the DDR stored data to generate a continuous output data stream. Specifically, data is located by decoding the ID identification code, reads from the FIFO by depth scheduling, and the channel temporary storage FIFO monitors the data volume to trigger read requests. Dynamic bandwidth allocation and energy efficiency optimization are performed throughout the data stream processing process to generate efficient parallel processing results.

The invention discloses a dynamic priority dyeing task scheduling and conflict resolution method, which comprises the following steps of: pre-checking task conflicts, extracting all step sequences of a newly-added task and reversing, and detecting whether irreversible opposite node conflicts exist between the step sequences of the newly-added task and all the step sequences of the executed task or not; a to-be-updated execution task queue is generated, the task dip dyeing state of each dyeing site is traversed, to-be-updated execution tasks are screened out, task weights are calculated, and the to-be-updated execution task queue is generated according to descending sort of the weights; a dynamic priority decision is made, and the task executed by the mechanical arm in the next step is determined according to the weight sorting result of the to-be-updated execution task and the step precision level; and a mechanical arm operation instruction set is generated, whether space-time conflicts exist or not is judged, if yes, a new to-be-updated execution task is selected to generate the mechanical arm operation instruction set, and if not, the mechanical arm operation instruction set is executed. Multi-task parallel processing is supported, and the rapid processing requirement of large-scale pathological samples is met.

The invention discloses a CAE (Computer Aided Engineering) workflow control subsystem and method based on cloud computing, and belongs to the technical field of crossing of computer simulation and cloud computing, and the subsystem comprises a workflow configuration analysis module which is used for converting a CAE workflow configuration file possibly containing cyclic dependence into a directed acyclic graph state machine through strong connectivity component analysis; the workflow state machine operation module is used for dynamically deriving task groups which can be executed in parallel in the next stage in the state machine according to the current state; and the workflow scheduling module is used for submitting the tasks to a cloud solving platform in parallel and realizing fault recovery through state persistence. Through deep fusion of a graph theory method and a cloud computing architecture, automatic scheduling, efficient parallel execution and reliable fault tolerance of the CAE workflow are realized, and the computing resource utilization rate and the system reliability are remarkably improved.

The invention discloses a GPU cluster resource allocation method and device, equipment and a storage medium, and relates to the technical field of GPU cluster resource allocation. According to the method, a hardware performance description vector is generated by fusing static hardware parameters and dynamic micro-benchmark test data, the limitation that the GPU performance is represented only by depending on the static parameters is broken through, and the actual operation capacity of different architecture GPUs in a heterogeneous cluster is accurately matched; task feature vectors are generated by extracting task calculation features and resource demands, and quantitative description of reasoning task resource consumption features is achieved; information of hardware, tasks and load dimensions is integrated through a machine learning model, performance degradation characteristics during multi-task parallel are effectively captured, and the accuracy of execution time prediction is improved; scheduling decision operation including candidate node screening, comprehensive cost evaluation and resource reservation backfilling is executed in combination with the predicted execution time, and the task execution efficiency and cluster load balancing are both considered.

The invention provides an efficient operation and maintenance software architecture setting method for multi-task parallel scheduling, and relates to the technical field of operation and maintenance automation. The method comprises the steps that resource state data of operation and maintenance nodes in an available area are collected in real time; dynamically calculating a comprehensive load index of each available area operation and maintenance node based on the resource state data to obtain a calculation result; according to the calculation result, identifying a high-load available area with the comprehensive load index exceeding a preset threshold value and a low-load available area with the comprehensive load index lower than the preset threshold value; and based on the high-load available area, performing priority grading on operation and maintenance tasks to be processed in the high-load available area according to task types to form a high-priority core task queue and a low-priority conventional task queue. According to the invention, accurate matching and efficient utilization of cross-regional resources are realized.

The invention provides an asynchronous parallel-based service processing method, system and device and a storage medium, and the method comprises the steps: initializing a service context object according to a service type identifier in a request; a preprocessing rule script corresponding to the service type is loaded and executed through a dynamic rule engine, and parameter verification, service pre-check and context initialization are carried out; a plurality of asynchronous processing rule scripts corresponding to the service types are loaded and executed in parallel, each asynchronous processing rule script corresponds to one service task and is packaged into an asynchronous task, the asynchronous tasks are executed in parallel, and a result is written into a service context object; and loading and executing a result processing rule script corresponding to the service type through a dynamic rule engine, aggregating results of the different step tasks, and generating and returning a service response after applying a final service logic. Parallel processing can be realized through a three-stage asynchronous processing architecture, multi-core CPU resources are fully utilized, and the throughput and response speed of the system are improved.

The invention provides a watershed spatio-temporal data storage and retrieval method and system based on multistage indexing, and the method comprises the steps: S1, obtaining watershed multi-source heterogeneous spatio-temporal data, and carrying out the logic partitioning of watershed spatial raster data; s2, constructing a global, local and memory multi-level index architecture; s3, executing a query task based on the multi-level index, scheduling the query task to a target node and performing parallel processing; comprising the following steps: positioning a target logic sub-block according to a query range, scheduling a task to a data storage node in combination with dynamic re-partitioning, and realizing end-to-end efficient retrieval through multi-stage indexing; and S4, executing partition adjustment according to the node load, namely dynamically triggering partition refining or merging operation according to the resource use condition of each node in the task running process, and optimizing the matching degree of data locality and task parallelism. According to the method, a multi-level collaborative index and dynamic re-partitioning mechanism is constructed, so that quick response to a complex query request and efficient scheduling of computing resources are realized.

The invention discloses a bank batch processing system and a working method thereof. The system comprises a task scheduling module, the task scheduling module is configured to automatically trigger batch processing tasks according to a preset scheduling strategy, and a task fragmentation module is configured to divide the batch processing tasks into a plurality of subtasks for parallel processing according to a fragmentation strategy and dynamically adjust the fragmentation number according to system resources. The task execution and fault-tolerant processing module is configured to capture task execution anomalies and execute retry or skipping operation on tasks which are executed abnormally based on a preset exception execution strategy, the monitoring and log module is configured to monitor task execution states in real time and record log information, and the performance optimization module is configured to optimize task execution performance. Through deep fusion of technical innovation and financial scenes, systematic improvement is realized in the aspects of efficiency, fault tolerance, extension, operation and maintenance, safety and the like, and an efficient, reliable and compliant solution is provided for bank batch business processing.

The invention relates to the technical field of large-model lightweight fine tuning, and discloses a parallel optimization method of a low-rank adapter and a task awareness scheduling system.The parallel optimization method comprises the steps that an increment matrix of the low-rank adapter is fragmented to a tensor parallel group according to rows, the tensor parallel group comprises a plurality of computing devices, and the computing devices are used for computing the increment matrix of the low-rank adapter; the fragmentation granularity is dynamically determined according to the equipment hardware capability and the dimension of the increment matrix; dynamically scheduling the tasks with strong conflicts to different task parallel groups based on the calculated inter-task gradient conflict coefficient; and for a plurality of tasks scheduled to the same task parallel group, adapter calculation is merged into unified large matrix operation by adopting a micro-batch processing technology, and LayerNorm and adapter projection calculation are merged into a single calculation kernel by adopting a kernel fusion technology. According to the method, communication redundancy and synchronization overhead in distributed training are reduced, task interference during multi-task parallel is effectively eliminated, and the problem of computing resource fragmentation is solved.

The embodiment of the invention provides an inference task processing method and an inference server system. The method comprises the following steps: determining a plurality of calculation tasks from a plurality of reasoning tasks; configuring task labels corresponding to the operator types for the plurality of calculation tasks; a plurality of target operators are determined from a computing power unit of a computing power group module according to the task labels, parallel processing is conducted on the computing tasks through the target operators, and each target operator processes the computing tasks of the task labels corresponding to the operator type of the target operator. Through the embodiment of the invention, the problem of low reasoning efficiency of the reasoning server in the related technology can be solved, so that the effect of improving the reasoning efficiency of the reasoning server is achieved.

The invention provides an asynchronous traceable API system for a large language model performance benchmark test, and belongs to the field of natural language processing and artificial intelligence. User-defined configuration is received through an API, an asynchronous thread is adopted to execute a task, and a task state and a result are persistently stored; multi-task parallelism, state traceability and performance data integrated output are achieved, the test automation degree and efficiency of the large language model are remarkably improved, and the method is suitable for model research and development, verification and operation and maintenance scenes.

The invention provides a time window perception priority task scheduling method and system for a detection process, and relates to the field of task scheduling and intelligent optimization, and the method comprises the following steps: S1, inputting data of a to-be-scheduled task set; s2, priority scoring: based on the priority weights of the tasks and the width of the time window, calculating a comprehensive sorting factor, and performing preliminary sorting on all the tasks according to the comprehensive sorting factor to form a scheduling preliminary sorting sequence; s3, time window adjustment: on the basis of scheduling the initial arrangement sequence, detecting the task arrangement sequence one by one by utilizing a time window perception adjustment mechanism until the time window constraint requirement is met; and S4, outputting a complete scheduling result. According to the method, scheduling sequence optimization, resource conflict relief and urgent task priority response can be realized in a multi-task parallel environment on the basis of a score sorting and time window adaptive adjustment mechanism, and the method has good universality and expansibility and is suitable for intelligent scheduling optimization requirements of multiple types of businesses.

The embodiment of the invention provides a multi-node parallel acceleration method, device and equipment based on a high-performance computing platform and an RoCE network and a medium, and relates to the technical field of cluster computing, and the method comprises the following steps: constructing a communication stack based on the RoCE network, and realizing an interconnection transmission network between nodes from a sending end to a receiving end; a computing task and a communication transmission task of the high-performance computing platform are executed through a CPU core binding technology of the high-performance computing platform; based on the transparent large page function, processing a memory allocation request initiated by an application program of an operating system and a request of the application program of the operating system for accessing a memory through a virtual address; before a main computing task of the high-performance computing platform starts, pre-processing is executed through multiple threads, open type multi-processing threads are started, the main computing task is executed through the multiple open type multi-processing threads, and a next round of pre-processing task is started in parallel. According to the scheme, parallel acceleration of multiple nodes in cluster calculation is realized.

The application discloses a kind of electromechanical-electromagnetic transient hybrid simulation calculation method and system, combine Kubernetes management cluster and the actual calculation demand of electromechanical-electromagnetic transient hybrid simulation, the computing task of containing numerous computing cases is distributed to each work node, the computing task of each work node can be effectively balanced, while the computing task of single case is allocated to the same CPU, from the two angles of task parallel and sub-network parallel, efficiency is improved, the electromechanical-electromagnetic transient hybrid simulation time consumption is reduced as a whole, and the electromechanical-electromagnetic transient hybrid simulation efficiency of multiple computing cases is improved.

The invention relates to the technical field of laboratory control, in particular to a multitask parallel experiment system and method for a laboratory, and the system comprises a task management module which is used for receiving and splitting a plurality of experiment plans submitted by a user, and obtaining a plurality of experiment tasks corresponding to the experiment plans; the task constraint module is used for performing condition constraint on the experiment task according to the constraint rule to obtain a constraint experiment task with a constraint condition; the material detection module is used for acquiring material container resource information of the experiment platform, performing material container resource detection on the constraint experiment task according to the material container resource information, and generating a resource satisfaction signal; and the multi-task scheduling module is used for constructing a multi-thread scheduling strategy based on the thread pool management mechanism and the resource satisfaction signal, and scheduling the execution module to execute a plurality of experiment plans according to the multi-thread scheduling strategy. The utilization rate of experimental resources can be optimized, so that the execution efficiency of experimental tasks and the stability of experimental operation are improved.

The application discloses a multi-stage dynamic arrangement method for Modelica task parallel compilation load, the method designs an arrangement optimization strategy according to the granularity of each stage, the front compilation stage improves the translation speed by setting the number of parallel CPUs for single-machine compilation of the task, but if multiple tasks are compiled concurrently on the single machine, the speed will decrease exponentially, so when the execution nodes are distributed, all the tasks currently arrived are arranged globally; the number of parallel compilation sub-threads of the total splitting and the distributed compilation optimization Modelica task parallel compilation load are set in the post-compilation stage, aiming at solving the problem that the Modelica model cannot be distributed compiled, improving the overall compilation efficiency of the system, and reducing the slow simulation speed problem caused by the single-machine computing power problem.

The invention discloses a multi-AGV cooperative scheduling system for realizing load balancing. The multi-AGV cooperative scheduling system comprises a task management module, a state monitoring module, a load balancing scheduling module, an AGV cooperative scheduling module, an edge server optimization module, a task compression module, a task parallel processing module and a performance evaluation module. Wherein the load balancing scheduling module adopts a mode of combining first-stage weighted polling initial allocation and second-stage optimal allocation based on multi-agent deep reinforcement learning, the AGV collaborative scheduling module adopts an AxTD3 method to divide an Actor network into two parallel sub-networks, and joint training is carried out by sharing a Critic network. The edge server optimization module takes execution time deviation as a state, constructs a lightweight state vector in combination with a server isomerism index, and introduces federated learning and multi-Critic network architecture. According to the invention, adaptive collaborative scheduling of multiple AGVs in an edge computing environment is realized, and the system load balancing capability, the resource utilization efficiency and the task processing response speed are effectively improved.

This invention discloses a method and system for batch processing and monitoring large files, relating to the field of computer technology, and particularly to a method and system for batch processing and monitoring large files. The method includes: sending a file to a batch processor via a file transfer platform to obtain a first sent file; triggering a first creation instruction when the file arrives; creating a first batch processing task execution graph through task relationship analysis; obtaining a first distributed execution task, wherein the first distributed execution task includes multiple subtasks; obtaining a first invocation instruction; invoking the logical module service configured corresponding to the first distributed execution task for parallel task execution according to a distributed scheduling framework and the first invocation instruction; and obtaining the first file batch processing result by managing the task status through a callback scheduling center when the task execution is completed. This solves the technical problems of long processing times and the inability to promptly identify and address anomalies in existing technologies for batch processing of large files.

The application relates to a kind of automation control devices of test and control station based on editable script, including at least one script file, for defining the test and control task flow of test and control station in editable text form;Script analysis execution module is used for parsing script file and mapping to internal function for calling;At least one device control command execution module is used for controlling the test and control equipment in test and control station;Script server, by calling script analysis execution module and device control command execution module, carries out parallel control to test and control station according to preset execution mode multiple script tasks.The application also relates to a kind of automation control method of test and control station based on editable script.Can effectively improve the standardization, flexibility and scalability of spaceflight test and control station, to execute the tracking and control task of various spacecrafts with low cost flexibility, high efficiency.

The application discloses a state information collection method and related components thereof, and relates to the field of task execution. When a collection task is being executed, a task for obtaining the current state of each hardware is executed in parallel. When it is determined that the current state of a hardware has changed, a first weight value of each preset virtual execution environment at the current time is determined. The first weight value is positively correlated with the idle degree and execution force of the preset virtual execution environment. The state information of the hardware whose current state has changed is collected by using the preset virtual execution environment with the highest first weight value. Since the task for obtaining the current state and the collection task are executed in parallel, the normal execution of the collection task is not affected. The state information is collected by using the preset virtual execution environment with the highest first weight value, so that the situation of redundant and complex collection process caused by collecting multiple state information is avoided, and the situation of collection task execution failure caused by too long execution time of the collection task is also avoided.

The application discloses a multi-role and multi-task parallel execution method and device in a round-based competitive application, which can obtain a first action task sequence that does not conflict with at least one second action task in execution through conflict judgment on a real-time acquired action queue of a current round to be acted and the second action task in current execution, and simultaneously execute each first action task in the executable first action task sequence in the process of continuing to execute the second action task, so as to realize the purpose of parallel execution of action tasks in the case of no conflict, so that multiple action tasks in the same round can be simultaneously executed without waiting, thereby greatly improving the round execution speed, improving the execution rhythm of the round-based competitive application on the basis of ensuring the consistency and fairness of the execution of the round-based competitive application, and improving the user experience.