19results about How to "Improve parallel efficiency" patented technology

The invention relates to the technical field of computers, in particular to an operation method of a DeePMD-kit model in a Shenwei super computer, six core groups CG of the Shenwei super computer are respectively responsible for reasoning of six sub-regions of an MPI process, each core group CG comprises an MPE and 64 CPEs, and the MPE contains a 256-bit SIMD unit; each CPE contains a 512-bit SIMD (Single Instruction Multiple Data) unit; each CPE comprises a local data memory, and each local data memory LDM realizes continuous data transmission through a DMA strategy and a main memory; the CPEs in the same core group CG realize data transmission through remote memory access (RMA); the reasoning method of each core group CG comprises the following steps: accelerating a time-consuming operator through an SACA, and distributing a calculation-intensive part of the time-consuming operator to 64 CPEs by MPE; after calculation is completed, the MPE synchronizes calculation results of the 64 CPEs and stores the results in a main memory; when the SACA runs, the SIMD is parallel, and a 512-bit vectorization instruction is sent out through the CPE. By means of the technical scheme, the method has the advantage that the DeePMD-kit model can be applied to the Shenwei computer.

The invention is suitable for the technical field of big data mining, and particularly relates to an entity resolution parallel processing method and device, equipment and a medium. According to the method, entity tuples in a to-be-resolved data set are divided into non-intersecting partitions, the entity tuples in any partition are equally divided into N intervals, M continuous intervals form a window, all the windows are subjected to serial processing, a thread block is distributed to each interval in the window for any window, and a thread block is distributed to each interval in the window. The thread blocks are used for matching target tuples in the corresponding intervals with entity tuples except the target tuples in the partitions to obtain matching results, and the thread blocks in the window are processed in parallel; and if not, processing the interval in the next window of the window by using the executed thread block until all the intervals are processed by the thread block to obtain all matching results. The entity data is effectively segmented, resource utilization and parallelization are optimized, and the entity resolution efficiency is improved.

The application discloses an electromagnetic multilayer fast multipole hierarchical GPU parallel acceleration implementation method, relates to the field of computational electromagnetics, and comprises the following steps: acquiring parameters of incident electromagnetic waves and a grid file of a target to be measured; adopting a multilayer fast multipole algorithm to construct an octree corresponding to the grid file and to construct a process tree; performing hierarchical division on the octree based on a ternary parallel strategy to obtain a plurality of parallel layers corresponding to the octree; adopting a mapping allocation method to allocate the operation load of each parallel layer to all operation processes of the process tree; according to the operation load corresponding to the operation process and the number of GPUs in the operation process, distributing the operation load corresponding to the operation process to each GPU in the operation process; and splicing the operation results of all GPUs and solving the radar scattering cross section of the target to be measured according to the splicing result. The application can effectively reduce the communication and synchronization overhead of the multilayer fast multipole algorithm, improve the multi-card load balancing and the overall computing efficiency.

The present invention discloses a method for predicting UAV conflicts based on a safety interval progressive decision tree, including: establishing a UAV safety interval parameter library; establishing a coordinate conversion model parameter library for converting WGS-84 coordinate system and GLONASS-PZ90 coordinate system to CGCS2000 coordinate system, and encapsulating the constructed Bursa model as a coordinate conversion function; parsing flight dynamic data and performing standardized conversion; predicting airspace UAV conflicts, and dividing the prediction results into a data set to be observed, a data set of key attention, and a data set of warnings according to the conflict risk level; calculating the time when the UAV pairs in the data set of key attention may reach the warning level; calculating the time when the conflicting UAV pairs in the warning data set are expected to collide. The present invention can maximize the utilization of airspace space resources, thereby effectively improving the flight flow in the airspace.

The invention provides a system architecture for cross-data center large model training and a resource allocation method and system, and belongs to the technical field of cross-wide area distributed large model training. According to the method, large-scale model cooperative training across multiple data centers can be realized, and the bottleneck that the computing power of a single data center is limited is broken through. Through unified modeling and scheduling of calculation, memory and network resources, task loads can be intelligently allocated according to hardware performance and network bandwidth of different data centers, and efficient collaborative utilization of computing power resources is realized. The training task of the super-large-scale model can be rapidly completed in the heterogeneous computing power environment, and the training time is remarkably shortened. The provided flexible parallelism degree allocation method can be adaptive to different task and resource conditions, the proportion of data parallelism, model parallelism and pipeline parallelism is automatically adjusted, the parallelism efficiency is improved, and the communication overhead is reduced. A training time estimation function is integrated, the overall time delay and resource requirements can be predicted before task execution, and a basis is provided for scheduling decision making.

This invention discloses a parallel optimization system for massive video processing. The system adopts a four-layer integrated parallel and collaborative processing architecture, comprising, from top to bottom: a parallel task layer for video stream access, grouping, splitting, encapsulation, and queue management; a resource abstraction layer for unified hardware modeling, status acquisition, topology construction, and capability assessment; a parallel scheduling layer for task-hardware matching, load balancing, priority scheduling, and dynamic adjustment; and an execution optimization layer for data stream localization, parallel read / write, cache optimization, and zero-copy processing. The parallel task layer, resource abstraction layer, parallel scheduling layer, and execution optimization layer form a complete parallel processing link: task input, resource awareness, accurate scheduling, and optimized execution. This invention implements concurrent rate limiting and smooth access for the input video stream to prevent traffic surges; and sets synchronization points and timing control for parallel tasks to ensure orderly output.

The application discloses a self-adaptive balanced division method based on GPU acceleration sparse-dense matrix multiplication, and comprises the following steps: analyzing a sparse matrix, determining long rows affecting parallel performance; dividing the long rows into equal "parts"; storing the divided sparse matrix by using a new storage format; loading all nonzero elements in the "parts" to a shared memory on a GPU; consuming the nonzero elements in the shared memory and multiplying corresponding elements of a dense matrix; adding the parts from the same row and obtaining a final result. The application aims at the current situation that long rows lead to low parallel efficiency when a row division mechanism is used in GPU acceleration sparse-dense multiplication, and proposes a self-adaptive balanced division method based on the structure characteristics of a GPU. The "parts" are used to replace rows as parallel granularity, and the size of the "parts" is controlled within a reasonable range to ensure that a thread group has enough threads to effectively parallelize the nonzero element task, and the similar size of the "parts" avoids the load imbalance among the thread groups.

The invention provides an automatic testing method and device based on test sequence generation, equipment and a medium, and relates to the technical field of server testing. The method comprises the following steps: acquiring natural language test requirement data, and generating a structured target test step sequence based on a test requirement; then resource state information of the environment to be tested is collected, the resource state information is combined with the target test step sequence, and a test task scheduling instruction used for indicating a step execution strategy is generated; and distributing the test task scheduling instruction to a plurality of distributed execution nodes in a to-be-tested environment, and executing test steps in the scheduling instruction by the distributed execution nodes to obtain a corresponding test execution result. According to the scheme, automatic generation from natural language requirements to executable test steps can be achieved, reasonable scheduling is carried out based on the environment resource condition, and the execution efficiency and accuracy of the test task in the distributed environment are improved.

The invention belongs to the technical field of nuclear reaction, and discloses a nuclear reactor physical numerical simulation method and system based on a CPU / GPU heterogeneous platform, and the method comprises the steps: assuming that each resonance nuclide adopts a subgroup fixed source equation needing to be solved for N times on average; according to subgroup transport calculation, circulation is carried out on each subgroup of each resonance nuclide in a calculation system, for M resonance nuclides, the number of calling times of a characteristic line method transport solver by a subgroup fixed source equation is MN, and characteristic line scanning calculation needs to be carried out for each calling; the MN subgroups are combined together to form a'multi-group 'structure; the multi-group structure is applied to subgroup fixed source transport calculation, feature line scanning is carried out in the whole multi-group structure at the same time, and fixed source flux of all subgroups is solved at the same time; a multi-group grouping calculation method is adopted, it is assumed that the number of energy groups adopted by final multi-group transport calculation is G, and sub-group fixed source calculation of a multi-group structure is conducted through existing multi-group transport variables.

The invention provides a neutron transport scanning operator optimization method based on built-in function and instruction level parallelism, and the method comprises the steps: firstly reconstructing a data layout, declaring an array according to an energy group sequence, carrying out the memory alignment, and enabling a calculation process to adapt to an SIMD vectorization architecture; then implementing explicit vectorization calculation by adopting a built-in function, performing polar angle cycle expansion and instruction recombination, and fully mining instruction-level parallel potential; meanwhile, key index calculation is optimized into vectoring polynomial approximation. Based on the optimized data flow and the parallel computing mode, efficient updating of the angular flux and the standard flux is achieved, and convergence judgment and calculation are completed. According to the method, the calculation throughput rate and the hardware resource utilization rate of the scanning operator are remarkably improved, and the method can be coordinated with coarse-grained parallel strategies such as MPI / OpenMP and is suitable for a multi-architecture high-performance calculation environment.

The application discloses a kind of analog parallel intelligent conversation CPU and its control method.The CPU is constituted by parallel computing array using 32 completely independent analog operation channels, each channel is composed of 8 operational amplifiers in series, each operational amplifier is composed of 16 basic units, and each basic unit is composed of 4 transistors.A gate array is arranged at the front end, and the number of open gates and combinations are dynamically controlled according to the task size by the conversational logic reasoning instruction set.The application quantifies all human and social science knowledge, natural language logic and conversational reasoning rules into mathematical vectors, points, lines, circles, distances, angles, weights and switches, fully mathematizes the knowledge of liberal arts, and is directly executed by hardware.The application has the advantages of simple hardware, low power consumption and high parallel efficiency, and can realize human-like intelligent conversation and reasoning ability based on mathematical logic closed loop without big data training, deep learning stacking parameters or high-performance digital chips.

This invention discloses a task resource scheduling management system and method based on intelligent algorithms, belonging to the field of big data analysis technology. The invention constructs a first task set, then breaks down tasks into subtasks to form a subtask set; counts the number of subtasks for each task, constructs a task stage sequence, and adds pending processing tags to the subtasks of the first stage; obtains the CPU utilization, GPU utilization, and estimated execution time of the pending subtasks, selecting the combination with the largest product of the sum of CPU utilization and the sum of GPU utilization as the execution set; if the products are the same, the combination with the smallest sum of estimated execution times is selected; monitors the execution set, assigns a completed tag to each completed subtask and activates the corresponding subtask for the next stage, dynamically replenishing the execution set until all tasks are completed. This invention avoids task dependency deadlock, achieves low-overhead dynamic scheduling, and improves the execution efficiency of heterogeneous computing platforms.

This invention discloses a method for parallelizing and pipelining a nested multiplier, relating to the field of intelligent warehousing technology. Based on the nested multiplication concept, the multiplication operation is decomposed into a formula to obtain the squaring operation of the high-bit part and the difference of squares plus shift operation of the low-bit part. Based on the characteristic of 2ab in the multiplication formula, the intermediate values ​​required for high-bit calculation are obtained by querying a preset low-bit square value table through the low-bit results. For batch data, multiple sets of data are processed simultaneously through a parallel architecture. The overall calculation process is divided into multiple pipeline stages, and continuous calculation is achieved through stage cascading. The pipeline output results of each parallel computing unit are collected, verified, and integrated to output a complete set of multiplication results. This invention achieves breakthroughs in hardware resources, computational latency, and batch throughput through the collaborative design of nested recursive simplification, pipeline reuse, and parallel architecture, providing an efficient solution for high real-time and high-parallelism digital computing scenarios.

The invention discloses a computing power resource dynamic scheduling system supporting trusted data exchange. The computing power resource dynamic scheduling system comprises an underlying hardware adaptation layer, a trusted data exchange layer, a core computing power optimization layer, a dynamic scheduling layer and a monitoring and feedback layer, wherein the bottom hardware adaptation layer comprises a chip migration adaptation module and is used for realizing instruction set mapping and capability portraying of heterogeneous hardware; the core computing power optimization layer comprises a low-delay reasoning system, an efficient N-dimensional parallel system and a heterogeneous memory management system; and the dynamic scheduling layer outputs a computing power distribution strategy based on the capability portrait and the task characteristics. According to the scheme, through deep fusion of hardware adaptation, credibility guarantee, computing power optimization and intelligent scheduling, a full-stack computing power scheduling system in a credible scene can be constructed, and the scheduling efficiency is improved.

This application discloses a generative semantic communication system for 3D content generation, comprising: a transmitter deployed on an edge device, a receiver deployed on a cloud server, and a wireless channel connecting the transmitter and receiver; the transmitter extracts visual semantic tokens from a single input 2D image to form a visual semantic token sequence; the visual semantic token sequence and the signal-to-noise ratio of the current wireless channel are used as input and mapped to a channel input symbol sequence for transmission; the receiver recovers the semantic token sequence from the received signal; and a 3D mesh model is generated based on the recovered semantic token sequence; thereby, by transmitting compact visual semantic tokens instead of raw data, the wireless bandwidth requirement is significantly reduced; an efficient adaptive transmission architecture based on structured state space duality is adopted, which improves parallel efficiency while maintaining linear computational complexity, and integrates a channel adaptive state adjustment module to dynamically adapt to time-varying channels.

The invention belongs to the technical field of large language models, and particularly relates to a large language model KV Cache unloading optimization method. Comprising the following steps: inputting a plurality of marks obtained by performing word segmentation processing on a to-be-processed text into a large language model; aiming at the current mark of the current attention layer, pre-selecting a key KV index set of the next mark of the current attention layer by using a cross-mark pre-selection mechanism; loading the key KV of the current mark in the next attention layer pre-selected by the previous mark by using a cross-mark pre-selection mechanism to the GPU from the external storage; the GPU adopts a KV cache adaptive hierarchical management strategy to manage the cache; the GPU executes sparse reasoning calculation which is currently marked on the current attention layer; taking the next attention layer as the current attention layer, and repeating the reasoning process until a final reasoning result of the large language model is output; according to the method, the transmission bottleneck caused by interlayer heterogeneity is relieved fundamentally, and low-delay and high-throughput large language model reasoning is realized.

The application belongs to the technical field of offshore target drift tracking, and specifically discloses a multi-core parallel particle tracking acceleration method for large-range marine drifting targets. First, an offshore target drift prediction model is constructed; a forecast area satellite remote sensing image is acquired to obtain an initial distribution position of the offshore target; model calculation area, particle number and prediction time length model parameters are set; model driving fields such as wind field and flow field are acquired; parallel setting is performed on a multi-core processor, including the number of calculation CPUs and the number of physical cores, the number of running threads, task division, one-time thread pool creation and scheduling strategy setting; particle state is initialized, a time advancement and drift numerical simulation process is performed, large-scale drift particles are tracked and calculated in parallel, and finally a particle trajectory set is predicted. The method realizes high-fidelity and fine drift prediction within a minute-level time limit.

The invention provides a density functional calculation-oriented cluster task scheduling method, which comprises the following steps of: estimating the execution time of each cluster task of density functional calculation, and dividing the cluster task into a large task and a small task based on the execution time; decomposing the large-scale task which is allowed to be decomposed by the task characteristics into a plurality of small-scale tasks; when the tasks run, all the small tasks are packaged into a plurality of task packages, and task distribution and running are carried out according to the sequence of first large tasks and then task packages; wherein the task package meets the condition that the total execution time of the task processes of all the execution nodes is within a gradually reduced time range. The cluster task scheduling method can well cope with the characteristics of batch tasks in the background technology, and has the advantages of load balancing scheduling based on task load evaluation, decomposition and reordering, variable-length task group packaging, an error tolerance mechanism, large-scale expandability of hierarchical expansion and relatively high parallel efficiency.

The invention provides a log collection method and system based on parallel sorting optimization, and the method specifically comprises the steps: (1) data sampling: collecting log data from different log sources, transmitting the log data to a data processing node, and obtaining a representative sample from the log data through a big data-oriented extensible regular sampling method; (2) parallel sorting: distributing the representative samples to a plurality of nodes in a system according to a certain rule, and sorting the log data on each node by using a parallel sorting algorithm; (3) parallel writing: collecting the sorted representative samples of the log data into a file system in parallel; and (4) monitoring and optimizing: monitoring the space utilization rate of a file system in real time through a verification file and an inspection tool, and regularly inspecting the integrity of log data. According to the method, a big data-oriented extensible regular sampling method is adopted to pre-process and sample logs, and a parallel sorting technology is utilized to efficiently collect log data. And then, the sampled log sample data is distributed to a plurality of processing nodes, so that parallel sorting operation is realized. According to the method, sample storage and processing tasks are scattered to each process for sorting, and pivot elements are iteratively updated, so that the parallel efficiency is improved, the processing bottleneck caused by non-uniform data distribution is reduced, and the overall performance and reliability of a log management system are improved.