A data processing method and device for high-performance parallel simulation

Abstract

The present invention discloses a data processing method and device for high-performance parallel simulation, which includes: obtaining simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks; analyzing and processing the simulation device information to obtain target device resource information; using the target device resource information to match the simulation task information to obtain target simulation resource configuration information.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method and device for high-performance parallel simulation. Background Art

[0002] In the simulation of modern complex systems, such as aerospace, weather forecasting, biomedicine and other fields, traditional serial simulation methods have been unable to meet the requirements of large-scale data processing and real-time performance. Parallel simulation technology can significantly improve simulation efficiency by decomposing computing tasks onto multiple processors or computing nodes. However, existing parallel simulation methods still have bottlenecks in data processing, task allocation and communication efficiency, which limits their application in high-performance computing. Therefore, a data processing method and device for high-performance parallel simulation are provided to improve the efficiency and level of simulation task resource optimization configuration, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the existing technology. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a data processing method and device for high-performance parallel simulation, which is conducive to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the existing technology.

[0004] In order to solve the above technical problems, a first aspect of an embodiment of the present invention discloses a data processing method for high-performance parallel simulation, the method comprising:

[0005] Acquire simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks;

[0006] Analyzing and processing the simulated device information to obtain target device resource information; the target device resource information includes first target category resource information, second target category resource information, and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information;

[0007] The simulation task information is matched with the target device resource information to obtain target simulation resource configuration information.

[0008] A second aspect of an embodiment of the present invention discloses a data processing device for high-performance parallel simulation, the device comprising:

[0009] An acquisition module is used to acquire simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks;

[0010] a first processing module configured to analyze and process the simulated device information to obtain target device resource information; the target device resource information includes first target category resource information, second target category resource information, and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information;

[0011] The second processing module is used to match the simulation task information with the target device resource information to obtain target simulation resource configuration information.

[0012] A third aspect of the present invention discloses another data processing device for high-performance parallel simulation, the device comprising:

[0013] a memory storing executable program code;

[0014] a processor coupled to a memory;

[0015] The processor calls the executable program code stored in the memory to execute part or all of the steps in the data processing method for high-performance parallel simulation disclosed in the first aspect of the embodiment of the present invention.

[0016] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the data processing method for high-performance parallel simulation disclosed in the first aspect of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 Schematic diagram of a data processing system for high-performance parallel simulation provided by an embodiment of the present invention;

[0019] Figure 2This is a flow chart of a data processing method for high-performance parallel simulation disclosed in an embodiment of the present invention;

[0020] Figure 3 This is a schematic structural diagram of a data processing device for high-performance parallel simulation disclosed in an embodiment of the present invention;

[0021] Figure 4 This is a schematic structural diagram of another data processing device for high-performance parallel simulation disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0023] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0024] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0025] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0026] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, the corresponding data exist for the computer device to process. The details will not be repeated here.

[0027] It should be noted that the artificial intelligence related technologies that may be involved in this application are briefly described. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0028] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0029] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying and measuring objects, performing further image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0030] Unimodal information is data consisting of only one type, such as text, images, audio, video, or electromagnetic signals. Multimodal information is data that includes at least two types of unimodal information. Furthermore, multimodal information is suitable for complex tasks that require integrating multiple information sources, such as sentiment analysis, robot interaction, and autonomous driving. By integrating information from multiple modalities, higher performance and accuracy can often be achieved on the task.

[0031] A large model refers to an artificial neural network model with a very large number of parameters. In the field of artificial intelligence, a large model generally refers to a model with hundreds of millions to trillions of parameters. Models usually need to be trained on large-scale data sets and require a large amount of computing resources to be optimized and adjusted. Large models are generally used to solve complex tasks such as natural language processing, computer vision, and speech recognition. Generative AI is an AI that can create new content and ideas, including conversations, stories, images, videos, and music. In the embodiment of the present application, the large model can be ChatGPT, BERT, XLNet, Zhipu model, Claude, Moonshot AI model, ChatGLM model, Tongwen Qianyi model, MiniMax model, Spark model, Llama model, 360GPT model, Qwen model, Baichuan model, Skylark model, vivoLM model, Wenxin Yiyan and other large-scale language models, which are not limited in the embodiment of the present application.

[0032] The embodiments of the present application provide a data processing method, apparatus, computer device, and computer-readable storage medium for high-performance parallel simulation, which are described in detail below.

[0033] See also Figure 1 , Figure 1This is a schematic diagram of a scenario of a data processing system for high-performance parallel simulation provided by an embodiment of the present application. The data processing system for high-performance parallel simulation may include a computer device 100, in which a data processing device for high-performance parallel simulation is integrated, such as Figure 1 Computer equipment in.

[0034] In the embodiment of the present application, the computer device 100 is mainly used to obtain simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks;

[0035] Analyzing and processing the simulated device information to obtain target device resource information; the target device resource information includes first target category resource information, second target category resource information, and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information;

[0036] The simulation task information is matched with the target device resource information to obtain target simulation resource configuration information.

[0037] It can improve the efficiency and level of optimal configuration of simulation task resources, thereby solving problems such as low data processing efficiency and unbalanced task distribution in existing technologies.

[0038] In the embodiments of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiments of the present application includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. A cloud server is composed of a large number of computers or network servers based on cloud computing.

[0039] It is understood that the computer device 100 used in the embodiments of the present application can be a device that includes both receiving and transmitting hardware, that is, a device that has receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device that has a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. The specific computer device 100 can be a desktop terminal or a mobile terminal. The computer device 100 can also be a mobile phone, a tablet computer, a laptop computer, etc.

[0040] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer computer devices as shown in Figure 1 Only one computer device is shown. It can be understood that the data processing system for high-performance parallel simulation can also include one or more other services, which are not specifically limited here.

[0041] In addition, if Figure 1 As shown, the data processing system for high-performance parallel simulation may further include a memory 200 for storing data, such as image data, position information, and the like.

[0042] It should be noted that Figure 1 The scenario diagram of the data processing system for high-performance parallel simulation shown is merely an example. The data processing system and scenario for high-performance parallel simulation described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person skilled in the art will appreciate that, with the evolution of the data processing system for high-performance parallel simulation and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is equally applicable to similar technical problems.

[0043] The present invention discloses a data processing method and device for high-performance parallel simulation, which is beneficial for improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and uneven task allocation in the prior art. A detailed description is given below.

[0044] Example 1

[0045] See also Figure 2 , Figure 2 This is a flow chart of a data processing method for high-performance parallel simulation disclosed in an embodiment of the present invention. Figure 2 The data processing method for high-performance parallel simulation described above is applied to a management system, such as a local server or a cloud server for management, and is not limited in the embodiment of the present invention. Figure 2 As shown, the data processing method for high-performance parallel simulation may include the following operations:

[0046] 101. Obtain simulation task information and simulation device information.

[0047] In an embodiment of the present invention, the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; and the simulation task information includes M simulation tasks.

[0048] 102. Analyze and process the simulated device information to obtain target device resource information.

[0049] In an embodiment of the present invention, the target device resource information includes first target category resource information, second target category resource information and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information.

[0050] 103. Use the target device resource information to match the simulation task information to obtain the target simulation resource configuration information.

[0051] It should be noted that the above-mentioned target simulation resource configuration information is used to guide the high-performance parallel operation of simulation tasks, and the embodiment of the present invention does not limit this.

[0052] It should be noted that the above simulation tasks can be various types of numerical simulations, semi-physical simulations, etc., which are not limited in the embodiments of the present invention. Furthermore, the above simulation tasks can be obtained by user setting assumptions and then analyzing them, which are not limited in the embodiments of the present invention.

[0053] It should be noted that the above-mentioned simulation hardware resource information can be automatically collected by the system. Furthermore, the above-mentioned initial weight information can be set by the user, or it can be the target weight information of the hardware resources formed after the previous simulation task information is completed. The embodiment of the present invention does not limit this.

[0054] It should be noted that the first hardware resource may represent the remaining available GPU / CPU ratio of a server, which is not limited in the embodiment of the present invention.

[0055] It should be noted that the second hardware resource information may represent the remaining available memory ratio of a server, which is not limited in the embodiment of the present invention.

[0056] It should be noted that the first target category resource information represents a server with sufficient remaining available hardware resources, which is not limited in the present embodiment. Furthermore, the server corresponding to the first target category resource information can host and run more simulation tasks to achieve an effective match between hardware resources and simulation tasks, which is not limited in the present embodiment.

[0057] It should be noted that the above second target category resource information indicates that the tasks carried by the server are relatively balanced. If the first target category resources are sufficient, the second target category resources may not be used to carry and run simulation tasks, which is not limited in this embodiment of the present invention.

[0058] It should be noted that the third target category resource information indicates that the simulation task carried and run by the server is already in a heavy load state and is no longer suitable for allocating simulation tasks, which is not limited in the embodiment of the present invention.

[0059] It should be noted that the above M, N and H are positive integers greater than or equal to 1, and I and J are integers greater than or equal to 0, which are not limited in the embodiment of the present invention.

[0060] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0061] In an optional embodiment, the above-mentioned analysis and processing of the simulated device information to obtain the target device resource information includes:

[0062] Calculating and updating the simulated device information to obtain first device resource information; the first device resource information includes N device update resource information; the device update resource information includes target weight information, first hardware resource information, and second hardware resource information;

[0063] The first device resource information is classified and processed to obtain target device resource information.

[0064] It should be noted that the above-mentioned calculation and update processing of the simulation device information is for the purpose of dynamically updating the status of the hardware resources, so as to more efficiently match the hardware resources and simulation tasks according to the remaining status of each hardware resource before executing a new simulation task, thereby achieving effective load balancing configuration, which is not limited in the embodiments of the present invention.

[0065] It should be noted that the above-mentioned classification processing of the first device resource information is performed after the available status of the hardware resources is dynamically updated, and then a static secondary classification of the hardware resources is performed to further refine the classification of the hardware resource conditions of each server, so as to more accurately and efficiently classify the real-time available resources of the hardware resources, so as to more effectively allocate hardware resources to simulation tasks, thereby improving the effective parallel utilization efficiency of hardware resources. The embodiments of the present invention do not limit this.

[0066] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0067] In another optional embodiment, the simulated device information is calculated and updated to obtain the first device resource information, including:

[0068] For any simulated hardware resource information in the simulated device information, performing calculation and analysis on the simulated hardware resource information to obtain target resource availability information corresponding to the simulated hardware resource information;

[0069] Utilizing the first resource calculation model to calculate and process all target resource availability information to obtain system resource availability information;

[0070] Among them, the first resource calculation model is:

[0071]

[0072] Where, XTZY represents the available information of system resources; MBKY i Representing target resource availability information corresponding to the i-th simulation hardware resource information in the simulation device information;

[0073] First device resource information is determined based on the system resource availability information and the simulation device information.

[0074] It should be noted that the above-mentioned system resource availability information represents the average status of available resources of all servers, so as to comprehensively reflect the resource availability of the entire server cluster, thereby facilitating the subsequent dynamic update of the weight of hardware resources. The embodiment of the present invention does not limit this.

[0075] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0076] In yet another optional embodiment, the simulation hardware resource information is subjected to calculation and analysis to obtain target resource availability information corresponding to the simulation hardware resource information, including:

[0077] Determine whether the first hardware resource information and the second hardware resource information corresponding to the simulation hardware resource information meet the first resource condition, and obtain a first resource determination result;

[0078] Among them, the first resource condition is:

[0079]

[0080] Wherein, ZY1 and ZY2 represent the first hardware resource information and the second hardware resource information respectively; YZ1 and YZ2 represent the first resource threshold and the second resource threshold respectively;

[0081] When the first resource determination result is no, determining 0 as the target resource availability information corresponding to the simulation hardware resource information;

[0082] When the first resource judgment result is yes, the first hardware resource information and the second hardware resource information are calculated and processed using the second resource calculation model to obtain available candidate resource information corresponding to the simulated hardware resource information;

[0083] Among them, the second resource calculation model is:

[0084]

[0085] Wherein, BXZY represents the available information of alternative resources; CSQZ represents the initial weight information; a1 and a2 represent the first calculation coefficient and the second calculation coefficient respectively;

[0086] Determine whether a value corresponding to the available information of the candidate resource is greater than or equal to a third resource threshold, and obtain a second resource determination result;

[0087] When the second resource determination result is no, determining 0 as the target resource availability information corresponding to the simulation hardware resource information;

[0088] When the second resource determination result is yes, the candidate resource available information is determined as the target resource available information corresponding to the simulation hardware resource information.

[0089] It should be noted that the first resource threshold and the second resource threshold are values ​​between 0.03 and 0.1, which are not limited in the embodiment of the present invention. Furthermore, by setting resource thresholds, it is possible to directly and quickly identify servers that are running at full load, thereby quickly updating the server's availability status (i.e., modifying the target resource availability information to 0) to indicate that it is currently unsuitable for reallocation of simulation tasks, which is not limited in the embodiment of the present invention.

[0090] It should be noted that the first calculation coefficient and the second calculation coefficient are values ​​between 0 and 1, and the sum of the two is 1, which is not limited in the embodiment of the present invention.

[0091] It should be noted that the third resource threshold is a value between 0.03 and 0.05, which is not limited in the embodiment of the present invention. Furthermore, by calculating the available information of alternative resources and analyzing and comparing it with the third resource threshold, it is intended to consider that when analyzing a single category of hardware resources, which is still relatively abundant, it is still not appropriate to allocate new simulation tasks relative to the simulation tasks that are still running on the server, so as to ensure that the running simulation tasks can be effectively run. Therefore, the third resource threshold is used to analyze the operating status of the server, thereby ensuring the effective parallel operation of the simulation tasks, which is not limited in the embodiment of the present invention.

[0092] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0093] In yet another optional embodiment, determining the first device resource information based on the system resource availability information and the simulation device information includes:

[0094] For any simulated hardware resource information in the simulated device information, the target resource availability information and initial weight information corresponding to the simulated hardware resource information, as well as the system resource availability information, are calculated and processed using the third resource calculation model to obtain target weight information of the device update resource information corresponding to the simulated hardware resource information;

[0095] Among them, the third resource calculation model is:

[0096]

[0097] Where MBZQ represents the target weight information; MBKY represents the target resource availability information; CSQZ represents the initial weight information;

[0098] The first hardware resource information and the second hardware resource information corresponding to the simulated hardware resource information are determined as the first hardware resource information and the second hardware resource information of the device update resource information corresponding to the simulated hardware resource information.

[0099] It should be noted that the above-mentioned system resource available information and simulation device information updates the task allocation weight of a single server according to the available resource situation of a single server and the available resource situation of the entire server cluster, so that the simulation tasks of the entire server cluster are evenly configured on the server, and the embodiments of the present invention do not limit this.

[0100] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0101] In an optional embodiment, the classification processing of the first device resource information to obtain the target device resource information includes:

[0102] For any device update resource information in the first device resource information, determining whether a value corresponding to target weight information corresponding to the device update resource information is greater than or equal to a first classification threshold, and obtaining a first classification determination result;

[0103] When the first classification judgment result is yes, the device update resource information is determined as a first sub-target category resource information;

[0104] When the first classification judgment result is no, determining whether the value corresponding to the target weight information corresponding to the device update resource information is greater than or equal to the second classification threshold, and obtaining a second classification judgment result;

[0105] When the second classification judgment result is yes, the device update resource information is determined as a second sub-target category resource information;

[0106] When the second classification judgment result is no, the device update resource information is determined as a third sub-target category resource information.

[0107] It should be noted that the above-mentioned first classification threshold is greater than the second classification threshold. Furthermore, the above-mentioned first classification threshold and the second classification threshold are greater than or equal to 0.5 and less than 1. This is not limited in the embodiment of the present invention. Furthermore, the above-mentioned update of the server weight dynamically updates the available status of the server's hardware resources, but its classification status in the entire server cluster is still in a relatively vague state. Therefore, in order to more accurately determine the available status of a single server relative to the entire server cluster, the hardware resource status of each server is further classified through two classification thresholds, so as to allocate hardware resources to the simulation tasks according to the classification situation, achieve balanced configuration of the simulation tasks of the entire server cluster, and improve the parallel operation efficiency of the simulation tasks. This is not limited in the embodiment of the present invention.

[0108] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0109] In another optional embodiment, the simulation task information is matched with the target device resource information to obtain the target simulation resource configuration information, including:

[0110] Determine whether M is greater than or equal to 1.5 times H, and obtain a numerical judgment result;

[0111] When the numerical judgment result is yes, the simulation task information is matched using the first target category resource information and the second target category resource information to obtain target simulation resource configuration information;

[0112] When the numerical judgment result is negative, the simulation task information is matched with the first target category resource information to obtain target simulation resource configuration information.

[0113] It should be noted that the above-mentioned judgment of whether M is greater than or equal to 1.5 times of H is based on the consideration that the available hardware resources of the first target category resource information are still relatively abundant, and the simulation task ratio of 1:1.5 can fully utilize the hardware resources of the server. If it is greater than this ratio, part of the second target category resource information in a relatively balanced load state will be requisitioned to improve the efficiency of parallel operation of simulation tasks, thereby achieving effective adaptation of efficient utilization of hardware resources and high-performance parallel operation of simulation tasks, which is not limited in the embodiments of the present invention.

[0114] It should be noted that the above-mentioned target simulation resource configuration information includes the first simulation resource configuration corresponding relationship and / or the second simulation resource configuration corresponding relationship, which is not limited in the embodiment of the present invention.

[0115] In this optional embodiment, as an optional implementation manner, the above-mentioned matching processing of the simulation task information using the first target category resource information and the second target category resource information to obtain the target simulation resource configuration information includes:

[0116] Sort the simulation tasks in the simulation task information by the hardware resources required for running them from large to small to obtain a simulation task sequence;

[0117] Determine the first Z simulation tasks in the simulation task sequence as a first simulation task queue, and determine the remaining simulation tasks as a second simulation task queue;

[0118] Sort the first sub-target category resource information in the first target category resource information from large to small according to the values ​​corresponding to the target weight information to obtain a first sub-target category resource information queue;

[0119] Sort the second sub-target category resource information in the second target category resource information from large to small according to the values ​​corresponding to the target weight information to obtain a second sub-target category resource information queue;

[0120] Establishing an association relationship between the first sub-goal category resource information queue and the first simulation task queue to obtain a first simulation resource configuration correspondence relationship;

[0121] An association relationship is established between the second sub-goal category resource information queue and the second simulation task queue to obtain a second simulation resource configuration correspondence relationship.

[0122] It should be noted that the above Z is This is not limited in the embodiments of the present invention.

[0123] It should be noted that the above-mentioned first simulation resource configuration correspondence relationship represents that when the simulation task is running, the simulation tasks in the first simulation task queue are sequentially configured to the server corresponding to the first sub-target category resource information in the first sub-target category resource information queue to run the simulation task. After a simulation task is completed, a simulation task is sequentially extracted from the first simulation task queue and configured to the server corresponding to the newly vacant first sub-target category resource information. The embodiment of the present invention does not limit this.

[0124] It should be noted that the operating logic of the second simulation resource configuration correspondence relationship is consistent with the first simulation resource configuration correspondence relationship, and the embodiment of the present invention does not limit this.

[0125] It should be noted that the above-mentioned matching process of the simulation task information with the first target category resource information to obtain the target simulation resource configuration information only establishes the first simulation resource configuration correspondence, which is not limited in the embodiment of the present invention.

[0126] It can be seen that implementing the data processing method for high-performance parallel simulation described in the embodiment of the present invention is conducive to improving the efficiency and level of optimization configuration of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task distribution in the existing technology.

[0127] Example 2

[0128] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a data processing device for high-performance parallel simulation disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., and the embodiment of the present invention does not limit this. Figure 3 As shown, the device may include:

[0129] Acquisition module 201, for acquiring simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks;

[0130] The first processing module 202 is configured to analyze and process the simulated device information to obtain target device resource information; the target device resource information includes first target category resource information, second target category resource information, and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information;

[0131] The second processing module 203 is configured to perform matching processing on the simulation task information using the target device resource information to obtain target simulation resource configuration information.

[0132] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0133] In another optional embodiment, Figure 3 As shown, the simulated device information is analyzed and processed to obtain the target device resource information, including:

[0134] Calculating and updating the simulated device information to obtain first device resource information; the first device resource information includes N device update resource information; the device update resource information includes target weight information, first hardware resource information, and second hardware resource information;

[0135] The first device resource information is classified and processed to obtain target device resource information.

[0136] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0137] In another optional embodiment, Figure 3 As shown, the simulation device information is calculated and updated to obtain the first device resource information, including:

[0138] For any simulated hardware resource information in the simulated device information, performing calculation and analysis on the simulated hardware resource information to obtain target resource availability information corresponding to the simulated hardware resource information;

[0139] Utilizing the first resource calculation model to calculate and process all target resource availability information to obtain system resource availability information;

[0140] Among them, the first resource calculation model is:

[0141]

[0142] Where, XTZY represents the available information of system resources; MBKY i Representing target resource availability information corresponding to the i-th simulation hardware resource information in the simulation device information;

[0143] First device resource information is determined based on the system resource availability information and the simulation device information.

[0144] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0145] In another optional embodiment, Figure 3 As shown, the simulation hardware resource information is calculated and analyzed to obtain the target resource availability information corresponding to the simulation hardware resource information, including:

[0146] Determine whether the first hardware resource information and the second hardware resource information corresponding to the simulation hardware resource information meet the first resource condition, and obtain a first resource determination result;

[0147] Among them, the first resource condition is:

[0148]

[0149] Wherein, ZY1 and ZY2 represent the first hardware resource information and the second hardware resource information respectively; YZ1 and YZ2 represent the first resource threshold and the second resource threshold respectively;

[0150] When the first resource determination result is no, determining 0 as the target resource availability information corresponding to the simulation hardware resource information;

[0151] When the first resource judgment result is yes, the first hardware resource information and the second hardware resource information are calculated and processed using the second resource calculation model to obtain available candidate resource information corresponding to the simulated hardware resource information;

[0152] Among them, the second resource calculation model is:

[0153]

[0154] Wherein, BXZY represents the available information of alternative resources; CSQZ represents the initial weight information; a1 and a2 represent the first calculation coefficient and the second calculation coefficient respectively;

[0155] Determine whether a value corresponding to the available information of the candidate resource is greater than or equal to a third resource threshold, and obtain a second resource determination result;

[0156] When the second resource determination result is no, determining 0 as the target resource availability information corresponding to the simulation hardware resource information;

[0157] When the second resource determination result is yes, the candidate resource available information is determined as the target resource available information corresponding to the simulation hardware resource information.

[0158] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0159] In another optional embodiment, Figure 3 As shown, based on the system resource available information and the simulation device information, the first device resource information is determined, including:

[0160] For any simulated hardware resource information in the simulated device information, the target resource availability information and initial weight information corresponding to the simulated hardware resource information, as well as the system resource availability information, are calculated and processed using the third resource calculation model to obtain target weight information of the device update resource information corresponding to the simulated hardware resource information;

[0161] Among them, the third resource calculation model is:

[0162]

[0163] Where MBZQ represents the target weight information; MBKY represents the target resource availability information; CSQZ represents the initial weight information;

[0164] The first hardware resource information and the second hardware resource information corresponding to the simulated hardware resource information are determined as the first hardware resource information and the second hardware resource information of the device update resource information corresponding to the simulated hardware resource information.

[0165] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0166] In another optional embodiment, Figure 3 As shown, the first device resource information is classified and processed to obtain target device resource information, including:

[0167] For any device update resource information in the first device resource information, determining whether a value corresponding to target weight information corresponding to the device update resource information is greater than or equal to a first classification threshold, and obtaining a first classification determination result;

[0168] When the first classification judgment result is yes, the device update resource information is determined as a first sub-target category resource information;

[0169] When the first classification judgment result is no, determining whether the value corresponding to the target weight information corresponding to the device update resource information is greater than or equal to the second classification threshold, and obtaining a second classification judgment result;

[0170] When the second classification judgment result is yes, the device update resource information is determined as a second sub-target category resource information;

[0171] When the second classification judgment result is no, the device update resource information is determined as a third sub-target category resource information.

[0172] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0173] In another optional embodiment, Figure 3 As shown, the simulation task information is matched with the target device resource information to obtain the target simulation resource configuration information, including:

[0174] Determine whether M is greater than or equal to 1.5 times H, and obtain a numerical judgment result;

[0175] When the numerical judgment result is yes, the simulation task information is matched using the first target category resource information and the second target category resource information to obtain target simulation resource configuration information;

[0176] When the numerical judgment result is negative, the simulation task information is matched with the first target category resource information to obtain target simulation resource configuration information.

[0177] It can be seen that implementation Figure 3 The described data processing device for high-performance parallel simulation is beneficial to improving the efficiency and level of optimal allocation of simulation task resources, thereby solving the problems of low data processing efficiency and unbalanced task allocation in the prior art.

[0178] Example 3

[0179] See also Figure 4 , Figure 4This is a schematic diagram of the structure of another data processing device for high-performance parallel simulation disclosed in an embodiment of the present invention. Figure 4 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., and the embodiment of the present invention does not limit this. Figure 4 As shown, the device may include:

[0180] A memory 301 storing executable program code;

[0181] a processor 302 coupled to the memory 301;

[0182] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the data processing method for high-performance parallel simulation described in the first embodiment.

[0183] Example 4

[0184] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the data processing method for high-performance parallel simulation described in the first embodiment.

[0185] Example 5

[0186] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the data processing method for high-performance parallel simulation described in embodiment 1.

[0187] The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0188] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0189] Finally, it should be noted that the data processing method and device for high-performance parallel simulation disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data processing method for high-performance parallel simulation, characterized in that: The method comprises: Acquire simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks; Analyzing and processing the simulated device information to obtain target device resource information; the target device resource information includes first target category resource information, second target category resource information, and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information; Matching the simulation task information with the target device resource information to obtain target simulation resource configuration information; The step of matching the simulation task information with the target device resource information to obtain target simulation resource configuration information includes: Determine whether M is greater than or equal to 1.5 times H, and obtain a numerical judgment result; When the numerical judgment result is yes, matching the simulation task information with the first target category resource information and the second target category resource information to obtain target simulation resource configuration information; When the numerical judgment result is negative, matching the simulation task information with the first target category resource information to obtain target simulation resource configuration information; The matching process of the simulation task information using the first target category resource information and the second target category resource information to obtain the target simulation resource configuration information includes: Sort the simulation tasks in the simulation task information by the hardware resources required for running them from large to small to obtain a simulation task sequence; Determine the first Z simulation tasks in the simulation task sequence as a first simulation task queue, and determine the remaining simulation tasks as a second simulation task queue; Sort the first sub-target category resource information in the first target category resource information from large to small according to the values ​​corresponding to the target weight information to obtain a first sub-target category resource information queue; Sort the second sub-target category resource information in the second target category resource information from large to small according to the values ​​corresponding to the target weight information to obtain a second sub-target category resource information queue; Establishing an association relationship between the first sub-goal category resource information queue and the first simulation task queue to obtain a first simulation resource configuration correspondence relationship; An association relationship is established between the second sub-goal category resource information queue and the second simulation task queue to obtain a second simulation resource configuration correspondence relationship.

2. The data processing method for high-performance parallel simulation according to claim 1, characterized in that: The analyzing and processing the simulated device information to obtain target device resource information includes: Performing calculation and update processing on the simulated device information to obtain first device resource information; the first device resource information includes N device update resource information; the device update resource information includes target weight information, the first hardware resource information, and the second hardware resource information; The first device resource information is classified and processed to obtain target device resource information.

3. The data processing method for high-performance parallel simulation according to claim 2, characterized in that: The calculating and updating the simulation device information to obtain the first device resource information includes: For any of the simulated hardware resource information in the simulated device information, performing calculation and analysis on the simulated hardware resource information to obtain target resource availability information corresponding to the simulated hardware resource information; Utilizing the first resource calculation model to calculate and process all the target resource availability information to obtain system resource availability information; The first resource calculation model is: Wherein, XTZY represents the available information of the system resources; MBKY i Representing target resource availability information corresponding to the ith simulation hardware resource information in the simulation device information; First device resource information is determined based on the system resource availability information and the simulation device information.

4. The data processing method for high-performance parallel simulation according to claim 3, characterized in that: The calculation and analysis processing of the simulation hardware resource information to obtain target resource availability information corresponding to the simulation hardware resource information includes: Determine whether the first hardware resource information and the second hardware resource information corresponding to the simulation hardware resource information meet a first resource condition, and obtain a first resource determination result; The first resource condition is: Wherein, ZY1 and ZY2 represent the first hardware resource information and the second hardware resource information respectively; YZ1 and YZ2 represent the first resource threshold and the second resource threshold respectively; When the first resource determination result is no, determining 0 as the target resource availability information corresponding to the simulation hardware resource information; When the first resource determination result is yes, the first hardware resource information and the second hardware resource information are calculated and processed using a second resource calculation model to obtain available candidate resource information corresponding to the simulated hardware resource information; The second resource calculation model is: Wherein, BXZY represents the available information of the candidate resources; CSQZ represents the initial weight information; a1 and a2 represent the first calculation coefficient and the second calculation coefficient respectively; Determine whether a value corresponding to the available information of the candidate resource is greater than or equal to a third resource threshold, and obtain a second resource determination result; When the second resource determination result is no, determining 0 as the target resource availability information corresponding to the simulation hardware resource information; When the second resource determination result is yes, the candidate resource available information is determined as the target resource available information corresponding to the simulation hardware resource information.

5. The data processing method for high-performance parallel simulation according to claim 3, characterized in that: The determining of first device resource information based on the system resource availability information and the simulation device information includes: For any of the simulated hardware resource information in the simulated device information, use a third resource calculation model to calculate the target resource availability information and initial weight information corresponding to the simulated hardware resource information, as well as the system resource availability information, to obtain target weight information of the device update resource information corresponding to the simulated hardware resource information; Wherein, the third resource calculation model is: Wherein, MBZQ represents the target weight information; MBKY represents the target resource availability information; CSQZ represents the initial weight information; The first hardware resource information and the second hardware resource information corresponding to the simulated hardware resource information are determined as the first hardware resource information and the second hardware resource information of the device update resource information corresponding to the simulated hardware resource information.

6. The data processing method for high-performance parallel simulation according to claim 2, characterized in that: The classifying and processing the first device resource information to obtain target device resource information includes: For any of the device update resource information in the first device resource information, determining whether a value corresponding to the target weight information corresponding to the device update resource information is greater than or equal to a first classification threshold, and obtaining a first classification determination result; When the first classification judgment result is yes, determining the device update resource information as a first sub-target category resource information; When the first classification judgment result is no, determining whether the value corresponding to the target weight information corresponding to the device update resource information is greater than or equal to a second classification threshold, and obtaining a second classification judgment result; When the second classification judgment result is yes, determining the device update resource information as a second sub-target category resource information; When the second classification judgment result is no, the device update resource information is determined as resource information of the third sub-target category.

7. A data processing device for high-performance parallel simulation, characterized in that: The device comprises: An acquisition module is used to acquire simulation task information and simulation device information; the simulation device information includes N simulation hardware resource information; the simulation hardware resource information includes initial weight information, first hardware resource information and second hardware resource information; the simulation task information includes M simulation tasks; a first processing module configured to analyze and process the simulated device information to obtain target device resource information; the target device resource information includes first target category resource information, second target category resource information, and third target category resource information; the first target category resource information includes H first sub-target category resource information; the second target category resource information includes I second sub-target category resource information; and the third target category resource information includes J third sub-target category resource information; A second processing module is used to match the simulation task information with the target device resource information to obtain target simulation resource configuration information; The step of matching the simulation task information with the target device resource information to obtain target simulation resource configuration information includes: Determine whether M is greater than or equal to 1.5 times H, and obtain a numerical judgment result; When the numerical judgment result is yes, matching the simulation task information with the first target category resource information and the second target category resource information to obtain target simulation resource configuration information; When the numerical judgment result is negative, matching the simulation task information with the first target category resource information to obtain target simulation resource configuration information; The matching process of the simulation task information using the first target category resource information and the second target category resource information to obtain the target simulation resource configuration information includes: Sort the simulation tasks in the simulation task information by the hardware resources required for running them from large to small to obtain a simulation task sequence; Determine the first Z simulation tasks in the simulation task sequence as a first simulation task queue, and determine the remaining simulation tasks as a second simulation task queue; Sort the first sub-target category resource information in the first target category resource information from large to small according to the values ​​corresponding to the target weight information to obtain a first sub-target category resource information queue; Sort the second sub-target category resource information in the second target category resource information from large to small according to the values ​​corresponding to the target weight information to obtain a second sub-target category resource information queue; Establishing an association relationship between the first sub-goal category resource information queue and the first simulation task queue to obtain a first simulation resource configuration correspondence relationship; An association relationship is established between the second sub-goal category resource information queue and the second simulation task queue to obtain a second simulation resource configuration correspondence relationship.

8. A data processing device for high-performance parallel simulation, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the data processing method for high-performance parallel simulation according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the data processing method for high-performance parallel simulation according to any one of claims 1 to 6.

Images