Model reasoning acceleration method, system, computer device and readable storage medium

Abstract

This application relates to a model inference acceleration method, system, computer device, and readable storage medium. The method includes: obtaining a model file for a model to be processed; parsing the model file to obtain all operators of the model to be processed; if a physical accelerator card is not present, loading the model file into the CPU's memory space, determining a target operator that matches all operators from a user-preset virtual operator library; mapping each operator to its corresponding target operator, and accelerating the inference of the model to be processed based on the target operator. This method can improve resource utilization.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model reasoning acceleration method, system, computer device, and readable storage medium. Background Art

[0002] In edge / device inference scenarios, neural network models require high computational complexity and place high demands on hardware performance. Traditional central processing units (CPUs), as general-purpose processors, offer high flexibility but are limited in their efficiency when handling large-scale parallel computing tasks. To accelerate specific tasks, specialized acceleration chips such as graphics processing units (GPUs), tensor processing units (TPUs), and field-programmable gate arrays (FPGAs) (collectively known as xPUs) have emerged. These chips feature architectural optimizations for compute-intensive tasks like neural networks, significantly improving computational speed.

[0003] In the related art, model inference acceleration relies directly on physical accelerator cards (xPUs) to perform neural network inference tasks. This involves loading the model directly into the accelerator card's memory and utilizing the card's built-in operator library to perform inference calculations. However, this reliance on accelerator cards results in low resource utilization. Therefore, a model inference acceleration method that can improve resource utilization is needed. Summary of the Invention

[0004] Based on this, it is necessary to provide a model reasoning acceleration method, system, computer device, computer-readable storage medium and computer program product that can improve resource utilization in response to the above technical problems.

[0005] In a first aspect, the present application provides a model reasoning acceleration method, comprising:

[0006] Obtain the model file to be processed of the model to be processed;

[0007] Parsing the model file to be processed to obtain model information of the model to be processed;

[0008] If there is no physical accelerator card, the model file to be processed is loaded into the memory space of the CPU, and a target operator matching all operators in the model information is determined from a virtual operator library preset in the user state;

[0009] Each of the operators is mapped to a corresponding target operator, and the inference of the model to be processed is completed based on the target operator.

[0010] In one embodiment, the method further comprises:

[0011] Obtaining the memory capacity required for executing reasoning on the model file to be processed and the current memory pool capacity;

[0012] If the current memory pool capacity is less than the memory capacity, a dynamic expansion mechanism is triggered to determine the memory capacity allocated to the model file to be processed in the current memory pool;

[0013] The parsing of the model file to be processed to obtain model information of the model to be processed includes:

[0014] Converting the model file to be processed into a model intermediate representation, and determining a computational graph of the model file to be processed, namely, a computational graph;

[0015] According to the calculation graph, all operators of the model to be processed are obtained.

[0016] In one embodiment, mapping each of the operators to a corresponding target operator and completing the inference of the model to be processed based on the target operator includes:

[0017] Mapping each of the operators to its corresponding target operator, and calling the target operator to perform calculations based on the dependency relationship between the operators in the calculation graph to obtain calculation results;

[0018] Processing is performed according to the calculation results to obtain inference results, and the inference of the model to be processed is completed.

[0019] In one embodiment, the method further comprises:

[0020] If a physical accelerator card exists and the physical accelerator card supports all operators in the model information, then the model file to be processed is loaded into the video memory space of the physical accelerator card;

[0021] Call the user-state application layer interface, route all the operators to the inference operator library of the physical acceleration card through the dynamic routing engine of the unified operator library, map each operator to its corresponding target operator in the inference operator library, and complete the inference of the model to be processed based on the inference operator library.

[0022] In one embodiment, the method further comprises:

[0023] If there is a physical acceleration card and the physical acceleration card supports some of the operators, the model file to be processed is loaded into the video memory space of the physical acceleration card;

[0024] Calling the user-state application layer interface, routing some of the operators to the inference operator library of the physical accelerator card through the dynamic routing engine of the unified operator library, and mapping each of the operators in the part of the operators to the corresponding target operator in the inference operator library;

[0025] Load the remaining operators into the memory space of the CPU, determine a virtual operator library that matches all the remaining operators from the virtual accelerator card operator library preset in the user state, and map the remaining operators to the virtual operator library;

[0026] Based on the inference operator library and the virtual operator library, the inference of the model to be processed is completed.

[0027] In one embodiment, the accelerating the inference of the model to be processed based on the inference operator library and the virtual operator library includes:

[0028] Mapping the remaining operators to the virtual operator library, and calling corresponding operators in the virtual operator library to perform calculations based on the dependency relationships between the remaining operators to obtain a first calculation result;

[0029] Sending the first calculation result to the physical accelerator card through a preset shared memory channel;

[0030] Mapping some of the operators to the inference operator library, and calling corresponding operators in the inference operator library to perform calculations based on dependencies between some of the operators to obtain a second calculation result;

[0031] The first calculation result and the second calculation result are integrated through the physical acceleration card to complete the reasoning of the model to be processed.

[0032] In one embodiment, the method further comprises:

[0033] When it is detected that the physical acceleration card is abnormal or the target operator is missing, the operator supported by the physical acceleration card or the operator corresponding to the target operator is loaded into the memory space of the CPU, and the step of determining the target operator that matches all operators in the model information from the virtual operator library preset in the user state is performed.

[0034] In a second aspect, the present application further provides a model reasoning acceleration system, the system comprising a physical hardware layer, a virtual acceleration card, and an application layer, the physical hardware layer comprising a CPU, wherein:

[0035] The application layer is used to initiate an inference request and determine a to-be-processed model file of the to-be-processed model carried in the inference request;

[0036] The virtual accelerator card is used to obtain the model file to be processed; parse the model file to be processed to obtain model information of the model to be processed; if no physical accelerator card exists, load the model file into the memory space of the CPU, determine the target operator that matches all operators in the model information from the virtual operator library preset in the user state; map each of the operators to its corresponding target operator, and complete the inference of the model to be processed based on the target operator.

[0037] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0039] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0040] The above-mentioned model reasoning acceleration method, system, computer device, computer-readable storage medium and computer program product, when reasoning on the model, parses the pending model file of the pending model to obtain all the operators of the model, and in the absence of a physical acceleration card, loads the model file into the memory space of the CPU, and determines the target operator that matches all operators from the virtual operator library preset in the user state; maps each operator to its corresponding target operator, and accelerates the reasoning of the pending model based on the target operator, avoiding the current situation where hardware resource configuration is limited and reasoning acceleration cannot be achieved without an acceleration card, fully taps the potential of the CPU, improves the resource utilization of the CPU, and in addition, meets diverse business needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of the architecture of a model reasoning acceleration system in one embodiment;

[0043] Figure 2 1. A schematic diagram of a flow chart of a model reasoning acceleration method according to an embodiment;

[0044] Figure 3 A schematic flow chart of a model reasoning acceleration method according to another embodiment;

[0045] Figure 4 This is a timing diagram of a model reasoning acceleration method based on a virtual accelerator card and an xPU in one embodiment;

[0046] Figure 5 This is an application flow chart of a model reasoning acceleration method in one embodiment;

[0047] Figure 6 A structural block diagram of a model reasoning acceleration system in one embodiment;

[0048] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0050] In edge / device inference scenarios, neural network models require high computational complexity and demand high hardware performance. To accelerate specific tasks, specialized accelerator chips such as GPUs, TPUs, and FPGAs (collectively referred to as xPUs) have emerged. However, xPUs are subject to high costs and adaptability issues. Different vendors offer varying programming models and interfaces for xPUs, making development challenging. Furthermore, in some scenarios, hardware resources are limited, making it impossible to equip a complete set of accelerator cards, or accelerator cards may lack certain features, making it difficult to meet diverse business needs.

[0051] Current solutions for model inference acceleration typically rely directly on physical accelerator cards (xPUs) for neural network inference tasks. During the model loading phase, the model is loaded directly into the accelerator card's memory space, and inference calculations are performed using the accelerator card's built-in operator library. When encountering operators or tasks unsupported by the accelerator card, or when a physical accelerator card is not available, a complex adaptation layer is often required to attempt to convert the operator or abandon the task branch, resulting in inefficient development and prone to errors. Furthermore, system resource management is relatively extensive, failing to fully exploit the potential of collaborative work between the CPU and accelerator card. This makes it difficult to flexibly allocate resources, especially in resource-constrained scenarios, and can easily lead to task stalls or even failures.

[0052] Aiming at the situation where model inference acceleration cannot be achieved due to the absence of a physical acceleration card, a model inference acceleration method is proposed. By introducing a virtual acceleration card, a virtual operator library is preset in the user state of the virtual acceleration card. When there is no physical acceleration card, the model file to be processed is loaded into the memory space of the CPU, and the target operator that matches all operators is determined from the virtual operator library preset in the user state to achieve model inference acceleration.

[0053] The model reasoning acceleration method provided in the embodiment of the present application can be applied to Figure 1 In the system shown, the system is deployed at a terminal, which may be an edge device. Figure 1 The system architecture shown, from bottom to top, consists of the physical hardware layer, kernel mode, user mode, and application layer. The virtual accelerator card consists of both kernel mode and user mode components. The physical hardware layer is the underlying resource managed by the virtual accelerator card and actually performs computing tasks. It can include the CPU, or the CPU and xPU. The CPU may include at least SIMD and vector registers, and the xPU may include at least GPUs, TPUs, and FPGAs. It should be noted that the virtual accelerator card does not include underlying module support; the physical hardware layer is the underlying resource managed by the virtual accelerator card system. As a software-defined middle layer, the virtual accelerator card abstracts and schedules the physical hardware (CPU, xPU) through kernel mode and user mode modules. However, the virtual accelerator card itself does not contain physical hardware components. The relationship between the physical hardware layer and the virtual accelerator card can be compared to that between an operating system and computer hardware: the operating system manages the hardware, but the hardware itself exists independently.

[0054] The virtual accelerator card consists of two parts: user mode and kernel mode. The user mode includes the application interface layer and the computing function layer. The application interface layer (vCardMg) serves as the interface for user programs to interact with the virtual accelerator card system and provides an interface for managing the virtual accelerator card throughout its lifecycle. The application interface layer includes multiple submodules, including virtual card instance management and resource status monitoring.

[0055] Virtual card instance management is used to call the instance creation API to trigger kernel-mode memory pool allocation and generate a virtual card object with a unique ID. Memory pool allocation uses a "pre-allocation + dynamic expansion" strategy. The "pre-allocation + dynamic expansion" strategy can be understood as the system initially allocating base memory. When insufficient initial memory is detected, on-demand expansion is triggered. The on-demand expansion here can be based on a pre-set ratio or a ratio determined based on the size of the model file to be processed.

[0056] Virtual card instance management can also be used to destroy instances, releasing associated memory pools and operator resources and notifying the kernel to reclaim them. For example, during model inference acceleration, idle memory pools and operator resources are released, or when inference ends, the associated memory pools and operator resources are released. Furthermore, virtual card instance management can be used for real-time status queries, obtaining metrics such as memory usage of each virtual card through the virtual file system, and implementing an event subscription mechanism: it supports asynchronous monitoring of accelerator card abnormal events (such as xPU overheating and insufficient memory).

[0057] The computing function layer includes a unified operator library (vOplib) and an image processing library (vImage). The unified operator library, implemented based on the SIMD instruction set, provides the operator library required for inference. This can also be understood as providing a unified operator interface across hardware, enabling dynamic routing and optimized execution of computing tasks. Core submodules include a virtual operator library and an operator dynamic routing engine. The virtual operator library primarily implements commonly used operators, including those used in common neural networks, such as convolution and pooling. The dynamic routing engine maintains a hardware-supported matrix to determine the execution path for each operator. The SIMD (Single Instruction Multiple Data) optimization kernel automatically selects the optimal implementation for different CPU instruction sets.

[0058] The Image Processing Library (vImage) provides hardware-accelerated image preprocessing based on SIMD, compatible with the OpenCV interface specification. The core submodules of the Image Processing Library (vImage) include image preprocessing and SIMD optimization algorithms. Image preprocessing: Supports common image preprocessing operations such as scaling and normalization. SIMD optimization algorithms: Provides hardware-optimized image processing algorithms based on SIMD, compatible with OpenCV.

[0059] The kernel state includes the core driver layer (vCard_Kernel) and the resource management layer. The core driver layer is used to manage physical hardware resources and provide a secure and isolated underlying operation interface. Memory pool management includes dynamic memory allocation through "pre-allocation + dynamic expansion". At the same time, a secure isolation mechanism is established to create an independent memory namespace for each virtual card instance to prevent cross-boundary access. It should be noted that one virtual accelerator card corresponds to one virtual card instance, or there can be multiple virtual card instances. This example uses one virtual accelerator card corresponding to one virtual card instance as an example. Providing a secure and isolated underlying operation interface includes defining a hardware abstraction interface (HAL) and a standardized xPU operation set to shield differences in drivers from different manufacturers.

[0060] The resource management layer includes a cross-device memory mapping module and a hardware monitor. The cross-device memory mapping module is used to create an independent memory mapping table for each virtual card instance; it receives memory allocation instructions from vCardMg (user mode) and is coordinated and executed by vCard_Kernel. Specifically, it can be manifested as: unified address space management: mapping CPU memory and xPU video memory to the same virtual address space to achieve zero-copy data transmission, and memory sharing protocol: defining cross-device memory access rules.

[0061] The hardware monitor monitors hardware resource usage in real time, providing real-time data support for the virtual accelerator's dynamic routing (vOplib) and memory allocation (vCard_Kernel). This includes real-time hardware status collection: using the PMU (Performance Monitoring Unit) to read xPU computing power utilization, temperature, power consumption, and other indicators.

[0062] In the above system, by introducing a virtual accelerator card and combining it with the CPU's memory, vector registers and SIMD instruction set technology, the differences between different xPUs can be shielded, and a unified resource management and operator calling interface can be provided. Regardless of whether there is an accelerator card at the bottom layer or whether the accelerator card function is complete, the continuity of the reasoning task can be guaranteed; on the other hand, the potential of the CPU is deeply tapped. Through the optimized memory management module and rich operator library, the CPU is used for efficient image processing and neural network reasoning when the accelerator card resources are insufficient, realizing dynamic collaboration between the CPU and xPU, and greatly improving the overall computing performance and resource utilization.

[0063] In one embodiment, Figure 2 As shown, a model reasoning acceleration method is provided. This embodiment uses the method applied to a terminal as an example. The terminal can be an edge device with Figure 1 The model reasoning acceleration system shown in the figure can be understood that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0064] Step 202: Obtain the model file to be processed of the model to be processed.

[0065] The model to be processed can be, but is not limited to, an image model. Image models can be used for image classification and object recognition. For example, they can automatically identify objects (such as cats, dogs, and cars) in photos. Another example is determining the presence of abnormal tissue in medical image analysis, or identifying specific people or behaviors in security monitoring scenarios. The model file to be processed can be pre-trained by the user and stored locally or in the cloud, depending on the application deployment method. The model file can contain the neural network model structure and parameters. The model file format can be, but is not limited to, ONNX (Open Neural Network Exchange), TensorFlowSavedModel / PB format, and PyTorch .pth / .pt format. ONNX (Open Neural Network Exchange): An open deep learning model exchange format that supports model conversion between multiple frameworks. TensorFlowSavedModel / PB format: A file format for saving TensorFlow models. PyTorch .pth / .pt format: A file format for saving PyTorch models.

[0066] It's important to note that before model inference acceleration begins, the system deployed on the terminal must be initialized. This includes initializing the memory subsystem, loading and optimizing operators, and instantiating the virtual accelerator card. Memory subsystem initialization involves coordinating the user-mode management interface (vCardMg) with the kernel-mode driver (vCard_Kernel) to start the kernel-mode memory management module, specifically memory pool management, to reserve isolated memory space for the operator library, model parameters, and virtual accelerator card instances. Furthermore, a cross-device memory mapping module is used to establish a shared memory channel between the CPU and the xPU.

[0067] Operator loading and optimization involves loading the user-mode unified operator library (vOplib), parsing the hardware support matrix, dynamically selecting the optimal operator implementation based on the current CPU instruction set, and optimizing operator parameters based on CPU cache characteristics. Virtual accelerator card instantiation involves calling vCardMg to create a virtual card instance and establishing a communication link between user mode and kernel mode. This communication link can be established using ioctl commands and shared memory.

[0068] Exemplarily, the model inference acceleration system deployed on the terminal is initialized and, in response to a model inference acceleration instruction, the model file of the model to be processed is obtained from a local disk, a remote server, or cloud storage. Furthermore, after obtaining the model file, to ensure the reliability of the model inference acceleration, the accuracy of the model file can be verified, for example, by verifying the file format and integrity of the model file.

[0069] Step 204: parse the model file to be processed to obtain model information of the model to be processed.

[0070] Model information includes the model's structure and parameters, and the structure includes the operators of the model to be processed. Operators can be understood as the basic computational units in neural networks, such as convolution, pooling, and activation functions. The model file to be processed can be parsed using existing methods and will not be detailed here.

[0071] For example, file parsing can include identifying the format of a model file and extracting the model's structural information and parameters. Based on this, operator extraction is then performed to identify and extract all operators in the model and their attributes, including input and output dimensions, weights, and so on.

[0072] Step 206: If there is no physical accelerator card, the model file to be processed is loaded into the memory space of the CPU, and a target operator that matches all operators in the model information is determined from the virtual operator library preset in the user state.

[0073] Among them, physical acceleration cards (xPUs): such as GPUs, TPUs, FPGAs and other dedicated hardware accelerators, are used to accelerate neural network inference tasks.

[0074] It is understandable that model inference acceleration in related technologies relies heavily on a specific xPU. If the accelerator card is missing or partially functional, the entire inference task cannot proceed smoothly or performance will be severely degraded. Therefore, a virtual accelerator card was introduced.

[0075] Step 208: Map each operator to its corresponding target operator, and complete the inference of the model to be processed based on the target operator.

[0076] For example, when the model file of the model to be processed is obtained, the kernel-state memory management module and the user-state vOplib are used to load and map the model file, parse the model file to be processed, and obtain all operators of the model to be processed. The operator library has intelligent self-adaptation capabilities, and the runtime system will detect the hardware environment in real time. When it is detected that there is no physical accelerator card, the model file is loaded into the memory space of the CPU, and the user-state dynamic routing vOplib will quickly switch to the operator library based on the SIMD instruction set, that is, the target operator that matches all operators is determined from the virtual operator library preset in the user state, and the CPU takes over the corresponding operator calculation task, and uses SIMD parallel computing optimization and other technologies based on the target operator to accelerate the calculation process. It can be understood that when performing model reasoning based on the operator, model input data related to the model reasoning task will be input, and the model input data can be image data.

[0077] Furthermore, the CPU has pre-integrated vector registers and SIMD instruction sets. When the model file is loaded into the CPU's memory space, the model and data will be optimized for parallel computing. The parallel computing optimization methods include but are not limited to: data layout optimization (such as allocating high-frequency access data to the CPU's L3 cache proximal memory, etc.), adaptive data blocking (i.e. dividing data blocks according to the number of SIMD channels), and instruction-level optimization (such as vectorization and instruction pipelining).

[0078] In the above embodiment, when there is a lack of dedicated hardware accelerators, the reasoning task can be completed on a general-purpose CPU by using a virtual operator library, avoiding the situation where reasoning cannot be performed due to lack of hardware, thereby improving the flexibility and compatibility of the system. In other words, regardless of whether there is a physical acceleration card, the model reasoning task can be successfully completed, thereby enhancing the adaptability and flexibility of the system. In the absence of dedicated hardware, the existing CPU resources can be fully utilized, resource waste can be avoided, and overall resource utilization can be improved. In addition, the dependence on expensive dedicated hardware is reduced, the hardware procurement and maintenance costs are reduced, and more application scenarios can achieve efficient model reasoning capabilities to meet diverse business needs.

[0079] It is understandable that before accelerating inference, the model needs to determine the capacity of the current memory pool and the memory capacity required to perform inference on the model file to be processed to ensure that there is enough memory to complete the inference task. In an exemplary embodiment, the model inference acceleration method further includes:

[0080] Obtain the memory capacity required for inference execution of the model file to be processed and the current memory pool capacity; if the current memory pool capacity is less than the memory capacity, trigger the dynamic expansion mechanism to determine the memory capacity in the current memory pool to be allocated to the model file to be processed.

[0081] The dynamic expansion mechanism involves the virtual card instance manager calling the instance creation API to trigger kernel-mode memory pool allocation. When insufficient initial memory is detected, on-demand expansion is triggered. The specific expansion method can be achieved by using a "pre-allocation + dynamic expansion" strategy for memory pool allocation, which is not detailed here.

[0082] Accordingly, the model file to be processed is parsed to obtain all operators of the model to be processed, including: converting the model file to be processed into an intermediate representation of the model, extracting a calculation graph according to the intermediate representation of the model; and obtaining all operators of the model to be processed according to the calculation graph.

[0083] For example, the ONNX / TensorFlow model is converted into a model intermediate representation (Unified IR), the computational graph is extracted, and all operators of the model to be processed are obtained.

[0084] In an exemplary embodiment, each operator is mapped to its corresponding target operator, and the inference of the model to be processed is accelerated based on the target operator, including:

[0085] Map each operator to its corresponding target operator, call the target operator to perform calculations based on the dependency between operators in the calculation graph, and obtain the calculation results; process the calculation results to obtain the inference results, and complete the inference of the model to be processed.

[0086] The target operator can be identified in the virtual operator library based on the operator type and properties obtained through analysis. Post-processing is performed based on the calculation results to obtain inference results. Post-processing can be understood as performing a series of operations on the model's original output to obtain results that better meet practical application requirements. Post-processing includes result analysis and visualization. For example, post-processing involves converting the probability values output by the model into specific category labels. In another example, in object detection tasks, redundant bounding boxes are removed from the output results to retain the most accurate detection results.

[0087] An inference task is the process of using a trained model to predict or classify new data. Inference tasks are performed based on the results of computations that have already been mapped and calculated. For example, for an image classification model, the inference task involves using the model to assign a category label to the input image data. The inference result refers to the final output obtained after the inference task is completed, such as the classification label or predicted value.

[0088] Furthermore, during the inference calculation process, a feedback control thread can be started to monitor the memory access mode of each layer in real time. During the mapping process, a dynamic memory allocation strategy can be adopted. Specifically, by monitoring the changes in computing requirements and data access frequency fluctuations of each layer of the model during the inference process in real time, the feedback control mechanism is used to dynamically adjust the CPU's memory allocation ratio.

[0089] For model inference acceleration, there may also be physical accelerator cards, but the physical accelerator cards do not support operators or only support some operators. In addition to the above inference acceleration cases, there are also the following two cases:

[0090] Case 1: If a physical accelerator card exists and supports all operators in the model information, the model file to be processed is loaded into the video memory space of the physical accelerator card. The user-mode application layer interface is called, and all operators are routed to the inference operator library of the physical accelerator card through the dynamic routing engine of the unified operator library. Each operator is mapped to its corresponding target operator in the inference operator library, and the inference of the model to be processed is completed based on the inference operator library.

[0091] For example, if there is a physical accelerator card and the physical accelerator card supports all operators, the model file to be processed is loaded into the video memory space of the physical accelerator card; the application layer interface of the user state is called, and all operators are routed to the inference operator library of the physical accelerator card through the dynamic routing engine, and each operator is mapped to the corresponding target operator in the inference operator library. The parallel computing advantage of the accelerator card is used for rapid processing, and the inference of the model to be processed is accelerated based on the inference operator library. It can be understood that when executing an inference task, the execution path will be selected based on the hardware support matrix and the real-time load status. For example, the data to be inferred and the input data such as the model structure and parameters are transferred to the xPU video memory through the DMA engine to start the asynchronous computing task. Monitor the xPU task queue and trigger load diversion when the queuing delay exceeds the threshold (such as 5ms).

[0092] Case 2: If a physical accelerator card exists and supports some operators, the model file to be processed is loaded into the video memory space of the physical accelerator card. The user-state application layer interface is called, and some operators are routed to the inference operator library of the physical accelerator card through the dynamic routing engine. Each operator is mapped to its corresponding target operator in the inference operator library. The remaining operators are loaded into the CPU memory space, and a virtual operator library that matches all the remaining operators is determined from the virtual accelerator card operator library preset in the user state. The remaining operators are mapped to the virtual operator library. Based on the inference operator library and the virtual operator library, the inference of the model to be processed is completed.

[0093] Furthermore, based on the inference operator library and virtual operator library, the inference of the model to be processed is accelerated, including:

[0094] The remaining operators are mapped to a virtual operator library. Based on the dependency relationships between the remaining operators, the corresponding operators in the virtual operator library are called to perform calculations to obtain a first calculation result. The first calculation result is sent to the physical accelerator card through a preset shared memory channel. Based on the dependency relationships between some operators, the corresponding operators in the inference operator library are called to perform calculations to obtain a second calculation result. The first calculation result and the second calculation result are integrated through the physical accelerator card to complete the inference of the model to be processed.

[0095] Understandably, this approach involves the physical accelerator card and CPU collaborating to perform inference tasks. The physical accelerator card initiates asynchronous computation by transferring input data to the xPU memory via the DMA engine. The CPU utilizes SIMD mode, activating the vector register management module, partitioning data by SIMD channel count, and using double buffering to overlap data loading and computation. This synergy effectively improves the efficiency and performance of model inference.

[0096] There may be acceleration anomalies during the model inference acceleration process. In order to ensure the accuracy of inference acceleration, optionally, in an exemplary embodiment, the model inference acceleration method also includes: when a physical acceleration card anomaly is detected or the target operator is missing, the operator supported by the physical acceleration card or the model operator corresponding to the target operator is loaded into the memory space of the CPU, and the step of determining the target operator that matches all operators in the model information from the virtual operator library preset in the user state is executed.

[0097] For example, when an xPU operator is missing or a hardware failure is detected, the task is migrated to the virtual operator library corresponding to the CPU. The target operator is determined by mapping the operator to the virtual operator library, and the model inference to be processed is completed based on the target operator. Furthermore, the hardware support matrix is updated and the abnormal event is recorded for reference in subsequent routing decisions. This approach avoids the situation where the entire inference task cannot be carried out smoothly or performance is severely degraded when the accelerator card is missing or partially functional.

[0098] To improve resource utilization, resources need to be reclaimed when inference ends. Optionally, in an exemplary embodiment, based on the kernel-mode memory management module and user-mode vOplib, a secure erase is performed on the memory pool associated with the virtual card instance to prevent data residue, release DMA mapping relationships, and reset the mapping table, thereby releasing memory resources. The vector register state of the SIMD operator is cleaned to prevent cross-task data contamination, and the compiled temporary code cache is reclaimed to free up space, thereby resetting the operator state. Optionally, a hardware-level reset is performed on the xPU to ensure there are no residual computing tasks, and the performance monitoring unit (PMU) counter is disabled to release interrupt resources, thereby resetting the hardware context.

[0099] In one embodiment, Figure 3 As shown, a model reasoning acceleration method is provided. This embodiment uses the method applied to a terminal as an example for illustration. In this embodiment, the method includes the following steps:

[0100] Step 302: Obtain the model file to be processed of the model to be processed.

[0101] Step 304: parse the model file to be processed to obtain all operators of the model to be processed.

[0102] Step 306 , determine whether a physical accelerator card exists. If not, execute step 308 ; if so, execute step 312 .

[0103] Step 308: If there is no physical acceleration card, the model file to be processed is loaded into the memory space of the CPU, and a target operator that matches all operators is determined from the virtual operator library preset in the user state.

[0104] Step 310: Map each operator to its corresponding target operator, and accelerate the inference of the model to be processed based on the target operator.

[0105] Step 312: If a physical acceleration card exists and the physical acceleration card supports all operators, the model file to be processed is loaded into the video memory space of the physical acceleration card.

[0106] Step 314: Call the user-mode application layer interface, route all operators to the inference operator library of the physical acceleration card through the dynamic routing engine, map each operator to its corresponding target operator in the inference operator library, and accelerate the inference of the model to be processed based on the inference operator library.

[0107] Step 316: If a physical accelerator card exists and supports some operators, the model file to be processed is loaded into the video memory space of the physical accelerator card, the remaining operators are loaded into the memory space of the CPU, a virtual operator library that matches all the remaining operators is determined from the virtual accelerator card operator library preset in the user state, and the remaining operators are mapped to the virtual operator library.

[0108] Step 318: Complete the inference of the model to be processed based on the inference operator library and the virtual operator library.

[0109] It should be noted that the specific implementation of this embodiment can be achieved through the above-mentioned limited methods, which will not be elaborated here.

[0110] Based on the above model reasoning acceleration method, a timing diagram of the model reasoning acceleration method based on the virtual accelerator card and xPU is provided, such as Figure 4 As shown, the system includes an application, a virtual accelerator card API, a scheduling engine, a physical accelerator card, and a CPU (central processing unit) computing unit. The application initiates an inference request, obtains the model file of the model to be processed through the virtual accelerator card API, parses the model file to obtain a computation graph, and performs operator hierarchical analysis based on the computation graph to allocate computing tasks. Operator hierarchical analysis includes three scenarios: L1 (full match with the accelerator card's native instructions), L2 (requiring CPU-assisted computation), and L3 (complete reliance on the CPU). For the L1 level, if a physical accelerator card is available and supports all operators, the model file is loaded into the card's video memory for inference acceleration, and the intermediate results are returned to the scheduling engine. For the L2 level, if a physical accelerator card is available and supports some operators, the scheduling engine allocates computing tasks, assigning the supported operators to the physical accelerator card to obtain block result A. The remaining operators are loaded into the CPU, and SIMD is used to calculate the inference tasks for the remaining operators to obtain block result B. Integrate block results A and B, return the intermediate results to the scheduling engine, and aggregate the final results through the scheduling engine and return them to the application.

[0111] For L3 level, if there is no physical accelerator card, the entire model file can be loaded into the CPU memory space, and model reasoning can be achieved through full SIMD calculation.

[0112] In the above-mentioned model reasoning acceleration method, for the case where there is no physical acceleration card, the model file is directly loaded into the CPU, and the previously integrated vector registers and SIMD instruction set are used to optimize the parallel processing of the model data to achieve model reasoning acceleration, avoiding high dependence on a specific xPU. When the acceleration card is missing or some functions are not sound, the entire reasoning task cannot be carried out smoothly or the performance is seriously degraded; for the case where a physical acceleration card exists and only supports some operators, the operators of the model file are split into the physical acceleration card and the CPU, and the physical acceleration card and the CPU are used for collaborative reasoning, which solves the incompatibility of the operator libraries of different acceleration cards. When encountering operators that are not supported by the acceleration card, there is a lack of effective alternatives, which increases development complexity and cost. In addition, it solves the problem of the single model loading and memory mapping method in the related technology, which is not optimized for the characteristics of neural networks, has poor memory access performance, and affects the reasoning speed.

[0113] In an exemplary embodiment, an application based on the above-mentioned model reasoning acceleration method is provided, such as Figure 5 As shown in the figure, an application flow chart of the model inference acceleration method is provided, which specifically includes:

[0114] In response to an inference task request, the model file of the model to be processed is obtained, the model file is read, the memory capacity required for inference execution of the model file to be processed and the current memory pool capacity are obtained, and a memory judgment is performed, that is, whether the current memory pool capacity meets the memory capacity. If it is insufficient and there is currently a model inference task occupying memory that can be released, the memory occupied by the model is released. If the current memory pool capacity is greater than the corresponding storage capacity required for inference execution of the model file, the model file is loaded and a load success judgment is performed. If the load is successful, the model file to be processed is converted into a model intermediate representation, and a computation graph is extracted based on the model intermediate representation; based on the computation graph, all operators of the model to be processed are obtained.

[0115] Operator hierarchical processing is performed based on the operator. The hardware environment is detected in real time through the virtual accelerator card, and the input image data is obtained. The image data is preprocessed, including image data processing such as flipping and splicing. The processed data is input into the model for inference calculation. If a physical accelerator card exists and supports all operators, the model inference is completed based on the accelerator card through memory allocation. If a physical accelerator card exists and only supports some operators, a hybrid computing mode is adopted to split the constructed calculation graph, that is, split the calculation path, and complete the model inference based on the collaboration of the CPU and xPU. If there is no physical accelerator card, a pure CPU mode is adopted to load the model file into the CPU memory. Through CPU memory optimization, the model inference is completed based on the virtual accelerator card. The inference result is aggregated according to the inference calculation result and output. For example, the inference result can be an image recognition result or an image classification result.

[0116] In the above method, by detecting the physical accelerator card and its support status, it is ensured that subsequent reasoning tasks can be executed in a suitable hardware environment, avoiding task failures caused by hardware not supporting certain operators; by loading the model file into the video memory space of the physical accelerator card, the delay in data transmission is reduced, the data access speed during the reasoning process is improved, and CPU resources are released at the same time; by calling the user-mode application layer interface, the interaction with the physical accelerator card is simplified, reducing the difficulty of development; by utilizing the high-performance operator library of the physical accelerator card, the reasoning speed and efficiency are significantly improved, and the integrity and accuracy of the reasoning results are ensured. Overall, by dynamically coordinating the CPU and xPU, resources are reasonably allocated according to task requirements, avoiding idle or overloaded resources, improving overall resource utilization, and effectively solving the problem of reasoning task interruption caused by missing accelerator cards or imperfect functions. Whether in low-cost embedded devices or complex server environments, reasoning tasks can be stably run, broadening the range of application scenarios.

[0117] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0118] Based on the same inventive concept, the embodiments of the present application also provide a model reasoning acceleration system for implementing the aforementioned model reasoning acceleration method. The implementation solution provided by this system is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations in one or more model reasoning acceleration system embodiments provided below can be found in the limitations of the model reasoning acceleration method above and will not be repeated here.

[0119] In an exemplary embodiment, Figure 6 As shown, a model reasoning acceleration system is provided. The system includes a physical hardware layer, a virtual acceleration card, and an application layer. The physical hardware layer includes a CPU, wherein:

[0120] The application layer is used to initiate an inference request and determine the model file of the model to be processed carried by the inference request;

[0121] The virtual accelerator card is used to obtain the model file to be processed; parse the model file to be processed to obtain the model information of the model to be processed; if there is no physical accelerator card, the model file to be processed is loaded into the CPU's memory space, and the target operator that matches all operators in the model information is determined from the virtual operator library preset in the user state; each operator is mapped to its corresponding target operator, and the inference of the model to be processed is completed based on the target operator.

[0122] When reasoning on a model, the above-mentioned model inference acceleration system parses the model file of the model to be processed to obtain all the operators of the model. In the absence of a physical acceleration card, the model file is loaded into the memory space of the CPU, and the target operator that matches all operators is determined from the virtual operator library preset in the user state; each operator is mapped to its corresponding target operator, and the inference of the model to be processed is accelerated based on the target operator, avoiding the current situation where hardware resource configuration is limited and inference acceleration cannot be achieved without an acceleration card. It fully taps the potential of the CPU and improves the resource utilization of the CPU. In addition, it meets diverse business needs.

[0123] Optionally, in an exemplary embodiment, the virtual accelerator card is also used to obtain the memory capacity required for executing reasoning on the model file to be processed and the current memory pool capacity; if the current memory pool capacity is less than the memory capacity, the dynamic expansion mechanism is triggered to determine the memory capacity in the current memory pool to be allocated to the model file to be processed.

[0124] Convert the model file to be processed into an intermediate representation of the model, and extract the computational graph based on the intermediate representation of the model;

[0125] According to the computational graph, all operators of the model to be processed are obtained.

[0126] Optionally, in an exemplary embodiment, the virtual accelerator card is also used to map each operator to its corresponding target operator, call the target operator to perform calculations based on the dependency relationship between the operators in the calculation graph, and obtain the calculation results; and obtain the reasoning results based on the calculation results and the reasoning task.

[0127] Optionally, in an exemplary embodiment, the virtual accelerator card is further configured to load the model file to be processed into a video memory space of a physical accelerator card if a physical accelerator card exists and the physical accelerator card supports all operators in the model information;

[0128] Call the user-state application layer interface, route all operators to the inference operator library of the physical accelerator card through the dynamic routing engine of the unified operator library, map each operator to its corresponding target operator in the inference operator library, and complete the inference of the model to be processed based on the inference operator library.

[0129] Optionally, in an exemplary embodiment, the virtual accelerator card is also used to load the model file to be processed into the video memory space of the physical accelerator card if a physical accelerator card exists and the physical accelerator card supports some operators; call the application layer interface in user state, route some operators to the inference operator library of the physical accelerator card through the dynamic routing engine of the unified operator library, and map each operator in the partial operators to the corresponding target operator in the inference operator library; load the remaining operators into the memory space of the CPU, determine the virtual operator library that matches all the remaining operators from the virtual accelerator card operator library preset in user state, and map the remaining operators to the virtual operator library; and complete the inference of the model to be processed based on the inference operator library and the virtual operator library.

[0130] Optionally, in an exemplary embodiment, the virtual accelerator card is also used to map the remaining operators to the virtual operator library, and based on the dependency relationship between the remaining operators, call the corresponding operators in the virtual operator library to perform calculations to obtain a first calculation result; send the first calculation result to the physical accelerator card through a preset shared memory channel; map some operators to the inference operator library, and based on the dependency relationship between some operators, call the corresponding operators in the inference operator library to perform calculations to obtain a second calculation result; integrate the first calculation result and the second calculation result through the physical accelerator card to complete the inference of the model to be processed.

[0131] Optionally, in an exemplary embodiment, the virtual accelerator card is also used to load the operators supported by the physical accelerator card or the model operators corresponding to the target operators into the memory space of the CPU when an abnormality of the physical accelerator card or a missing target operator is detected.

[0132] Each module in the aforementioned model inference acceleration system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0133] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a model inference acceleration method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0134] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0135] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0136] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0137] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0138] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0139] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic based on quantum computing, artificial intelligence (AI) processors, and the like.

[0140] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0141] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A model reasoning acceleration method, characterized in that: The method comprises: Obtain the model file to be processed of the model to be processed; Parsing the model file to be processed to obtain model information of the model to be processed; If no physical accelerator card exists, the model file to be processed is loaded into the memory space of the CPU, and the target operator that matches all operators in the model information is determined from the virtual operator library preset in the user state of the virtual accelerator card. The virtual accelerator card acts as a software-defined middle layer to abstract and schedule the physical hardware. The virtual accelerator card consists of two parts: kernel state and user state. The kernel state implements hardware resource isolation and dynamic memory management, while the user state provides a unified operator library and SIMD optimization. Mapping each of the operators to a corresponding target operator, and completing the inference of the model to be processed based on the target operator; If there is a physical acceleration card and the physical acceleration card supports some operators in the model information, then the model file to be processed is loaded into the video memory space of the physical acceleration card; Calling the user-state application layer interface, routing some of the operators to the inference operator library of the physical accelerator card through the dynamic routing engine of the unified operator library, and mapping each of the operators in the part of the operators to the corresponding target operator in the inference operator library; Load the remaining operators into the memory space of the CPU, determine a virtual operator library that matches all the remaining operators from the virtual accelerator card operator library preset in the user state, and map the remaining operators to the virtual operator library; Based on the inference operator library and the virtual operator library, the inference of the model to be processed is completed.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining the memory capacity required for executing reasoning on the model file to be processed and the current memory pool capacity; If the current memory pool capacity is less than the memory capacity, a dynamic expansion mechanism is triggered to determine the memory capacity allocated to the model file to be processed in the current memory pool; The parsing of the model file to be processed to obtain model information of the model to be processed includes: Converting the model file to be processed into a model intermediate representation, and determining a computational graph of the model file to be processed; According to the calculation graph, all operators of the model to be processed are obtained.

3. The method according to claim 2, characterized in that Mapping each of the operators to a corresponding target operator and completing the inference of the model to be processed based on the target operator includes: Mapping each of the operators to its corresponding target operator, and calling the target operator to perform calculations based on the dependency relationship between the operators in the calculation graph to obtain calculation results; Processing is performed according to the calculation results to obtain inference results, and the inference of the model to be processed is completed.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: If a physical accelerator card exists and the physical accelerator card supports all operators in the model information, then the model file to be processed is loaded into the video memory space of the physical accelerator card; Call the user-state application layer interface, route all the operators to the inference operator library of the physical acceleration card through the dynamic routing engine of the unified operator library, map each operator to its corresponding target operator in the inference operator library, and complete the inference of the model to be processed based on the inference operator library.

5. The method according to claim 1, wherein Accelerating the inference of the model to be processed based on the inference operator library and the virtual operator library includes: Mapping the remaining operators to the virtual operator library, and calling corresponding operators in the virtual operator library to perform calculations based on the dependency relationships between the remaining operators to obtain a first calculation result; Sending the first calculation result to the physical accelerator card through a preset shared memory channel; Mapping some of the operators to the inference operator library, and calling corresponding operators in the inference operator library to perform calculations based on dependencies between some of the operators to obtain a second calculation result; The first calculation result and the second calculation result are integrated through the physical acceleration card to complete the reasoning of the model to be processed.

6. The method according to claim 1, characterized in that The method further comprises: When it is detected that the physical acceleration card is abnormal or the target operator is missing, the operator supported by the physical acceleration card or the operator corresponding to the target operator is loaded into the memory space of the CPU, and the step of determining the target operator that matches all operators in the model information from the virtual operator library preset in the user state is performed.

7. A model reasoning acceleration system, characterized in that: The system includes a physical hardware layer, a virtual accelerator card, and an application layer, wherein the physical hardware layer includes a CPU, wherein: The application layer is used to initiate an inference request and determine a to-be-processed model file of the to-be-processed model carried in the inference request; The virtual accelerator card is used to obtain the model file to be processed; parse the model file to be processed to obtain the model information of the model to be processed; if there is no physical accelerator card, the model file to be processed is loaded into the memory space of the CPU, and the target operator that matches all operators in the model information is determined from the virtual operator library preset in the user state of the virtual accelerator card; each of the operators is mapped to its corresponding target operator, and the inference of the model to be processed is completed based on the target operator; the virtual accelerator card serves as a software-defined middle layer to abstract and schedule physical hardware. The virtual accelerator card consists of two parts, kernel state and user state. Hardware resource isolation and dynamic memory management are realized through kernel state, and a unified operator library and SIMD optimization are provided in user state. The virtual accelerator card is further used to load the model file to be processed into the video memory space of the physical accelerator card if a physical accelerator card exists and the physical accelerator card supports some of the operators in the model information; call the application layer interface in the user state, route some of the operators to the inference operator library of the physical accelerator card through the dynamic routing engine of the unified operator library, and map each of the operators in the part of the operators to the corresponding target operator in the inference operator library; load the remaining operators into the memory space of the CPU, determine the virtual operator library that matches all the remaining operators from the virtual accelerator card operator library preset in the user state, and map the remaining operators to the virtual operator library; and complete the inference of the model to be processed based on the inference operator library and the virtual operator library.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images