Rapid deployment method, server and terminal equipment

Abstract

The invention provides a rapid deployment method, a server and terminal equipment. The method is applied to a server, and comprises the following steps: receiving a deployment request sent by terminal equipment; based on hardware attribute information of the terminal equipment included in the deployment request, querying a target computing kernel corresponding to the hardware attribute information; the target computing kernel at least comprises a computing instruction which can be directly operated on the WebGPU network graphic processor; the calculation instruction is matched with the hardware attribute information, and the calculation instruction is an instruction written through a WebGPU shader language; and sending the queried target computing kernel to the terminal equipment, so that the terminal equipment runs the target computing kernel through the WebGPU.

Description

Technical Field

[0001] The present application relates to computer technology, and in particular to a rapid deployment method, a server and a terminal device. Background Art

[0002] At present, in the related technology, the general kernel is deployed on the terminal device to perform computing tasks. The compilation process of the general kernel has unnecessary compilation optimization steps, which leads to long compilation time and high compilation complexity, and cannot meet the needs of rapid deployment. Alternatively, the computing kernel is pre-compiled during the deployment stage, but the pre-compilation method often fails to optimize the compilation for the hardware, resulting in low computing performance of the computing kernel. Summary of the invention

[0003] The embodiments of the present application provide a rapid deployment method, a server, and a terminal device.

[0004] According to a first aspect of the present application, a rapid deployment method is provided, which is applied to a server and includes:

[0005] Receiving a deployment request sent by a terminal device;

[0006] Based on the hardware attribute information of the terminal device included in the deployment request, query the target computing kernel corresponding to the hardware attribute information; the target computing kernel at least includes computing instructions that can be directly run on the WebGPU network graphics processor; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language;

[0007] The queried target computing kernel is sent to the terminal device, so that the terminal device runs the target computing kernel through the WebGPU.

[0008] According to an embodiment of the present application, querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request includes:

[0009] Based on the hardware attribute information, determining the parallel processing capability of the hardware computing resources of the terminal device;

[0010] Determining a target hardware utilization of a computing kernel on the hardware computing resources based on the parallel processing capability;

[0011] Based on the target hardware utilization, a target computing kernel corresponding to the hardware attribute information is determined.

[0012] According to an embodiment of the present application, querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request includes:

[0013] Based on the hardware attribute information, determining a plurality of candidate computing kernels corresponding to the hardware attribute information;

[0014] Based on the matching degree between the hardware attribute information and the candidate computing kernels, the target computing kernel is determined from a plurality of candidate computing kernels; the matching degree is determined according to the historical computing performance corresponding to the candidate computing kernels.

[0015] According to an embodiment of the present application, querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request includes:

[0016] Performing feature analysis on the hardware attribute information to obtain a performance weight; the performance weight represents the degree of influence of the hardware attribute information on the computing performance of the WebGPU;

[0017] Based on the performance weight and the hardware attribute information, a target computing kernel corresponding to the hardware attribute information is determined.

[0018] According to an embodiment of the present application, querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request includes:

[0019] Determine, based on the parallel processing capability and the performance weight corresponding to the hardware attribute information, a comprehensive hardware utilization rate of the computing kernel on the hardware computing resources; the comprehensive hardware utilization rate represents the utilization efficiency of the hardware computing resources when the computing kernel runs on the WebGPU;

[0020] Based on the comprehensive hardware utilization, a target computing kernel corresponding to the hardware attribute information is determined.

[0021] According to an embodiment of the present application, before receiving the deployment request sent by the terminal device, the method further includes:

[0022] Sending the plurality of computing kernels to be tested to the terminal device in sequence through a transmission queue;

[0023] Receiving a performance evaluation result sent by the terminal device;

[0024] Based on the performance evaluation results corresponding to the candidate computing kernels, a target computing kernel corresponding to the hardware attribute information is determined from the multiple computing kernels to be tested.

[0025] According to an embodiment of the present application, after querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request, the method further includes:

[0026] In response to the absence of a target computing kernel corresponding to the hardware attribute information, determining a parameter search space corresponding to the hardware attribute information based on the hardware attribute information;

[0027] Based on the parameter search space, determining a plurality of target computing parameters related to the hardware attribute information;

[0028] Based on the multiple target computing parameters, a target computing kernel corresponding to the hardware attribute information is generated.

[0029] According to a second aspect of the present application, a rapid deployment method is provided, which is applied to a terminal device and includes:

[0030] Sending a deployment request to a server; the deployment request includes hardware attribute information of the terminal device;

[0031] Receiving a target computing kernel sent by the server;

[0032] The target computing kernel is run through a WebGPU network graphics processor; the target computing kernel at least includes computing instructions that can be directly run on the WebGPU; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in a WebGPU shader language.

[0033] According to an implementation manner of the present application, before sending the deployment request to the server, the method further includes:

[0034] Receiving multiple computing kernels to be tested sent by the server;

[0035] Performing a performance evaluation test on each computing kernel to be tested by the WebGPU to obtain a corresponding performance evaluation result; the performance evaluation result represents the computing performance of the computing kernel to be tested running on the WebGPU;

[0036] The performance evaluation result is sent to the server.

[0037] According to an embodiment of the present application, the step of running the target computing kernel through a WebGPU network graphics processor includes:

[0038] Based on the target computing kernel and the hardware attribute information, the hardware computing resources of the terminal device are configured to obtain the configured hardware computing resources;

[0039] Scheduling computing tasks to the configured hardware computing resources through the WebGPU;

[0040] The computing task is executed based on the computing instructions included in the target computing kernel.

[0041] According to a third aspect of the present application, a server is provided, including:

[0042] A first receiving module, used to receive a deployment request sent by a terminal device;

[0043] A query module, configured to query a target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request; the target computing kernel at least includes a computing instruction that can be directly run on a WebGPU network graphics processor; the computing instruction matches the hardware attribute information, and the computing instruction is an instruction written in a WebGPU shader language;

[0044] The first sending module is used to send the queried target computing kernel to the terminal device, so that the terminal device runs the target computing kernel through the WebGPU.

[0045] According to a fourth aspect of the present application, a terminal device is provided, including:

[0046] A second sending module is used to send a deployment request to the server; the deployment request includes the hardware attribute information of the terminal device;

[0047] A second receiving module, used for receiving the target computing kernel sent by the server;

[0048] A deployment module is used to run the target computing kernel through a WebGPU network graphics processor; the target computing kernel at least includes computing instructions that can be directly run on the WebGPU; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in a WebGPU shader language. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0050] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0051] Figure 1 The following is a schematic diagram showing the process flow of the rapid deployment method provided in the embodiment of the present application. Figure 1 ;

[0052] Figure 2 The following is a schematic diagram showing the process flow of the rapid deployment method provided in the embodiment of the present application. Figure 2 ;

[0053] Figure 3 The following is a schematic diagram showing the process flow of the rapid deployment method provided in the embodiment of the present application. Figure 3 ;

[0054] Figure 4 The following is a schematic diagram showing the process flow of the rapid deployment method provided in the embodiment of the present application. Figure 4 ;

[0055] Figure 5 The following is a schematic diagram showing the process flow of the rapid deployment method provided in the embodiment of the present application. Figure 5 ;

[0056] Figure 6 A detailed processing flow diagram of the rapid deployment method provided by an embodiment of the present application is shown;

[0057] Figure 7 Another detailed processing flow diagram of the rapid deployment method provided in an embodiment of the present application is shown;

[0058] Figure 8 A schematic diagram of an application scenario of the rapid deployment method provided in an embodiment of the present application is shown;

[0059] Fig. 9 Another application scenario schematic diagram of the rapid deployment method provided in the embodiment of the present application is shown;

[0060] Fig.10 An optional structural diagram of a server provided in an embodiment of the present application is shown;

[0061] Fig.11 An optional structural diagram of a terminal device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0062] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0063] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0064] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] The processing flow of the rapid deployment method provided in the embodiment of the present application is described. Figure 1 , Figure 1 This is a schematic diagram of the processing flow of the rapid deployment method provided in the embodiment of the present application. Figure 1 , will combine Figure 1 Steps S101-S103 are shown for explanation.

[0067] Step S101: receiving a deployment request sent by a terminal device.

[0068] In some embodiments, the deployment request may include hardware attribute information of the terminal device. The hardware attribute information may include attribute information such as hardware ID and hardware configuration information. The embodiments of the present application do not limit the specific hardware attribute information. The hardware ID may include hardware device IDs such as CPU model and GPU model. The embodiments of the present application do not limit the specific hardware ID.

[0069] Step S102, based on the hardware attribute information of the terminal device included in the deployment request, query the target computing kernel corresponding to the hardware attribute information; the target computing kernel at least includes computing instructions that can be directly run on the WebGPU network graphics processor; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language.

[0070] In some embodiments, the server may query the optimization kernel library based on the hardware attribute information to obtain the target computing kernel corresponding to the attribute information. The optimization kernel library may include computing kernels corresponding to various hardware attribute information after hardware optimization. The target computing kernel may include computing instructions that can be run directly on WebGPU (network graphics processor); the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language. WebGPU can be used to perform graphics rendering and general computing using a graphics processor in a device browser. The WebGPU shader language may include a programming language for writing WebGPU computing instructions. Specific WebGPU shader languages ​​may include: WSGI (WebGPU Shading Language). The target computing kernel may include: WSGI computing kernel. The computing instructions may include optimized computing instructions that adapt to the hardware attribute information of the terminal device.

[0071] Step S103: Send the queried target computing kernel to the terminal device, so that the terminal device runs the target computing kernel through WebGPU.

[0072] In some embodiments, the target computing kernel can be directly run on the WebGPU of the terminal device. After sending the queried target computing kernel to the terminal device, the terminal device can register the target computing kernel on the terminal device, and the terminal device runs the target computing kernel through the WebGPU to perform the computing task after registration. The embodiments of the present application do not limit the specific computing tasks.

[0073] In the method of the embodiment of the present application, the server selects the most suitable computing kernel to be deployed on the terminal device according to the hardware attribute information of the terminal device. The computing kernel based on hardware characteristics improves the execution efficiency of computing tasks on WebGPU and ensures the full utilization of the hardware computing resources of the terminal device. The computing kernel is compiled by the WebGPU shader language and can be executed directly on WebGPU without the intermediate representation conversion step in the traditional compilation chain, and directly obtains the optimized computing kernel, which improves the deployment efficiency and can meet the needs of rapid deployment.

[0074] In some embodiments, the processing flow of the rapid deployment method is shown as follows: Figure 2 ,like Figure 2 As shown, in step S102, based on the hardware attribute information of the terminal device included in the deployment request, querying the target computing kernel corresponding to the hardware attribute information includes:

[0075] Step S201, based on the hardware attribute information, determine the parallel processing capability of the hardware computing resources of the terminal device.

[0076] Step S202: determining a target hardware utilization of the computing kernel on the hardware computing resources based on the parallel processing capability.

[0077] Step S203: determining a target computing kernel corresponding to the hardware attribute information based on the target hardware utilization.

[0078] In this embodiment, the hardware computing resources may include: CPU (Central Processing Unit), memory and GPU (Graphics Processing Unit). The hardware computing resources may also include other computing resources of the terminal device, which is not limited in the embodiment of the present application. Parallel processing capability may include: the ability of the hardware computing resources to process multiple computing tasks simultaneously. The parallel processing capability is related to the number of cores, computing units and concurrent threads supported by the hardware computing resources. The target hardware utilization may include: the hardware resource utilization rate expected to be achieved when running the computing kernel on the hardware computing resources. For example, if a GPU has multiple cores, the target hardware utilization may be 80% of the use of these cores. Based on the target hardware utilization, a target computing kernel that can meet the target hardware utilization is screened out from multiple computing kernels.

[0079] As an example, the attribute information of the GPU of the terminal device is determined through the hardware attribute information of the terminal device, and the number of computing units, memory bandwidth and core frequency included in the GPU are determined based on the attribute information of the GPU. The number of computing units, memory bandwidth and core frequency included in the GPU are analyzed to determine the parallel processing capability of the GPU of the terminal device. The GPU memory access mode is determined based on the parallel processing capability, and the target hardware utilization of the GPU is determined based on the GPU memory access mode. According to the target hardware utilization of the GPU, a target computing kernel that meets the target hardware utilization is determined from the computing kernels corresponding to the multiple hardware attribute information. The target computing kernel can maximize the parallel processing capability of the GPU to perform computing tasks.

[0080] In some embodiments, in step S102, based on the hardware attribute information of the terminal device included in the deployment request, querying the target computing kernel corresponding to the hardware attribute information may also include: determining the comprehensive hardware utilization of the computing kernel on the hardware computing resources based on the parallel processing capability and the performance weight corresponding to the hardware attribute information; and determining the target computing kernel corresponding to the hardware attribute information based on the comprehensive hardware utilization.

[0081] In some embodiments, the comprehensive hardware utilization may represent the efficiency of utilizing hardware computing resources when the computing kernel runs on the WebGPU.

[0082] As an example, the number of cores, memory bandwidth, and video memory size of the GPU are analyzed to determine the parallel processing capability of the GPU of the terminal device, and a performance weight is determined for each attribute information. For example, the number of cores can account for 50% of the weight, the memory bandwidth can account for 30%, and the video memory size can account for 20%. Based on the parallel processing capability and performance weight of the GPU, the comprehensive hardware utilization of the GPU is analyzed and determined, and based on the comprehensive hardware utilization of the GPU, a target computing kernel that meets the comprehensive hardware utilization is determined from the computing kernels corresponding to multiple hardware attribute information. The target computing kernel can maximize the use of the parallel processing capability of the GPU to perform computing tasks.

[0083] The method of the embodiment of the present application determines the corresponding computing kernel based on the parallel processing capability of the hardware and the target hardware utilization, and the server can adjust and select the computing kernel more finely to adapt to terminal devices with different hardware configurations. The hardware computing resources on the terminal device can be fully utilized for calculation, the execution efficiency of the computing tasks on the WebGPU is improved, the flexibility and adaptability of the deployment process are enhanced, and the needs of rapid deployment can be met.

[0084] In some embodiments, the processing flow of the rapid deployment method is shown as follows: Figure 3 ,like Figure 3 As shown, in step S102, based on the hardware attribute information of the terminal device included in the deployment request, querying the target computing kernel corresponding to the hardware attribute information includes:

[0085] Step S301: Based on the hardware attribute information, a plurality of candidate computing kernels corresponding to the hardware attribute information are determined.

[0086] Step S302 : determining a target computing kernel from a plurality of candidate computing kernels based on the matching degree between the hardware attribute information and the candidate computing kernels.

[0087] In this embodiment, the hardware computing resources corresponding to the hardware attribute information may support computing capabilities of different architectures. The computing capabilities of different architectures correspond to different computing kernels, and the multiple candidate computing kernels may include multiple different versions of candidate computing kernels optimized for the same hardware attribute information. The matching degree may include: compatibility and efficiency between the candidate computing kernel and the hardware attribute information. The matching degree may be determined based on the historical computing performance corresponding to the candidate computing kernel.

[0088] As an example, the terminal device includes a GPU of brand A. The server determines multiple different versions of computing kernels based on the hardware attribute information of the GPU of brand A. The multiple different versions of computing kernels can support the computing capabilities of different architectures. The historical performance of the candidate computing kernels is as follows: for example, the execution time of computing kernel 1 is 20ms and the resource utilization is 80%, the execution time of computing kernel 2 is 15ms and the resource utilization is 75%, and the execution time of computing kernel 3 is 18ms and the resource utilization is 85%. Based on the historical performance of the candidate computing kernels, it is determined that the hardware attribute information of the GPU of brand A has the highest match with computing kernel 3. Determine computing kernel 3 as the target computing kernel.

[0089] The method of the embodiment of the present application can select the most matching computing kernel from multiple candidate computing kernels by evaluating the matching degree between multiple candidate computing kernels and hardware attribute information. It can ensure that the computing kernel is not only compatible with the hardware computing resources of the terminal device, but also can provide the best computing performance, and can make full use of the hardware computing resources on the terminal device for computing, thereby improving the execution efficiency of computing tasks on WebGPU, enhancing the flexibility and adaptability of the deployment process, and meeting the needs of rapid deployment.

[0090] In some embodiments, the processing flow of the rapid deployment method is shown as follows: Figure 4 ,like Figure 4 As shown, in step S102, based on the hardware attribute information of the terminal device included in the deployment request, querying the target computing kernel corresponding to the hardware attribute information includes:

[0091] Step S401, performing feature analysis on hardware attribute information to obtain performance weights.

[0092] Step S402: determining a target computing kernel corresponding to the hardware attribute information based on the performance weight and the hardware attribute information.

[0093] In this embodiment, the performance weight may represent the degree of influence of the hardware attribute information on the computing performance of the WebGPU.

[0094] As an example, based on the hardware attribute information of the GPU of the terminal device, including model, number of cores and memory bandwidth. Perform feature analysis on these hardware attribute information, and determine a corresponding performance weight for each attribute information, and the performance weight represents the degree of influence of the attribute information on the computing performance of WebGPU. According to the performance weight and hardware attribute information, determine the target computing kernel from multiple candidate computing kernels. For example, if the GPU of the terminal device includes a large number of cores and a high memory bandwidth, the performance weight corresponding to the number of cores and memory bandwidth is higher. Based on the performance weight and hardware attribute information, determine the target computing kernel, which can be a computing kernel optimized for multi-core and can make full use of memory bandwidth.

[0095] The method of the embodiment of the present application, through feature analysis and determination of performance weights, enables the server to more accurately evaluate the impact of hardware attribute information on WebGPU computing performance. It is able to determine the computing kernel that can best improve computing performance, achieve more balanced resource allocation on different hardware computing resources, and fully utilize the hardware computing resources on the terminal device for computing, thereby improving the execution efficiency of computing tasks on WebGPU, enhancing the flexibility and adaptability of the deployment process, and meeting the needs of rapid deployment.

[0096] In some embodiments, the processing flow of the rapid deployment method is shown as follows: Figure 5 ,like Figure 5 As shown, including:

[0097] Step S501, sending a deployment request to a server; the deployment request includes hardware attribute information of a terminal device.

[0098] Step S502: receiving a target computing kernel sent by a server.

[0099] Step S503, running the target computing kernel through the WebGPU network graphics processor; the target computing kernel at least includes computing instructions that can be run directly on the WebGPU; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language.

[0100] In this embodiment, the terminal device can obtain the hardware attribute information corresponding to the hardware computing resource, and then generate a deployment request based on the hardware attribute information. After receiving the target computing kernel sent by the server, the terminal device can register the target computing kernel on the terminal device, and after registration, the target computing kernel can be run through the WebGPU network graphics processor.

[0101] In some embodiments, step S503 may include: configuring the hardware computing resources of the terminal device based on the target computing kernel and the hardware attribute information to obtain the configured hardware computing resources. Scheduling the computing task to the configured hardware computing resources through WebGPU. Executing the computing task based on the computing instructions included in the target computing kernel.

[0102] As an example, according to the requirements of the target computing kernel and the hardware attribute information, the hardware computing resources are configured, such as setting the size of the work group to 256 to match the number of cores of the GPU of the terminal device. Create a computing pipeline, which is used to define the execution environment of the image processing task. Scheduling schedules the computing task to the configured hardware computing resources through WebGPU, and the computing instructions in the target computing kernel begin to be executed in parallel on the GPU to process the data in the computing task.

[0103] In the method of the embodiment of the present application, the terminal device can send a deployment request containing hardware attribute information to the server, and receive the target computing kernel returned by the server. The most suitable computing kernel deployed on the terminal device. The computing kernel based on hardware characteristics improves the execution efficiency of computing tasks on WebGPU, ensuring full utilization of the hardware computing resources of the terminal device. The computing kernel is compiled by the WebGPU shader language and can be executed directly on WebGPU, without the need for the intermediate representation conversion step in the traditional compilation chain, and directly obtains the optimized computing kernel, which improves the deployment efficiency and can meet the needs of rapid deployment. At the same time, in application scenarios that require real-time response, such as augmented reality (AR), virtual reality (VR), real-time games and intelligent transportation systems, the target computing kernel deployed by the terminal device is used for reasoning calculation, which can significantly reduce network delays, achieve fast response and processing, thereby improving user experience and system performance. In an environment where the network cannot be stably connected, such as remote field operations, navigation, aviation and other scenarios, the terminal device can run the deployed target computing kernel for reasoning calculation in an offline state, and the terminal device can still perform complex reasoning tasks in an offline state, bringing a wider range of application scenarios and stronger computing power to Web development.

[0104] Figure 6 This is a detailed optional processing flow of the rapid deployment method provided in the embodiment of the present application, such as Figure 6 As shown, including:

[0105] Step S601: the server sends multiple computing kernels to be tested to the terminal device in sequence through a transmission queue.

[0106] In this embodiment, the server may send multiple computing cores to be tested to the terminal device in sequence in the form of a transmission queue, and the terminal device may perform performance evaluation on each computing core to be tested in the transmission queue in sequence.

[0107] Step S602: The terminal device receives multiple computing kernels to be tested sent by the server.

[0108] Step S603: The terminal device performs a performance evaluation test on each computing kernel to be tested through WebGPU to obtain a corresponding performance evaluation result.

[0109] In this embodiment, the terminal device performs a performance evaluation test of the computing kernel to be tested, measures the computing efficiency and performance of each computing kernel through WebGPU, and obtains a corresponding performance evaluation result. The performance evaluation test may include: verifying the computing efficiency and performance of the computing kernel under various conditions by simulating actual application scenarios and load conditions.

[0110] Step S604: The terminal device sends the performance evaluation result to the server.

[0111] Step S605: the server receives the performance evaluation result sent by the terminal device.

[0112] Step S606: The server determines a target computing kernel corresponding to the hardware attribute information from the plurality of computing kernels to be tested based on the performance evaluation results corresponding to the candidate computing kernels.

[0113] In this embodiment, the server determines the target computing kernel that best suits the hardware attribute information based on the performance evaluation result. The target computing kernel can maximize the computing performance of the hardware computing resources and can provide the best computing performance on the terminal device. The server can store the target computing kernel based on the association between the target computing kernel and the hardware attribute information. Based on the hardware attribute information, the target computing kernel corresponding to the hardware attribute information can be queried.

[0114] In the method of the embodiment of the present application, before sending a deployment request to the server, the terminal device can receive and test multiple computing kernels to be tested, and then the server can determine the best computing kernel based on the performance evaluation results. The accuracy of computing kernel selection is enhanced, and the server is assisted in optimizing and adjusting the deployment strategy of the computing kernel through the feedback of actual performance data, which can meet the needs of rapid deployment.

[0115] Figure 7 This is another detailed optional processing flow of the rapid deployment method provided in the embodiment of the present application, such as Figure 7 As shown, including:

[0116] Step S701: The terminal device obtains hardware attribute information of the terminal device.

[0117] Step S702: The terminal device generates a deployment request based on hardware attribute information.

[0118] In this embodiment, the terminal device can collect hardware attribute information of the terminal device through built-in sensors, operating system APIs, or specific hardware query tools. The hardware attribute information may include attribute information such as processor model, memory capacity, storage type, and GPU model. The hardware attribute information may also include other attribute information, which is not limited in the embodiment of the present application. Based on the collected hardware attribute information, the terminal device can create a deployment request.

[0119] Step S703: The terminal device sends a deployment request to the server.

[0120] Step S704: The server queries the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request.

[0121] In some embodiments, after step S704, the rapid deployment method may further include: in response to the absence of a target computing kernel corresponding to the hardware attribute information, determining a parameter search space corresponding to the hardware attribute information based on the hardware attribute information. Based on the parameter search space, determining multiple target computing parameters related to the hardware attribute information. Based on the multiple target computing parameters, generating a target computing kernel corresponding to the hardware attribute information.

[0122] As an example, based on the device type of the terminal device included in the hardware attribute information, the cache and register sizes are determined to establish a formula for hardware utilization. According to the formula for hardware utilization, the hardware utilization corresponding to each calculation parameter in the operator primitive set is calculated, and the hardware specification heuristic algorithm is used to filter out the target calculation parameters whose hardware utilization meets the set conditions. The hardware specification heuristic algorithm can be based on hardware constraints and WebGPU specifications. Taking WebGPU as an example, the block_tile size is used to determine the hardware utilization of registers, L1 caches, threads, and blocks. According to the number of blocks of registers, L1 caches, and GPU cores, the number of blocks that each GPU core can process is calculated. Thus, the target calculation parameters corresponding to the filtered block_tile are obtained. According to the filtered target calculation parameters, the WebGPU shader language is compiled into a WSGI target calculation kernel.

[0123] In some embodiments, before step S701, the computing parameters in the operator primitive set may be pre-screened to eliminate computing parameters that are not related to hardware computing resources. Specifically, first obtain an operator primitive set and a hardware attribute information set; the operator primitive set may include multiple computing parameters corresponding to different computing tasks, and the hardware attribute information set may include attribute information that characterizes the computing performance of different devices. Based on the hardware attribute information set, a performance evaluation is performed on each computing parameter in the operator primitive set to obtain a performance evaluation result corresponding to each computing parameter. Based on the performance evaluation results corresponding to each computing parameter, the computing parameters included in the operator primitive set are screened to obtain the screened computing parameters.

[0124] In some embodiments, the computing parameters included in the operator primitive set can be screened according to the consistency of the performance evaluation results corresponding to the computing parameters in the hardware attribute information set to obtain the screened computing parameters. For example, the cache_read computing parameter includes two parameter states, "on" and "off". When in the "on" parameter state, the intermediate results of the calculation can be stored in the shared memory for multiple calculation reuse, thereby reducing the overall amount of calculation. If the performance of the cache_read computing parameter in the "on" state in the hardware attribute information set is better than that in the "off" state, it is determined that cache_read is "on" and consistent, and the parameter state of the computing parameter can be fixed to the "on" state, and the cache_read computing parameter is determined to be a computing parameter that is not related to hardware computing resources, and the cache_read computing parameter is removed from the operator primitive set.

[0125] Step S705: The server sends the queried target computing kernel to the terminal device.

[0126] The specific description of each step of steps S704-S705 is the same as that of the above steps S102-S103, and will not be repeated here.

[0127] Step S706: The terminal device runs the target computing kernel through WebGPU.

[0128] The specific description of each step of steps S703 and S706 is the same as that of the above steps S501 and S503, and will not be repeated here.

[0129] According to the method of the embodiment of the present application, the terminal device can send a deployment request containing hardware attribute information to the server, and receive the target computing kernel returned by the server. The most suitable computing kernel deployed on the terminal device. Quick deployment and operation without the need to install additional software. The user only needs to open the device browser to automatically obtain and load the latest target computing kernel, reasoning engine and reasoning model. By performing reasoning tasks on the local device through the target computing kernel, reasoning engine and reasoning model, user privacy and data security can be protected. When there is no target computing kernel directly corresponding to the hardware attribute information, the server generates an adapted kernel through the parameter search space. By optimizing the computing parameters related to the device attributes using heuristic methods, the search space for computing parameter selection can be effectively narrowed, the parameter search time can be reduced, the deployment efficiency can be improved, and the demand for rapid deployment can be met. Generate the computing kernel that best suits the hardware attributes, and the computing kernel can achieve efficient computing performance even in a hardware-diverse environment.

[0130] refer to Figure 8 , application scenarios of the rapid deployment method provided in the embodiments of the present application Figure 1 , which is used for the rapid deployment of WebGPU-based reasoning systems. Figure 8 As shown, the inference system may include a cloud server A and a device browser B, wherein the cloud server A may include a Web application A1, a model library A2, and a cloud compilation inference module A3. The device browser B may include a Web application A1 and a WebGPU B2.

[0131] Web application A1 may include: a packaged reasoning system Web application. Model library A2 may include reasoning models. Cloud-compiled reasoning module A3 may include: a reasoning engine, an optimization parameter filter, a parameter space to be searched, a Web JIT compiler (Web Just-In-Time Compiler), a kernel candidate queue, and an optimized kernel library. Web application A1 in device browser B can be obtained by downloading Web application A1 from cloud server A when device browser B1 is initialized. The reasoning model can also be downloaded from cloud server A when device browser B1 is initialized.

[0132] The inference engine in the device browser B can be the latest version of the inference engine downloaded from the cloud server A when the device is initialized. The inference engine can be used to perform actual inference tasks and use the optimized computing kernel to perform computing tasks.

[0133] The optimization parameter filter can be used to pre-filter the computing parameters in the operator primitive set and remove the computing parameters that are irrelevant to the hardware computing resources.

[0134] The parameter space to be searched can be used to: calculate the hardware utilization corresponding to each computing parameter in the operator primitive set based on the hardware attribute information, and use the hardware specification heuristic algorithm to filter out the target computing parameters whose hardware utilization meets the set conditions.

[0135] The Web JIT compiler can be used to compile the filtered target computing parameters into a WSGI target computing kernel through the WebGPU shader language. The Web JIT compiler can be deployed on the cloud server A and / or the device browser B according to the needs.

[0136] The kernel candidate queue can be used to push the computing kernel generated by the Web JIT compiler to the device browser B in the form of a candidate queue for performance evaluation.

[0137] The optimized kernel library can be used to: store the target computing kernel based on the association between the target computing kernel and the hardware attribute information, and query the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information.

[0138] The kernel evaluation queue can be used to: use WebGPU B2 to evaluate the performance of the candidate computing kernels in the candidate queue in turn, and send the evaluation results to the cloud server A.

[0139] WebGPU B2 can be used to perform graphics rendering and general computing using GPU1, GPU2, and GPU3 in device browser B.

[0140] refer to Fig. 9 , application scenarios of the rapid deployment method provided in the embodiments of the present application Figure 2 , applied to the rapid deployment of WebGPU inference systems based on device browsers.

[0141] Step S70, cloud offline parameter space screening. The cloud performs a one-time large-scale offline computing parameter screening, pre-screens the computing parameters in the operator primitive set, and removes computing parameters that are not related to hardware computing resources. The screened computing parameters will be stored in the cloud and continuously used in subsequent processes.

[0142] Step S71, the device browser is initialized. When the device browser is started, the latest web application is obtained from the cloud, and the web application contains the latest version of the inference engine. In the web application, you can select the inference model you want to use, and pull the inference model from the cloud to the device to perform the inference task. When running in an offline state, the inference model on the device side can still perform the inference task.

[0143] Step S72: the device attempts to obtain an optimized computing kernel from the cloud, and attempts to obtain a computing kernel optimized for the hardware computing resource from the cloud according to the hardware ID.

[0144] Step S73, determine whether the optimized kernel is successfully obtained. If the optimized computing kernel is successfully queried, then proceed to step S79 to register and run the optimized computing kernel to perform the computing task. If the optimized computing kernel is not queried, then proceed to step S74 to screen the cloud online parameter space.

[0145] Step S74, cloud-based online parameter space screening: The hardware utilization corresponding to each computing parameter in the operator primitive set is calculated based on the hardware ID, and the target computing parameters whose hardware utilization meets the set conditions are filtered out using the hardware specification heuristic algorithm.

[0146] Step S75: The cloud JIT compiler generates a computing kernel, and compiles it into a WSGI computing kernel through the WebGPU shader language according to the screened target computing parameters.

[0147] Step S76: The cloud sends the computing kernel to be evaluated to the device through the kernel queue to be evaluated. The WSGI computing kernel generated by the Web JIT compiler is pushed to the device in the form of the kernel queue to be evaluated for kernel performance evaluation.

[0148] Step S77: the device performs kernel performance evaluation and returns the evaluation result to the cloud. The device performs performance test of the computing kernel to be evaluated, measures the computing efficiency and performance of each computing kernel to be evaluated, and then returns the evaluation result to the cloud.

[0149] Step S78, the cloud optimizes the kernel library and updates it. According to the evaluation result returned by the device, the cloud updates the optimized kernel library to ensure that the computing kernel stored in the optimized kernel library is always the version with the best performance and can be used by other devices with the same hardware specifications.

[0150] Step S79: The device registers the optimal kernel. The device registers and runs the optimized computing kernel to perform the computing task.

[0151] The following is a description of an exemplary structure of the server 80 provided in the embodiment of the present application implemented as a software module. In some embodiments, Fig.10As shown, the software modules in the server 80 may include: a first receiving module 801, which can be used to receive a deployment request sent by a terminal device; a query module 802, which can be used to query the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request; the target computing kernel at least includes computing instructions that can be directly run on the WebGPU network graphics processor; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language; a first sending module 803, which can be used to send the queried target computing kernel to the terminal device, so that the terminal device runs the target computing kernel through WebGPU.

[0152] In some embodiments, the query module 802 can be used to: determine the parallel processing capability of the hardware computing resources of the terminal device based on the hardware attribute information; determine the target hardware utilization of the computing kernel on the hardware computing resources based on the parallel processing capability; and determine the target computing kernel corresponding to the hardware attribute information based on the target hardware utilization.

[0153] In some embodiments, the query module 802 can be used to: determine multiple candidate computing kernels corresponding to the hardware attribute information based on the hardware attribute information; determine a target computing kernel from multiple candidate computing kernels based on the matching degree between the hardware attribute information and the candidate computing kernels; the matching degree is determined according to the historical computing performance corresponding to the candidate computing kernels.

[0154] In some embodiments, the query module 802 can be used to: perform feature analysis on the hardware attribute information to obtain performance weights; the performance weights represent the degree of influence of the hardware attribute information on the computing performance of the WebGPU; based on the performance weights and the hardware attribute information, determine the target computing kernel corresponding to the hardware attribute information.

[0155] In some embodiments, the query module 802 can be used to: determine the comprehensive hardware utilization of the computing kernel on the hardware computing resources based on the parallel processing capability and the performance weights corresponding to the hardware attribute information; the comprehensive hardware utilization represents the efficiency of the computing kernel in utilizing the hardware computing resources when running on the WebGPU; and determine the target computing kernel corresponding to the hardware attribute information based on the comprehensive hardware utilization.

[0156] In some embodiments, the server 80 may also include an evaluation module, which can be used to: send multiple computing cores to be tested to the terminal device in sequence through a transmission queue; receive performance evaluation results sent by the terminal device; and determine the target computing core corresponding to the hardware attribute information from the multiple computing cores to be tested based on the performance evaluation results corresponding to each candidate computing core.

[0157] In some embodiments, the server 80 may further include a generation module, which may be used to: in response to the absence of a target computing kernel corresponding to the hardware attribute information, determine a parameter search space corresponding to the hardware attribute information based on the hardware attribute information; determine multiple target computing parameters related to the hardware attribute information based on the parameter search space; and generate a target computing kernel corresponding to the hardware attribute information based on the multiple target computing parameters.

[0158] The following further describes an exemplary structure of the terminal device 90 provided in the embodiment of the present application implemented as a software module. In some embodiments, for example Fig.11 As shown, the software modules in the terminal device 90 may include: a second sending module 901, which can be used to send a deployment request to the server; the deployment request includes the hardware attribute information of the terminal device; a second receiving module 902, which can be used to receive the target computing kernel sent by the server; a deployment module 903, which can be used to run the target computing kernel through the WebGPU network graphics processor; the target computing kernel at least includes computing instructions that can be run directly on the WebGPU; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language.

[0159] In some embodiments, the terminal device 90 also includes a testing module, which can be used to: receive multiple computing kernels to be tested sent by the server; perform performance evaluation tests on each computing kernel to be tested through WebGPU to obtain corresponding performance evaluation results; the performance evaluation results represent the computing performance of the computing kernel to be tested running on WebGPU; and send the performance evaluation results to the server.

[0160] In some embodiments, the deployment module 903 can be used to: configure the hardware computing resources of the terminal device based on the target computing kernel and hardware attribute information to obtain the configured hardware computing resources; schedule the computing tasks to the configured hardware computing resources through WebGPU; and execute the computing tasks based on the computing instructions included in the target computing kernel.

[0161] It should be noted that the description of the terminal device and the server in the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figures 1 to 9 The present invention can be understood by referring to the description of any one of the accompanying drawings.

[0162] According to an embodiment of the present application, the present application also provides a non-transitory computer-readable storage medium.

[0163] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0165] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0167] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0168] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0169] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps disclosed in this solution can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in this application can be achieved, and this document does not limit this.

[0170] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0171] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A rapid deployment method, applied to a server, comprising: Receiving a deployment request sent by a terminal device; Based on the hardware attribute information of the terminal device included in the deployment request, query the target computing kernel corresponding to the hardware attribute information; The target computing kernel at least includes computing instructions that can be directly run on a WebGPU network graphics processor; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in a WebGPU shader language; The queried target computing kernel is sent to the terminal device, so that the terminal device runs the target computing kernel through the WebGPU.

2. The method according to claim 1, querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request, comprises: Based on the hardware attribute information, determining the parallel processing capability of the hardware computing resources of the terminal device; Determining a target hardware utilization of a computing kernel on the hardware computing resources based on the parallel processing capability; Based on the target hardware utilization, a target computing kernel corresponding to the hardware attribute information is determined.

3. The method according to claim 1, wherein the querying, based on the hardware attribute information of the terminal device included in the deployment request, the target computing kernel corresponding to the hardware attribute information comprises: Based on the hardware attribute information, determining a plurality of candidate computing kernels corresponding to the hardware attribute information; Determining the target computing kernel from a plurality of candidate computing kernels based on the matching degree between the hardware attribute information and the candidate computing kernels; The matching degree is determined according to the historical computing performance corresponding to the candidate computing kernel.

4. The method according to claim 1, querying the target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request, comprises: Performing feature analysis on the hardware attribute information to obtain a performance weight; The performance weight represents the influence of the hardware attribute information on the computing performance of the WebGPU; Based on the performance weight and the hardware attribute information, a target computing kernel corresponding to the hardware attribute information is determined.

5. The method according to claim 2, wherein the querying, based on the hardware attribute information of the terminal device included in the deployment request, the target computing kernel corresponding to the hardware attribute information comprises: Determining a comprehensive hardware utilization rate of a computing kernel on the hardware computing resources based on the parallel processing capability and the performance weight corresponding to the hardware attribute information; The comprehensive hardware utilization represents the efficiency of the computing kernel in utilizing the hardware computing resources when the computing kernel is running on the WebGPU; Based on the comprehensive hardware utilization, a target computing kernel corresponding to the hardware attribute information is determined.

6. The method according to claim 1, before receiving the deployment request sent by the terminal device, the method further comprises: Sending the plurality of computing kernels to be tested to the terminal device in sequence through a transmission queue; Receiving a performance evaluation result sent by the terminal device; Based on the performance evaluation results corresponding to the candidate computing kernels, a target computing kernel corresponding to the hardware attribute information is determined from the multiple computing kernels to be tested.

7. The method according to claim 1, after querying the target computing kernel corresponding to the hardware attribute information of the terminal device included in the deployment request, the method further comprises: In response to the absence of a target computing kernel corresponding to the hardware attribute information, determining a parameter search space corresponding to the hardware attribute information based on the hardware attribute information; Based on the parameter search space, determining a plurality of target computing parameters related to the hardware attribute information; Based on the multiple target computing parameters, a target computing kernel corresponding to the hardware attribute information is generated.

8. A rapid deployment method, applied to a terminal device, comprising: Send a deployment request to the server; The deployment request includes hardware attribute information of the terminal device; Receiving a target computing kernel sent by the server; Running the target computing kernel via a WebGPU network graphics processor; The target computing kernel includes at least computing instructions that can be directly run on the WebGPU; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language.

9. A server, comprising: A first receiving module, used to receive a deployment request sent by a terminal device; A query module, configured to query a target computing kernel corresponding to the hardware attribute information based on the hardware attribute information of the terminal device included in the deployment request; The target computing kernel at least includes computing instructions that can be directly run on a WebGPU network graphics processor; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in a WebGPU shader language; The first sending module is used to send the queried target computing kernel to the terminal device, so that the terminal device runs the target computing kernel through the WebGPU.

10. A terminal device, comprising: A second sending module is used to send a deployment request to the server; The deployment request includes hardware attribute information of the terminal device; A second receiving module, used for receiving the target computing kernel sent by the server; A deployment module, used to run the target computing kernel through a WebGPU network graphics processor; The target computing kernel includes at least computing instructions that can be directly run on the WebGPU; the computing instructions match the hardware attribute information, and the computing instructions are instructions written in the WebGPU shader language.

Images