Model processing method and device, equipment, storage medium and chip

Abstract

The invention relates to a model processing method and device, equipment, a storage medium and a chip. The method comprises the steps of obtaining to-be-processed data; calling an end-to-end model to perform reasoning calculation on the to-be-processed data to obtain a corresponding processing result; wherein the end-to-end model at least comprises a memory abstraction module, a memory access module and a memory memory module, the memory abstraction module is used for distributing corresponding memory space for to-be-cached data, and the memory access module is used for calling a preset handle to access the memory abstraction module. And the memory memory module is used for caching data of different timestamps so as to perform reasoning calculation of the end-to-end model by using the data of the corresponding timestamps. In this way, model reasoning calculation can be rapidly and efficiently carried out, and therefore the efficiency and convenience of model processing can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model processing method, device, equipment, storage medium and chip. Background Art

[0002] As AI models are increasingly used, different AI models need to be designed for different application scenarios. For example, in the scenario of smartphone face unlocking, a corresponding face recognition model needs to be designed; in the object classification scenario, a corresponding object classification model needs to be designed. When faced with complex task scenarios, high-precision AI models are often also more complex; under the limitations of factors such as terminal memory and cost, the deployment of high-precision AI models still faces huge challenges. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a model processing method, device, equipment, storage medium and chip, which can quickly and efficiently perform model reasoning calculations, thereby helping to improve the efficiency and convenience of model processing.

[0004] According to a first aspect of an embodiment of the present disclosure, a model processing method is provided, including: Get the data to be processed; Calling the end-to-end model to perform inference calculation on the data to be processed to obtain corresponding processing results; Among them, the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module. The memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data of different timestamps so as to use the data of corresponding timestamps to perform inference calculations of the end-to-end model.

[0005] In some embodiments, the end-to-end model also includes a view feature module, in which at least one cross-modal operator is inserted, and the cross-module operator is used to perform network segmentation on the view feature module to obtain at least a corresponding first module network and a second module network; wherein the first module network is used to obtain input data and preprocess the input data to obtain corresponding preprocessed data; the second module network is used to perform inference calculations on the preprocessed data, and the input data at least includes the data to be processed.

[0006] In some embodiments, calling the end-to-end model to perform inference calculation on the data to be processed includes: fusing m kernel functions corresponding to the end-to-end model to obtain a fused kernel function, where m is a positive integer; The fusion kernel function is called using an asynchronous interface to perform inference calculation on the data to be processed.

[0007] In some embodiments, the end-to-end model includes at least one operator, and before calling the end-to-end model to perform reasoning calculation on the data to be processed, the method further includes: Performing type determination on a target operator to obtain an operator type corresponding to the target operator, wherein the operator type includes a computationally intensive operator and a memory access intensive operator, and the target operator is an operator in the at least one operator; The target operator is optimized based on the operator type corresponding to the target operator.

[0008] In some embodiments, the method further comprises: Performing performance analysis on the end-to-end model to obtain performance indicators of each operator in the end-to-end model; The target operator is an operator whose performance indicator in the end-to-end model is not within a corresponding preset range.

[0009] In some embodiments, the data and an identifier of the data are associated and bound, and / or the memory space is allocated based on the size and / or type of the data to be cached.

[0010] In some embodiments, the data comprises a tensor.

[0011] In some embodiments, the corresponding timestamp is at least one timestamp within a preset time period determined by the memory storage module based on the current timestamp.

[0012] According to a second aspect of an embodiment of the present disclosure, there is provided a model processing device, including: An acquisition module, configured to acquire data to be processed; A processing module is configured to call an end-to-end model to perform reasoning calculations on the data to be processed to obtain corresponding processing results; Among them, the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module. The memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data of different timestamps so as to use the data of corresponding timestamps to perform inference calculations of the end-to-end model.

[0013] For the contents not introduced or described in the embodiments of the present disclosure, please refer to the relevant introduction in the aforementioned method embodiments, and the embodiments of the present disclosure are not limited thereto.

[0014] According to a third aspect of an embodiment of the present disclosure, there is provided a computer device, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to execute the executable instructions to implement the steps of the above-mentioned model processing method.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the model processing method provided in the first aspect of the present disclosure are implemented.

[0016] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, comprising: a processor and an interface; the processor is used to read instructions to execute the steps of the above-mentioned model processing method.

[0017] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: a computer device obtains data to be processed; an end-to-end model is called to perform inference calculations on the data to be processed to obtain corresponding processing results; wherein the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module, the memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data with different timestamps so as to perform inference calculations of the end-to-end model using the data with corresponding timestamps. In this way, through the abstraction of the above-mentioned memory abstraction module, memory access module and memory memory module, the time consumption of memory copying of data in the model is eliminated, and the overall latency of the end-to-end model deployment is optimized. At the same time, model inference calculations can be performed quickly and efficiently, which is conducive to improving the efficiency and convenience of model processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] Figure 1 It is a navigation planning flow chart provided by the prior art.

[0020] Figure 2 The figure is a schematic diagram of a model processing scenario according to an exemplary embodiment.

[0021] Figure 3 It is a structural diagram of an end-to-end model according to an exemplary embodiment.

[0022] Figure 4 It is a flowchart of a model processing method according to an exemplary embodiment.

[0023] Figure 5It is a schematic diagram of the structure of a model processing device according to an exemplary embodiment.

[0024] Figure 6 The diagram is a schematic structural diagram of a terminal device according to an exemplary embodiment.

[0025] Figure 7 The figure is a schematic diagram showing the structure of a server according to an exemplary embodiment.

[0026] Figure 8 The figure is a schematic diagram showing the structure of a chip according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0028] It should be noted that all actions of acquiring signals, information or data in the present disclosure are performed with the authorization given by the owner of the corresponding device.

[0029] With the introduction of AI models, many new technologies have emerged in autonomous driving, such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Generative Adversarial Network (GAN), attention mechanism Transformer, etc. It combines the three functions of perception, planning and control, and can output the final result by inputting raw data, clearing the boundaries between functional modules, simplifying the system framework, and improving operating efficiency. At the same time, it also reduces the dependence on LiDAR and high-precision maps, reducing costs, but the integrated model needs to process data more timely and improve the response speed of the system, which brings greater challenges and requirements to model deployment. For example, see Figure 1 This is a navigation control flow chart provided by the prior art. Figure 1The perception module collects corresponding sensor data through sensors, and then forwards the collected sensor data to the positioning module. The positioning module combines the NPN network to perform positioning processing and sends the processed data to the planning and control module. The planning and control module then combines the information of the navigation system to perform navigation planning and control, thereby planning the corresponding path trajectory. This solution requires reliance on components such as lidar and high-precision maps, which is not conducive to promotion and application, and undoubtedly brings greater challenges to model deployment.

[0030] To solve the above problems, the present disclosure provides a model processing method, device, equipment, storage medium and chip. Some embodiments applicable to the present disclosure are introduced below.

[0031] 1. End-to-end model An end-to-end model refers to the complete process of an AI model from input to output, without human intervention or intermediate steps. For example, in the field of autonomous driving, an end-to-end model can integrate multiple functional modules such as perception, planning, and control, directly input the raw data collected by the sensor into the model, and directly output operational control instructions after processing, such as steering wheel angle, accelerator pedal opening, etc. The end-to-end model involved in the present disclosure can be applied to any scenario that supports the use of model reasoning, for example, it may include but is not limited to navigation planning in the field of autonomous driving, computer equipment planning and control in smart homes, speech recognition scenarios, image classification scenarios, natural language processing scenarios or other custom scenarios, etc., and this disclosure does not make too many restrictions and details on this.

[0032] Taking navigation planning in the field of autonomous driving as an example, see Figure 2 FIG. 1 is a schematic diagram of a specific scenario of model processing according to an exemplary embodiment of the present disclosure. Figure 2 In the navigation planning scenario in the field of autonomous driving, the present disclosure can input navigation system-related data and sensor-related data (specifically, for example, raw data collected by sensors, road arrow signs of the navigation system, road condition feature data, etc.) into an end-to-end model for processing, and output a planned driving path / driving trajectory, etc. The internal processing of the end-to-end model will be described in detail below in the present disclosure and will not be elaborated here.

[0033] This disclosure does not limit the internal structure of the above end-to-end model. For example, see Figure 3 FIG. 1 is a schematic diagram showing a structure of an end-to-end model according to an exemplary embodiment. Figure 3The end-to-end model 100 shown may include: a memory abstraction module 101 (MemoryBuffer), a memory access module 102 (MemoryPlan), and a memory storage module 103 (MemoryBank). Optionally, the above end-to-end model 100 may also include a combination of one or more of the following modules according to actual needs: a view feature module 104 (Bird's Eye View Feature, BEV Feature), a backbone network 105 (Backbone), an encoding module 106, a decoding module 107, and an anchor module 108 (Anchor). The present disclosure takes the above end-to-end model 100 including all the above modules as an example, and it may increase or decrease corresponding functional modules according to actual conditions, and the present disclosure does not constitute a limitation on this. Among them: The above-mentioned memory abstraction module 101 is mainly used for abstraction of data memory, and can allocate central processing unit (CPU) or graphics processing unit (GPU) memory space for data to be cached. Generally, the data involved in the above-mentioned end-to-end model of the present disclosure (such as the intermediate data of the model or the above-mentioned data to be cached) can all be tensors. In a specific implementation, the present disclosure can allocate corresponding memory space according to the size and / or type of the above-mentioned data to be cached. For example, if the above-mentioned data to be cached is floating-point float data, 4 bytes of memory space can be allocated for each byte of the above-mentioned data to be cached; if the above-mentioned data to be cached is half-precision floating-point float16 data, 2 bytes of memory space can be allocated for each byte of the above-mentioned data to be cached. In this way, developers do not need to pay attention to the underlying implementation, shield the differences between different memories, and facilitate easy use.

[0034] The memory access module 102 is mainly used for memory application handles, which can be simply understood as pointers to memory and can be used to access the memory abstract module 101. For example, in a specific implementation, the memory access module 102 can call the corresponding handle previously applied for to access the memory abstract module 101.

[0035] The above-mentioned memory module 103 is mainly used to cache data of different timestamps involved in the end-to-end model, and can use vector design to simulate a circular queue. It stores the timestamp information corresponding to the data within a fixed period of time, such as the data of the corresponding timestamp within a fixed period of time. In this way, the historical timestamps within the preset time period can be searched according to the current timestamp, such as the previous one or more timestamps and timestamp data closest to the current timestamp. It is convenient to use these historical data for model reasoning or model training in the future, which can solve the problems of timing and speed errors during model calculation.

[0036] It should be noted that, through the abstraction and processing of the above-mentioned memory abstraction module 101 (MemoryBuffer), memory access module 102 (MemoryPlan) and memory memory module 103 (MemoryBank), the present disclosure can eliminate the time consumption of memory copy of input and output data (such as tensors) before and after model inference, and has the ability of temporal memory in both time and space dimensions, thereby optimizing the overall latency of end-to-end model deployment and effectively solving the problem of model timing.

[0037] The above-mentioned view feature module 104 is mainly used for feature extraction, such as extracting image features, converting images into a bird's-eye view, etc., which is not limited in this disclosure. In order to optimize the model performance under different precisions, for example, the model reasoning takes a long time under single-precision float32 precision and it is difficult to meet the real-time requirements, and there are errors in the model reasoning results under half-precision float16 precision, etc., the present disclosure inserts cross-modal operators (such as clip operators) to prevent gradient disappearance, limit the range of weights or activation values, etc., so as to avoid numerical instability or overflow problems, thereby further segmenting the above-mentioned view feature module 104, and realizing the deployment of end-to-end models under corresponding precision, thereby significantly improving the frame rate of the model. The present disclosure does not limit the number of the above-mentioned cross-modal operators, which can be determined according to actual conditions. The introduction of the above-mentioned cross-modal operators can be referred to the detailed description below in this disclosure, which will not be elaborated here. In practical applications, taking the navigation planning scenario in the field of autonomous driving as an example, the above-mentioned view feature module 104 can adopt the traditional BEV visual model fusion navigation matching algorithm to perceive the changing curbs, road arrow signs, intersection features and other data in real time, and then integrate lane features and navigation features to obtain ultra-long-range navigation capabilities, which can effectively solve the problem that complex road conditions are difficult to structure, etc. The present disclosure does not make too many limitations and details on this.

[0038] The backbone network 105 is mainly used to process the input raw data (such as raw data collected by sensors and / or raw data of the navigation system, etc.), and can be reused in different parts of the model structure, such as perception, decision-making and planning in navigation planning. The computational efficiency of the model can be improved by sharing the backbone network 105. The present disclosure does not limit the internal structure of the backbone network 105. For example, it may include a multi-head attention mechanism Transformer, etc. By utilizing the multi-head attention mechanism and serialization capability of the Transformer, it can effectively process complex data. For example, in the navigation planning scenario in the field of autonomous driving, it can analyze the data around the vehicle, understand the interaction between different objects, and predict their future behaviors, etc. This disclosure does not go into too much detail.

[0039] The encoding module 106 and the decoding module 107 are mainly used for data encoding and data decoding, and can be combined with the view feature module 104 to decode some key / important data. For example, in the navigation planning scenario in the field of autonomous driving, dynamic obstacles, road structures and other common obstacles can be decoded, thereby generating corresponding planning and control actions, and obtaining a planned driving path / driving trajectory, etc.

[0040] The anchor point module 108 is mainly used for detecting and selecting target objects. The target objects may refer to objects to be detected, which may include but are not limited to vehicles, drivers, passengers, trees, dogs, cats, other obstacles or other custom objects. The present disclosure does not limit the specific implementation of the anchor point module 108. For example, an object detection algorithm may be used to detect and select the corresponding target object. The object detection algorithm includes but is not limited to the YOLO algorithm, the region-based convolutional neural network RCNN algorithm, the single multi-frame detector SSD algorithm or other custom algorithms for object detection, etc. The present disclosure does not make too many limitations and details on this.

[0041] 2. Various Optimization Strategies (1) System overall performance analysis The present disclosure may use a system-wide performance analysis tool (such as NVIDIA Nsight System) to conduct an in-depth analysis of the end-to-end model 100, thereby analyzing and obtaining one or more performance indicators of the end-to-end model at the system level. The performance indicator refers to a parameter or indicator used to reflect the performance of the end-to-end model, which may include, but is not limited to, for example, any one or more combinations of the following indicators: the overall time consumption of the model, the time consumption of each operator in the model, the proportion, the hardware resource occupancy indicator of each operator in the model (such as CUDA core, tensor core, and memory read and write bandwidth, etc.) or other indicators used to evaluate model performance, etc., and the disclosure does not make too many restrictions and details on this. Optionally, the present disclosure may identify the performance bottleneck of the end-to-end model based on the above performance indicators. For example, when the overall time consumption of the above model exceeds 80%, it can be determined that the delay is long and the model processing speed is slow; the present disclosure may optimize the time consumption of the end-to-end model, such as optimizing the time consumption of each operator in the end-to-end model to shorten / save the processing time of each operator, thereby reducing / optimizing the overall time consumption of the model, etc., and the present disclosure does not make too many restrictions and details on this.

[0042] (2) Graphics processing CUDA Graphs technology The above-mentioned graphics processing CUDA Graphs technology is a performance optimization function technology, which can reduce process overhead and improve processing efficiency. The present disclosure can use the above-mentioned graphics processing CUDA Graphs technology to fuse kernel functions (kernel), and combine any number of asynchronous CUDA user programmable interface APIs together. Only one operation is required to start multiple GPU operations on the CUDA Graphs graph through a single CPU operation, thereby reducing the overhead of kernel function kernel launch, that is, greatly reducing the overhead of multiple kernel launches (kernel launch). The specific implementation of the above-mentioned graphics processing CUDA Graphs technology will be described in detail below in this disclosure and will not be elaborated here.

[0043] (3) Operator Performance Analysis The present disclosure may use operator performance analysis tools (such as NVIDIA Nsigh Compute) to perform performance analysis on the operators in the above end-to-end model. For example, it is usually possible to perform focused performance analysis on operators that take a long time and plug-in operators (such as Deformable Attention operators, etc.) in the model. Specifically, the type of operator such as the roofline curve analysis may be used. For example, it may be compute-intensive or memory-intensive, etc. The present disclosure does not limit this, and it can be determined according to actual conditions. Then, targeted in-depth optimization is performed based on the type of the operator to ensure that the kernel function has a certain occupancy rate and a high memory bandwidth utilization rate, etc. The specific implementation of the above in-depth optimization will be described in detail below in the present disclosure and will not be elaborated here.

[0044] (4) Memory Bank 103 zero-copy reasoning optimization technology This disclosure proposes a zero-copy reasoning optimization technology for the memory module 103 (MemoryBank) to optimize the model performance under different precisions through detailed statistics and analysis of the time consumption indicators of each part of the above-mentioned end-to-end model deployment. In the specific implementation, the data and data identifiers (such as tensors and tensor IDs) can be associated and bound to avoid repeated allocation of memory during the model reasoning process. Through the abstraction of the above-mentioned memory abstraction module 101 (MemoryBuffer), memory access module 102 (MemoryPlan) and memory memory module 103 (MemoryBank), the memory copy time of input and output data (such as tensors) before and after model reasoning can be eliminated, and the problem of model timing can be effectively solved. For the introduction of the above-mentioned modules, please refer to the relevant description in the aforementioned embodiment, which will not be repeated here.

[0045] The computer devices involved in the present disclosure may include, but are not limited to, for example, terminal devices, servers, service platforms or other custom settings. Among them, the above-mentioned terminal devices may also be referred to as terminal devices (terminal), user equipment (UE), mobile stations (MS), mobile terminal devices (MT), etc. The terminal device may be a car with communication function, a smart car, a mobile phone (Mobile Phone), a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (Virtual Reality, VR) terminal device, an augmented reality (Augmented Reality, AR) terminal device, a wireless terminal device in industrial control (Industrial Control), a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in smart grid (Smart Grid), a wireless terminal device in transportation safety (Transportation Safety), a wireless terminal device in smart city (Smart City), a wireless terminal device in smart home (Smart Home), etc. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the above-mentioned computer devices.

[0046] Based on the above examples, see Figure 4 FIG. 1 is a flow chart showing a model processing method according to an exemplary embodiment. Figure 4 The method shown can be applied to a computer device, and the method may include the following implementation steps: S401, obtaining data to be processed.

[0047] In the present disclosure, the above-mentioned data to be processed may refer to data waiting to be processed. For example, in the navigation planning scenario in the field of autonomous driving, the above-mentioned data to be processed may include raw data collected by sensors and raw data of the navigation system, such as image data collected by cameras, raw data collected by lidar, vehicle position data collected by the positioning system, navigation data of the navigation system (such as road arrow markings, road condition feature data), etc.

[0048] The present disclosure does not limit the implementation method for obtaining the above-mentioned data to be processed. For example, it can be obtained through corresponding sensor collection, or obtained from other devices (such as terminals, servers, etc.) through the network. The present disclosure does not make too many limitations on this.

[0049] S402. Call the end-to-end model to perform inference calculations on the data to be processed to obtain corresponding processing results; wherein the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module, the memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data with different timestamps so as to use the data with corresponding timestamps to perform inference calculations of the end-to-end model.

[0050] By implementing the embodiments of the present disclosure, a computer device obtains data to be processed; calls an end-to-end model to perform inference calculations on the data to be processed to obtain corresponding processing results; wherein the end-to-end model includes at least a memory abstraction module, a memory access module, and a memory memory module, wherein the memory abstraction module is used to allocate corresponding memory space for data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data with different timestamps so as to perform inference calculations of the end-to-end model using data with corresponding timestamps. In this way, through the abstraction of the above-mentioned memory abstraction module, memory access module, and memory memory module, the time consumption of memory copying of data in the model is eliminated, and the overall latency of the end-to-end model deployment is optimized. At the same time, model inference calculations can be performed quickly and efficiently, which is conducive to improving the efficiency and convenience of model processing.

[0051] In step S402, the present disclosure does not limit the specific form of expression of the data involved in the above end-to-end model (such as the above data to be cached), for example, it can usually be represented by tensors. The present disclosure does not limit the specific implementation method of the above reasoning calculation. For example, in the process of reasoning calculation, the present disclosure can avoid repeated allocation of data memory in the process of model reasoning by using the zero-copy reasoning optimization technology of the above memory memory module 103 (MemoryBank) and associating and binding data and data identifiers (such as tensor tensor and tensor identifier tensor ID). The introduction of the above memory abstraction module 101, memory access module 102 and memory memory module 103 in the present disclosure can refer to the relevant introduction in the above embodiment. In the specific implementation, the above memory abstraction module 101 can allocate corresponding memory space for the data to be cached involved in the model. The allocation implementation method of the memory space is not limited, for example, it can be allocated according to the size and / or type of the above data to be cached. Generally, the above data to be cached and the above allocated memory space are positively correlated. For example, the larger the above data to be cached, the larger the allocated memory space; conversely, the smaller the above data to be cached, the smaller the allocated memory space. For another example, when the data to be cached is floating-point data, 4 bytes of memory space can be allocated for each byte of the data to be cached; for another example, when the data to be cached is half-precision floating-point float16 data, 2 bytes of memory space can be allocated for each byte of the data to be cached, etc. The present disclosure does not make too many restrictions or details on this.

[0052] The memory access module 102 may call a preset handle to access the memory abstraction module 101, so that the memory abstraction module 101 allocates CPU or GPU memory space. The preset handle may be a handle pre-applied by the memory access module 102, which may be simply understood as a pointer to memory / memory space for accessing the memory abstraction module 101.

[0053] The above-mentioned memory storage module 103 can cache data of different timestamps (here specifically, it can be data of some historical timestamps, which can also be called historical data) so that the corresponding model reasoning calculation can be performed later using the data of the corresponding timestamp. The above-mentioned corresponding timestamp can be at least one timestamp within a preset time period determined based on the current timestamp. In specific implementation, the present disclosure can determine part or all of the timestamps within the preset time period according to the current timestamp as the above-mentioned corresponding timestamp, and then use the data of these timestamps (i.e., historical data) to perform corresponding model reasoning calculations in the above-mentioned end-to-end model. The data here can be the above-mentioned data to be processed, or other data; for example, in the navigation planning scenario in the field of autonomous driving, the above-mentioned data can be the image data of the current frame (i.e., the above-mentioned data to be processed), or image data of different time frames (historical data), etc. The above-mentioned end-to-end model can use the image data of each historical timestamp cached in the above-mentioned memory storage module 103 to perform reasoning calculations, thereby predicting the motion trajectory of the target object (e.g., pedestrians), and then planning the corresponding driving trajectory / path to avoid pedestrians, etc., and the implementation details thereof are not described in detail in this disclosure. The internal structure of the above end-to-end model can also correspond to the relevant introduction in the aforementioned embodiments, which will not be repeated here.

[0054] Some optional embodiments of the present disclosure are described below.

[0055] In an optional embodiment, the end-to-end model includes the view feature module 104. The present disclosure further optimizes the model performance under different precisions. For example, the model reasoning under single-precision float32 precision takes a long time and is difficult to meet the real-time requirements. The model reasoning result under half-precision float16 precision has errors. The present disclosure can insert at least one cross-modal operator in the view feature module 104. The present disclosure does not limit the number of the cross-modal operators, which can be determined according to actual conditions. For example, it can be two or more. The cross-modal operator can support data conversion or association between different modalities. For example, it can include but is not limited to the clip operator, which can perform multimodal learning, align images and texts, and realize the function of language description of images, thereby realizing information conversion between images and languages. The cross-modal operator can be specifically used to perform network division or function division on the view feature module 104, and at least obtain the corresponding first module network and second module network. Among them, the first module network can be specifically used to obtain corresponding input data and pre-process the input data to obtain corresponding pre-processed data. The preprocessing refers to a data processing method that is pre-customized by the system or user, which may include but is not limited to data cleaning, data conversion, data discretization or other customized data processing, etc., and the present disclosure does not make too many restrictions and details on this. The above-mentioned input data refers to the data input into the above-mentioned view feature module 104, which may include but is not limited to, for example, the above-mentioned data to be processed, or other customized data, etc. For example, in the navigation planning scenario in the field of autonomous driving, the above-mentioned input data may include but is not limited to, for example, raw data collected by sensors, raw data of the navigation system (such as curb change data, road arrow markings and intersection feature data, etc.) or other data. The above-mentioned second module network can be specifically used to perform inference calculations on the above-mentioned preprocessed data. The specific implementation method of the above-mentioned inference calculation is not limited, for example, using a convolutional neural network for inference calculations, etc., and the present disclosure does not make too many restrictions and details on this.

[0056] In another optional embodiment, the present disclosure does not limit the internal structure of the above-mentioned end-to-end model. For example, it may include at least one operator, and the number of the operators may be determined according to actual needs. The present disclosure may perform an overall performance analysis on the above-mentioned end-to-end model to obtain the performance indicators of each operator in the above-mentioned end-to-end model. The description of the performance indicator may refer to the relevant description in the aforementioned embodiment, and will not be repeated here. Specifically, for example, the present disclosure may use a system overall performance analysis tool (such as NVIDIA Nsight System) to perform an in-depth analysis of the above-mentioned end-to-end model 100, so as to analyze and obtain the performance indicators of each operator in the end-to-end model at the system level. Optionally, based on the above-mentioned performance indicators, the present disclosure may identify the performance bottleneck of the end-to-end model. For example, when the above-mentioned performance indicators include the overall time consumption of the model, if the overall time consumption of the model exceeds 80%, it can be determined that the delay is long and the model processing speed is slow; the present disclosure can optimize the time consumption of the above-mentioned end-to-end model, for example, optimize the time consumption of each operator in the above-mentioned end-to-end model to shorten / save the processing time of each operator, thereby reducing / optimizing the overall time consumption of the model, etc. The present disclosure does not make too many restrictions and details on this.

[0057] In another optional embodiment, the present disclosure may use graphics processing technology such as CUDA Graphs to fuse the m kernel functions occupied / corresponding to the above end-to-end model, thereby obtaining a fused kernel function. Wherein, m is a positive integer customized according to actual conditions. Furthermore, an asynchronous interface may be used to call the above fused kernel function to perform inference calculations on the above data to be processed. Multiple GPU operations corresponding to the m kernel functions are started through one operation, thereby greatly reducing the overhead of multiple kernel launches.

[0058] In another optional embodiment, the present disclosure may perform targeted in-depth optimization on the target operator in the above-mentioned end-to-end model, and the target operator may be any one or more of the at least one operator included in the above-mentioned end-to-end model. Preferably, the target operator may specifically be an operator or a specified operator whose performance index in the above-mentioned end-to-end model is not within the corresponding preset range, for example, it may generally be an operator that takes a long time (takes more than a preset threshold), etc. The preset range may be an indicator determination range pre-defined by the system or the user, which may be an experience range set according to user experience, or a statistical range calculated based on a series of experimental data, etc., and the present disclosure does not make too many restrictions and details on this. The present disclosure does not limit the number of the above-mentioned target operators, which may be determined according to actual conditions. For the convenience of describing the present disclosure, the following description of the relevant content takes a target operator as an example, but it does not constitute a limitation. In a specific implementation, the present disclosure may perform type determination on the above-mentioned target operator to obtain the operator type corresponding to the above-mentioned target operator. The above-mentioned operator type may be a type pre-defined by the system or the user, which may include but is not limited to, for example, computationally intensive, memory-intensive or other custom operator types. The present disclosure does not limit the specific implementation methods of the above-mentioned type determination. For example, the roofline curve can be used to analyze the operator type of the target operator. Specifically, the input computing power of the target operator and its corresponding output performance can be analyzed through the roofline curve, and then the operator type of the above-mentioned target operator can be determined based on the output performance. The output performance here is used to reflect the amount of calculation of the operator per unit time. Generally, the larger the output performance, the better the computing performance of the target operator; conversely, the smaller the output performance, the worse the computing performance of the target operator, etc. The present disclosure does not make too many limitations and details on this.

[0059] After obtaining the operator type corresponding to the target operator, the present disclosure can further optimize the target operator in a targeted manner based on the operator type corresponding to the target operator to ensure that the kernel function has a certain occupancy rate, ensure a high memory bandwidth and / or bandwidth utilization rate, etc. In a specific implementation, for example, when the operator type of the target operator is used to indicate that the target operator is a computationally intensive operator, that is, when the operator type of the target operator is computationally intensive, the present disclosure can perform time-consuming optimization processing on the target operator to reduce the instruction calculation time of the operator. For another example, when the operator type of the target operator is used to indicate that the target operator is a memory-access intensive operator, that is, when the operator type of the target operator is memory-access intensive, the present disclosure can perform resource optimization processing on the target operator to reduce the resources occupied by the operator and speed up the corresponding data access, such as helping the data related to the operator (such as input data, intermediate data or result data generated by the operator calculation, etc.) to be better and faster stored, etc., and the present disclosure does not make too many restrictions and details on this.

[0060] It can be seen that the deployment performance and response speed of the end-to-end model can be significantly improved by adopting the above-mentioned embodiments of the present disclosure. It can not only provide real and accurate processing results for model processing, but also provide an efficient optimization solution for the mass production deployment of the end-to-end model. The above-mentioned implementation scheme of the present disclosure can specifically have the following advantages: (1) Improving the response speed of the system. In the navigation planning scenario in the field of autonomous driving, algorithms, computing power and data are the three important foundations of autonomous driving systems. In the deployment process of the end-to-end model in autonomous driving, through the abstraction of each module of the memory abstraction module (MemoryBuffer), the memory access module (MemoryPlan) and the memory memory module (MemoryBank), the memory copy time of the input and output data before and after the model reasoning is eliminated, the overall delay of the end-to-end model deployment is optimized, and the problem of model timing is effectively solved. At the same time, a zero-copy reasoning technology based on the memory memory module (MemoryBank) is proposed. The copy operation of the input and output data during the conventional model deployment is avoided by memory binding. The historical time information within the preset time period is searched according to the current timestamp, which can effectively solve the problems of model timing information. The use of CUDA Graphs technology can reduce the number of memory copies and kernel launch overhead during model inference, thereby significantly reducing the latency of end-to-end model deployment and speeding up the system response time. This enables the autonomous driving system to process real-time data more quickly, improve the timeliness of decision-making, and provide stronger protection for autonomous driving safety. (2) Simplify the system framework design. The end-to-end model simplifies the traditional multi-module architecture into a unified deep learning model, significantly reducing the complexity of the system. In the traditional model system, each module needs to be developed, tested and debugged separately, while the end-to-end model only needs to develop and train one model, simplifying the development process. In the traditional model system, interfaces need to be designed between modules to ensure seamless data transmission and processing, while in the end-to-end model, all processing is completed through one model, and there is no need to design complex interfaces, which can reduce the difficulty of system integration. (3) Optimize resource consumption. This public solution improves system capabilities by simply expanding training resources through precision optimization, model segmentation, and operator insertion, combined with data-driven optimization, making it possible to run complex models with lower computing resources, greatly reducing the resource consumption and cost of the equipment, and improving energy efficiency. (4) Enhance user experience and safety. By improving the data processing capability and system stability of the model, this solution can improve the detection of lane lines and obstacles in navigation planning scenarios in the field of autonomous driving, and effectively reduce problems such as jitter, false detection, and missed detection. When using the autonomous driving function, users can enjoy a smoother and more stable visual interface. At the same time, the system's low-latency feedback also provides strong support for preventing potential dangers, significantly improving the safety and user experience of autonomous driving.(5) Improve the reliability and stability of the system. Through in-depth performance analysis and optimization of the end-to-end model, we ensure that the system can maintain high performance in various scenarios. The end-to-end model can adapt to scenario changes (such as new road conditions and driving conditions in navigation planning scenarios) through continuous learning and data updates, thereby improving the reliability and stability of the system. Since the end-to-end model learns directly from the data, it can adjust its behavior strategy more flexibly in different scenarios (such as different regions, different weather conditions, and different traffic regulations), which is also conducive to improving the reliability and stability of the model.

[0061] In a specific implementation, a computer device can obtain data to be processed; call an end-to-end model to perform inference calculations on the data to be processed to obtain corresponding processing results; wherein the end-to-end model includes at least a memory abstraction module, a memory access module, and a memory memory module, wherein the memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data with different timestamps so as to use the data with corresponding timestamps to perform inference calculations of the end-to-end model. In this way, through the abstraction of the above-mentioned memory abstraction module, memory access module, and memory memory module, the time consumption of memory copying of data in the model is eliminated, and the overall latency of the end-to-end model deployment is optimized. At the same time, model inference calculations can be performed quickly and efficiently, which is conducive to improving the efficiency and convenience of model processing.

[0062] Based on the above examples, see Figure 5 FIG. 1 is a schematic diagram showing the structure of a model processing device according to an exemplary embodiment. Figure 5 The device 500 shown can be applied to a computer device, and the device 500 may include an acquisition module 501 and a processing module 502. Among them: The acquisition module 501 is configured to acquire data to be processed; The processing module 502 is configured to call the end-to-end model to perform reasoning calculation on the data to be processed to obtain corresponding processing results; Among them, the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module. The memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data of different timestamps so as to use the data of corresponding timestamps to perform inference calculations of the end-to-end model.

[0063] In some embodiments, the end-to-end model also includes a view feature module, in which at least one cross-modal operator is inserted, and the cross-module operator is used to perform network segmentation on the view feature module to obtain at least a corresponding first module network and a second module network; wherein the first module network is used to obtain input data and preprocess the input data to obtain corresponding preprocessed data; the second module network is used to perform inference calculations on the preprocessed data, and the input data at least includes the data to be processed.

[0064] In some embodiments, the processing module 502 is configured to: fusing m kernel functions corresponding to the end-to-end model to obtain a fused kernel function, where m is a positive integer; The fusion kernel function is called using an asynchronous interface to perform inference calculation on the data to be processed.

[0065] In some embodiments, the end-to-end model includes at least one operator. Before calling the end-to-end model to perform reasoning calculation on the data to be processed, the processing module 502 is further configured to: Performing type determination on a target operator to obtain an operator type corresponding to the target operator, wherein the operator type includes a computationally intensive operator and a memory access intensive operator, and the target operator is an operator in the at least one operator; The target operator is optimized based on the operator type corresponding to the target operator.

[0066] In some embodiments, the processing module 502 is further configured to: Performing performance analysis on the end-to-end model to obtain performance indicators of each operator in the end-to-end model; The target operator is an operator whose performance indicator in the end-to-end model is not within a corresponding preset range.

[0067] In some embodiments, the data and an identifier of the data are associated and bound, and / or the memory space is allocated based on the size and / or type of the data to be cached.

[0068] In some embodiments, the data comprises a tensor.

[0069] In some embodiments, the corresponding timestamp is at least one timestamp within a preset time period determined by the memory storage module based on the current timestamp.

[0070] For the contents not introduced or described in the embodiments of the present disclosure, please refer to the relevant introduction in the aforementioned embodiments, which will not be repeated here.

[0071] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and the program instructions, when executed by a processor, implement the steps of the model processing method provided by the present disclosure.

[0072] Based on the above embodiments, the structure of the computer device is introduced below. The computer device involved in the present disclosure may include but is not limited to terminal devices, servers, or other custom devices. Figure 6 and Figure 7 The following uses computer devices as terminal devices and servers as examples to specifically introduce the corresponding structures. Figure 6 6 is a schematic diagram of a terminal device according to an exemplary embodiment. For example, the terminal device 600 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0073] Reference Figure 6 The terminal device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output interface 612 , a sensor component 614 , and a communication component 616 .

[0074] The processing component 602 generally controls the overall operation of the terminal device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned model processing method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0075] The memory 604 is configured to store various types of data to support operations on the terminal device 600. Examples of such data include instructions for any application or method operating on the terminal device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0076] The power supply component 606 provides power to various components of the terminal device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 600.

[0077] The multimedia component 608 includes a screen that provides an output interface between the terminal device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the terminal device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0078] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the terminal device 600 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0079] The input / output interface 612 provides an interface between the processing component 602 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0080] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the terminal device 600. For example, the sensor assembly 614 can detect the open / closed state of the terminal device 600, the relative positioning of the components, such as the display and keypad of the terminal device 600, and the sensor assembly 614 can also detect the position change of the terminal device 600 or a component of the terminal device 600, the presence or absence of contact between the user and the terminal device 600, the orientation or acceleration / deceleration of the terminal device 600, and the temperature change of the terminal device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0081] The communication component 616 is configured to facilitate wired or wireless communication between the terminal device 600 and other devices. The terminal device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0082] In an exemplary embodiment, the terminal device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned model processing method.

[0083] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of a terminal device 600 to complete the above-mentioned upper model processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0084] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned model processing method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction can be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned model processing method is implemented; alternatively, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned model processing method.

[0085] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above-mentioned model processing method when executed by the programmable device.

[0086] See also Figure 7 FIG. 1 is a schematic diagram showing a structure of a server according to an exemplary embodiment. Figure 7 The server 700 includes a processing component 722, which further includes one or more processors, and a memory resource represented by a memory 732 for storing instructions, such as applications, that can be executed by the processing component 722. The application stored in the memory 732 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to perform the above-mentioned image processing method.

[0087] The server 700 may also include a power supply component 726 configured to perform power management of the server 700, a wired or wireless network interface 750 configured to connect the server 700 to a network, and an input / output interface 758. The server 700 may operate based on an operating system stored in the memory 732.

[0088] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above-mentioned model processing method when executed by the programmable device.

[0089] See also Figure 8 FIG. 1 is a schematic diagram showing the structure of a chip according to an exemplary embodiment. Figure 8 The chip 800 shown includes a processor 801 and an interface 802. Optionally, it may also include a memory 803. The number of the processor 801 may be one or more, and the number of the interface 802 may be multiple.

[0090] In one embodiment, for a case where a chip is used to implement the method embodiment of the present disclosure: The interface 802 is used to receive or output signals; The processor 801 is used to execute part or all of the content in the embodiment of the model processing method.

[0091] It is understandable that the processor in the embodiment of the present disclosure can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0092] It is understandable that the memory in the embodiments of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0093] It should be noted here that the description of the above storage medium, device and chip embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

[0094] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0095] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A model processing method, characterized in that: include: Get the data to be processed; Calling the end-to-end model to perform inference calculation on the data to be processed to obtain corresponding processing results; Among them, the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module. The memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data of different timestamps so as to use the data of corresponding timestamps to perform inference calculations of the end-to-end model.

2. The method according to claim 1, characterized in that The end-to-end model also includes a view feature module, in which at least one cross-modal operator is inserted, and the cross-module operator is used to perform network segmentation on the view feature module to obtain at least a corresponding first module network and a second module network; wherein the first module network is used to obtain input data and preprocess the input data to obtain corresponding preprocessed data; the second module network is used to perform inference calculation on the preprocessed data, and the input data at least includes the data to be processed.

3. The method according to claim 1, characterized in that The calling of the end-to-end model to perform reasoning calculation on the data to be processed includes: fusing m kernel functions corresponding to the end-to-end model to obtain a fused kernel function, where m is a positive integer; The fusion kernel function is called using an asynchronous interface to perform inference calculation on the data to be processed.

4. The method according to claim 1, characterized in that The end-to-end model includes at least one operator. Before calling the end-to-end model to perform reasoning calculation on the data to be processed, the method further includes: Performing type determination on a target operator to obtain an operator type corresponding to the target operator, wherein the operator type includes a computationally intensive operator and a memory access intensive operator, and the target operator is an operator in the at least one operator; The target operator is optimized based on the operator type corresponding to the target operator.

5. The method according to claim 4, characterized in that The method further comprises: Performing performance analysis on the end-to-end model to obtain performance indicators of each operator in the end-to-end model; The target operator is an operator whose performance indicator in the end-to-end model is not within a corresponding preset range.

6. The method according to any one of claims 1 to 5, characterized in that The data and the identifier of the data are associated and bound, and / or the memory space is allocated based on the size and / or type of the data to be cached.

7. The method according to claim 6, characterized in that The data includes a tensor.

8. The method according to any one of claims 1 to 5, characterized in that The corresponding timestamp is at least one timestamp within a preset time period determined by the memory storage module based on the current timestamp.

9. A model processing device, characterized in that: include: An acquisition module, configured to acquire data to be processed; A processing module is configured to call the end-to-end model to perform reasoning calculations on the data to be processed to obtain corresponding processing results; Among them, the end-to-end model includes at least a memory abstraction module, a memory access module and a memory memory module. The memory abstraction module is used to allocate corresponding memory space for the data to be cached, the memory access module is used to call a preset handle to access the memory abstraction module, and the memory memory module is used to cache data of different timestamps so as to use the data of corresponding timestamps to perform inference calculations of the end-to-end model.

10. A computer device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the method described in any one of claims 1 to 8.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

12. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 8.

Images