Data pipeline optimization method for multi-layer convolution depth fusion based on compilation optimization

Abstract

The invention provides a data pipeline optimization method for multi-layer convolution depth fusion based on compilation optimization. The method comprises the following steps: an ISP module processes partial row pixel data of an image; the ISP module sends the processed pixel data to the AI module; the AI module is used for processing the received pixel data; the ISP module processes unprocessed pixel data of the image according to a preset line number; and repeating the steps S2-S4 until all rows of pixel data of the image are processed. After the ISP module processes part of the row pixel data of the image but not all the row pixel data, the processed pixel data is sent to the AI module for processing, so that the AI module does not need to wait for the ISP module to process all the row pixel data of the image and then starts to process the pixel data, the delay of the AI module is reduced, and the image processing efficiency is improved. And the execution efficiency of the artificial intelligence image signal processor is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization. Background Art

[0002] Artificial Intelligence Image Signal Processors (AI ISPs) are widely used in various fields, particularly in image and video processing, such as smartphone photography, security monitoring, autonomous driving, medical imaging, and industrial inspection. By combining AI (Artificial Intelligence) and ISPs (Image Signal Processors), image and video quality can be significantly improved, enhancing system performance and user experience.

[0003] Current AI neural network models have numerous parameters and high computational complexity. Furthermore, since the ISP is highly sensitive to latency and power consumption, very high requirements are placed on inference performance on edge devices. Therefore, AI ISP inference optimization has become a key research area. Current AI ISP inference performance optimization falls into the following categories: model optimization, compilation optimization, and dedicated hardware optimization. Model optimization primarily involves model compression, including pruning, quantization, and distillation. Model compression is an effective method for reducing model size and computational complexity. In recent years, lightweight models specifically designed for edge devices have also emerged. Compilation optimization primarily involves optimizing model execution using efficient compilers and runtime systems, such as TVM (Tensor Virtual Machine) and MLIR (Multi-Level Intermediate Representation). These systems convert high-level deep learning models into low-level instructions, and utilize various compilation optimization techniques, including graph optimization and automatic operator generation, to optimize execution efficiency. Dedicated hardware optimization primarily involves designing low-power and low-latency edge AI accelerators specifically for AI ISP application scenarios.

[0004] There are two different design approaches for optimizing AI ISP hardware. One approach is to integrate the ISP and AI modules and solidify the entire process. This approach offers the advantage of a high degree of coupling between the ISP and AI modules, resulting in lower latency and power consumption. However, this design wastes chip area because the ISP and AI modules are part of the same data pipeline. When the ISP is inactive, the AI ​​module also doesn't work, preventing time-sharing reuse. Furthermore, the fixed design of the ISP and AI modules limits the chip to supporting only a few fixed models. To run new models, the ISP and AI modules must be redesigned. Another approach is to separate the ISP and AI modules into two relatively independent modules. This design offers the advantage of time-sharing reuse of the AI ​​module, allowing it to be called by other applications even when the ISP is inactive. It also supports a wider range of neural network models and can be flexibly implemented through software. However, due to the low degree of coupling between the ISP and AI modules, latency and power consumption are higher.

[0005] For the solution where the ISP module and AI module are designed separately, the simplified data flow is as follows: Figure 1 As shown in the figure, the data pipeline of the ISP module is usually: sensor data is sent from the sensor to the ISP, pixel processing is performed on the ISP, and after the pixel processing is completed, it is transferred from the ISP to the memory, which can be DDR (Double Data Rate Synchronous Dynamic Random Access Memory). In actual ISP chips, multiple ISP modules can be designed and the pipeline also has multiple stages. In order to simplify the problem Figure 1 It is assumed that there is only one ISP module, which only contains three stages of pipeline. The pipeline of the AI ​​module is usually: data is copied from DDR to RAM (Random Access Memory), RAM sends the data to the AI ​​chip, the AI ​​chip processes the data and then sends it to RAM, and the data is copied from RAM to DDR. In actual AI chips, multiple AI modules can be designed and the pipeline also has multiple stages. In order to simplify the problem, Figure 1 It is assumed that there is only one AI module, which only contains three stages of pipeline. Figure 1Where Time represents time, Sensor_0-Sensor_3 represents that the ISP module generates data in 4 batches, ISP_0-ISP_3 represents that the ISP module processes data in 4 batches, DDR_0-DDR_3 represents that data is exchanged with the DDR unit in 4 batches, and the DDR unit can be shared by the ISP module and the AI ​​module; RAM_0-RAM_3 represents that the AI ​​module exchanges data with the RAM unit in 4 batches, and AI Model_0-AI Model_3 represents that the AI ​​module processes data in 4 batches. Figure 1 It can be seen that the data flow of the AI ​​module begins only after the ISP module processes the data flow of the entire image, which results in a higher delay in the AI ​​module. Summary of the Invention

[0006] The present invention provides a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization to solve the technical problem of high latency of the AI ​​module when the ISP module and the AI ​​module are designed separately.

[0007] To solve the above technical problems, the present invention provides a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization. The method is applied to an artificial intelligence image signal processor, which includes an ISP module and an AI module designed separately. The method includes the following steps:

[0008] S1, the ISP module processes pixel data of some rows of the image;

[0009] S2. The ISP module sends the processed pixel data to the AI ​​module;

[0010] S3, the AI ​​module processes the received pixel data;

[0011] S4. The ISP module processes the unprocessed pixel data of the image according to a preset number of rows;

[0012] S5. Repeat steps S2-S4 until all row pixel data of the image are processed by the ISP module and the AI ​​module.

[0013] Preferably, the ISP module includes an ISP unit and a memory unit, and the AI ​​module includes an AI unit and a RAM unit; step S2 specifically includes the following steps:

[0014] The ISP unit sends the processed pixel data to the memory unit;

[0015] The RAM unit obtains the processed pixel data from the memory unit;

[0016] The RAM unit sends the processed pixel data to the AI ​​unit.

[0017] Preferably, the following steps are further included between steps S3 and S4: the AI ​​unit sends the processed pixel data to the RAM unit in batches; and the RAM unit sends the received data to the memory unit.

[0018] Preferably, the step of the AI ​​unit sending the processed pixel data to the RAM unit in batches specifically includes the following steps: the AI ​​unit divides the processed pixel data into multiple batches with the same number of rows and then sends them to the RAM unit batch by batch.

[0019] Preferably, step S3 specifically includes the following steps: the AI ​​module processes the received pixel data according to a convolutional neural network, the convolutional neural network includes multiple convolutional layers, and the output end of the previous convolutional layer is connected to the input end of the next convolutional layer.

[0020] Preferably, step S3 specifically includes the following steps: the AI ​​module divides the received pixel data into multiple batches; the AI ​​module processes the multiple batches of pixel data batch by batch in batch order according to a convolutional neural network, and the convolutional neural network includes multiple convolutional layers, and the output end of the previous convolutional layer is connected to the input end of the next convolutional layer.

[0021] Preferably, the step of the AI ​​module dividing the received pixel data into multiple batches specifically includes the following steps: the AI ​​module divides the received pixel data into multiple batches with the same number of rows.

[0022] The present invention also provides another data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization, which is applied to an artificial intelligence image signal processor and includes the following steps:

[0023] S11. Initialize a convolutional neural network as an original network calculation graph, where the original network calculation graph includes multiple nodes, and each node includes one or more convolutional layers;

[0024] S12. Input the first node and the second node in the original network calculation graph, as well as the node formed by merging the first node and the second node, into the AI ​​compiler respectively;

[0025] S13, designing a different number of rows for each batch of pixels to be processed, and generating corresponding candidate solutions for the three nodes in S12 respectively;

[0026] S14. Using a preset mathematical evaluation model, a performance evaluation is performed on all candidate solutions of the three nodes to obtain the candidate solution with the best performance for each node. If the performance of the node after the fusion of the first node and the second node is better than the performance of the two nodes before the fusion, the first node and the second node are fused; otherwise, the first node and the second node are not fused.

[0027] S15. According to the result of S14, if the first node and the second node are fused, that is, they become a new fused node, then the original network computation graph is updated; if no fusion occurs, then the second node and the third node in the original network computation graph are fused;

[0028] S16. Using the same principles as steps S12 to S15, determine whether the second node and the third node can be fused, and so on, determine whether all adjacent nodes in the original network calculation graph can be fused, until the original network calculation graph is no longer updated;

[0029] S17. Use the final network calculation graph to implement any of the data pipeline optimization methods for multi-layer convolution deep fusion based on compilation optimization described above.

[0030] Preferably, the mathematical evaluation model is used to evaluate the execution time of the artificial intelligence image signal processor on the same image.

[0031] The present invention provides a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization. After the ISP module processes part of the pixel data of the image instead of all the pixel data of the image, the processed pixel data is sent to the AI ​​module for processing. In this way, the AI ​​module does not have to wait for the ISP module to process all the pixel data of the image before starting to process the pixel data, thereby reducing the delay of the AI ​​module and improving the execution efficiency of the artificial intelligence image signal processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a data flow diagram when the ISP module and AI module are designed separately in the existing technology.

[0033] Figure 2 This is a schematic diagram of the data flow between an ISP module and an AI module provided by one embodiment of the present invention.

[0034] Figure 3 This is a data flow diagram of an AI module provided by one embodiment of the present invention.

[0035] Figure 4 This is a schematic diagram of a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization provided by one embodiment of the present invention.

[0036] Figure 5 This is a schematic diagram of another data pipeline optimization method for multi-convolution deep fusion based on compilation optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the objectives, advantages, and features of the present invention more clear, the data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization proposed by the present invention is further described in detail below with reference to the accompanying drawings. It should be noted that the drawings are all in a very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0038] In the description of the present invention, qualifiers such as "first" and "second" are added for convenience of description and reference and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features qualified with qualifiers such as "first" and "second" may explicitly or implicitly include one or more of such features.

[0039] like Figure 2 and Figure 4 As shown, this embodiment provides a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization. The method is applied to an artificial intelligence image signal processor, which includes an ISP module and an AI module designed separately. The method includes the following steps:

[0040] S1. The ISP module processes pixel data of a portion of an image row. The processed image includes multiple rows of data. Conventional ISP modules process all image data before the AI ​​module processes it. However, in this method, the ISP module processes only a portion of the pixel data row, rather than all of it, before sending it to the AI ​​module for processing, thereby reducing latency in the AI ​​module. When the ISP module processes the pixel data row, the column pixel data included in the portion of the row is complete.

[0041] S2. The ISP module sends the processed pixel data to the AI ​​module.

[0042] S3. The AI ​​module processes the received pixel data.

[0043] S4. The ISP module processes the unprocessed pixel data of the image according to a preset number of rows.

[0044] S5. Repeat steps S2-S4 until all row pixel data of the image are processed by the ISP module and the AI ​​module.

[0045] This embodiment provides a data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization. After the ISP module processes part of the pixel data of the image rows instead of all the pixel data of the image rows, the processed pixel data is sent to the AI ​​module for processing. In this way, the AI ​​module does not have to wait for the ISP module to process all the pixel data of the image rows before starting to process the pixel data, thereby reducing the delay of the AI ​​module and improving the execution efficiency of the artificial intelligence image signal processor.

[0046] Preferably, Figure 2 As shown, the ISP module includes an ISP unit and a memory unit, and the AI ​​module includes an AI unit and a RAM unit. Step S2 specifically includes the following steps: the ISP unit sends the processed pixel data to the memory unit; the RAM unit obtains the processed pixel data from the memory unit; and the RAM unit sends the processed pixel data to the AI ​​unit. The ISP module and the AI ​​module can share a memory unit, which can be a DDR unit. The memory unit and the RAM unit can integrate the data pipelines of the ISP module and the AI ​​module, thereby reducing the latency of the AI ​​module. Figure 2 Where "Time" indicates time, "Sensor_0-Sensor_3" indicates that the ISP module generates data in four batches, "ISP_0-ISP_3" indicates that the ISP module processes data in four batches, "DDR_0-DDR_3" indicates that the ISP module exchanges data with the DDR unit in four batches, and the DDR unit can be shared by the ISP module and the AI ​​module, "RAM_0-RAM_3" indicates that the AI ​​module exchanges data with the RAM unit in four batches, and "AI Model_0-AI Model_3" indicates that the AI ​​module processes data in four batches.

[0047] Preferably, Figure 3 As shown, between steps S3 and S4, the following steps are also included: the AI ​​unit sends the processed pixel data to the RAM unit in batches; the RAM unit sends the received data to the memory unit. The AI ​​unit sends the processed pixel data to the RAM unit in batches, which can reduce the bandwidth demand for the RAM. The RAM unit sends the received data, i.e., the pixel data in batches, to the memory unit, which can reduce the bandwidth demand for the memory. Figure 3 In , Time represents time, DDR represents double rate synchronous dynamic random access memory, RAM represents random access memory, Conv0 represents the first convolution layer, Conv1 represents the second convolution layer, and to represents sending to and arriving at. Figure 3In the example, it is assumed that a convolutional neural network segment has only two convolutional layers, and these two convolutional layers are fused into one node, that is, after the Conv0 calculation is completed, the data does not need to be copied to DDR, and the convolution kernel of each convolutional layer is k*k. Figure 3 The data pipeline in the AI ​​module includes the following steps:

[0048] Step 1: At time t0, complete the copy of 2k-2 rows of data from DDR to RAM.

[0049] Step 2: At time t1, continue to copy m rows of data from DDR to RAM.

[0050] Step 3: At time t2, the copy of m rows of data from DDR to RAM is completed. Simultaneously, the first convolutional layer conv0 processes the m+2k-2 rows of data and generates m+k-1 rows of data for the second convolutional layer conv1. The second convolutional layer conv1 then processes the m+k-1 rows of data and generates m rows of data. At this point, data input and processing can be performed in parallel.

[0051] Step 4: At time t3, the copying of m rows of data from DDR to RAM is completed. The first convolutional layer conv0 processes the m+2k-2 rows of data and generates m+k-1 rows of data for the second convolutional layer conv1. The second convolutional layer conv1 processes the m+k-1 rows of data and generates m rows of data. Simultaneously, the m rows of output data generated at time t2 are copied from RAM to DDR. At this point, data input, data processing, and data output can be executed in parallel. Due to the pipeline characteristics, a double buffer is required to buffer the input and output. That is, space is allocated for -2m+2k-2 rows of data for the AI ​​model input, space is allocated for 2m rows of data for the AI ​​model output, space is allocated for m+k-1 rows of data for the output of the first convolutional layer conv0, and space is allocated for m rows of data for the output of the second convolutional layer conv1. This ensures the smooth operation of the entire data pipeline.

[0052] Step 5: Repeat step 4 until all data are processed.

[0053] Preferably, the step of the AI ​​unit sending the processed pixel data to the RAM unit in batches specifically includes the following steps: the AI ​​unit divides the processed pixel data into multiple batches with the same number of rows and then sends the batches to the RAM unit. Dividing the processed pixel data into multiple batches with the same number of rows can keep the data flow on the data pipeline at a fixed value, ensuring the stability of the pipeline.

[0054] Preferably, step S3 specifically includes the following steps: the AI ​​module processes the received pixel data according to a convolutional neural network, the convolutional neural network includes multiple convolutional layers, and the output end of the previous convolutional layer is connected to the input end of the next convolutional layer. Preferably, step S3 specifically includes the following steps: the AI ​​module divides the received pixel data into multiple batches; the AI ​​module processes the multiple batches of pixel data batch by batch in batch order according to the convolutional neural network, the convolutional neural network includes multiple convolutional layers, and the output end of the previous convolutional layer is connected to the input end of the next convolutional layer. The AI ​​module divides the received pixel data into multiple batches and then processes them according to the convolutional neural network, which can improve processing efficiency. The data pipeline optimization method based on compiled optimized multi-layer convolution deep fusion provided in this embodiment can increase the processing speed of the artificial intelligence image signal processor by about 20%.

[0055] Preferably, the step of the AI ​​module dividing the received pixel data into multiple batches specifically includes the following steps: the AI ​​module divides the received pixel data into multiple batches having the same number of rows. The AI ​​module divides the received pixel data into multiple batches having the same number of rows for further processing, thereby maintaining a fixed flow rate of the AI ​​module's data pipeline and ensuring pipeline stability.

[0056] like Figure 5 As shown, based on the same technical concept as the data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization, this embodiment provides another data pipeline optimization method for multi-convolution deep fusion based on compilation optimization, which is applied to an artificial intelligence image signal processor and includes the following steps:

[0057] S11. Initialize a convolutional neural network as an original network computation graph, wherein the original network computation graph comprises a plurality of nodes (network segments), and each of the nodes comprises one or more convolutional layers;

[0058] S12. Input the first node and the second node in the original network calculation graph, as well as the node formed by merging the first node and the second node, into the AI ​​compiler respectively;

[0059] S13. Design a different number of rows for each batch of pixels to be processed, and generate corresponding candidate solutions for each of the three nodes in S12. Based on experience, you can try to design a different number of rows for each batch of pixels to be processed, and then determine the optimal number of rows based on the optimal evaluation result of the mathematical evaluation model.

[0060] S14. Using a preset mathematical evaluation model, a performance evaluation is performed on all candidate solutions of the three nodes to obtain the candidate solution with the best performance for each node. If the performance of the node after the fusion of the first node and the second node is better than the performance of the two nodes before the fusion, the first node and the second node are fused; otherwise, the first node and the second node are not fused.

[0061] S15. According to the result of S14, if the first node and the second node are fused, that is, become a new fused node, then the original network computation graph is updated; if not, the second node and the third node in the original network computation graph are fused;

[0062] S16. Using the same principles as steps S12 to S15, determine whether the second node and the third node can be fused, and so on, determine whether all adjacent nodes in the original network calculation graph can be fused, until the original network calculation graph is no longer updated;

[0063] S17, using the network calculation graph finally obtained to implement the data pipeline optimization method of multi-layer convolution deep fusion based on compilation optimization described in any one of the above. Figure 5 The method shown can be obtained and further optimized Figure 4 The method shown in the figure is shown. AI compiler-based optimization requires layer-by-layer fusion of multiple convolution layers. Assume that a neural network model includes two convolution layers, conv0 and conv1. Without fusion, after computing conv0, data is first copied from RAM to DDR, then from DDR to RAM, and then conv1 is computed. After the computation is complete, data is copied from RAM to DDR. Therefore, the performance of conv0, conv1, and the fused structure of conv0 and conv1—that is, the performance of three network segments—is required to achieve deep fusion of multiple convolution layers.

[0064] This embodiment provides a data pipeline optimization method for multi-layer convolutional deep fusion based on compilation optimization. It integrates the data pipelines of the ISP module and the AI ​​module, enabling efficient connection between the two modules. Compilation optimization is then used to identify the optimal fusion and splitting scheme for the network. The optimization method corresponding to the optimal candidate solution can achieve lower latency.

[0065] Preferably, the mathematical evaluation model is used to evaluate the execution time of the AI ​​image signal processor for a given image. When the AI ​​image signal processor's execution time for the given image is the shortest, the mathematical evaluation model determines the current candidate solution as the optimal candidate solution. In other embodiments, the mathematical evaluation model may also select the optimal candidate solution based on factors such as the clarity and sharpness of the image output by the AI ​​module.

[0066] To sum up, the data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization provided by the present invention is that after the ISP module processes part of the row pixel data of the image instead of all the row pixel data, the processed pixel data is sent to the AI ​​module for processing. In this way, the AI ​​module does not have to wait for the ISP module to process all the row pixel data of the image before starting to process the pixel data, which reduces the delay of the AI ​​module and improves the execution efficiency of the artificial intelligence image signal processor.

[0067] The above description is only a description of the preferred embodiments of the present invention and does not limit the scope of the present invention. Any changes and modifications made by ordinary technicians in this field based on the above disclosure are within the scope of protection of the present invention.

Claims

1. A data pipeline optimization method for multi-layer convolutional deep fusion based on compilation optimization, characterized by: The method is applied to an artificial intelligence image signal processor, which includes an ISP module and an AI module designed separately. The method includes the following steps: S1, the ISP module processes pixel data of some rows of the image; S2. The ISP module sends the processed pixel data to the AI ​​module; S3, the AI ​​module processes the received pixel data; S4. The ISP module processes the unprocessed pixel data of the image according to a preset number of rows; S5. Repeat steps S2-S4 until all row pixel data of the image are processed by the ISP module and the AI ​​module.

2. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 1 is characterized in that: The ISP module includes an ISP unit and a memory unit, and the AI ​​module includes an AI unit and a RAM unit. Step S2 specifically includes the following steps: The ISP unit sends the processed pixel data to the memory unit; The RAM unit obtains the processed pixel data from the memory unit; The RAM unit sends the processed pixel data to the AI ​​unit.

3. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 2 is characterized in that: The following steps are also included between steps S3 and S4: the AI ​​unit sends the processed pixel data to the RAM unit in batches; and the RAM unit sends the received data to the memory unit.

4. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 3 is characterized in that: The step of the AI ​​unit sending the processed pixel data to the RAM unit in batches specifically includes the following steps: the AI ​​unit divides the processed pixel data into multiple batches with the same number of rows and then sends the batches to the RAM unit.

5. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 1, characterized in that: Step S3 specifically includes the following steps: the AI ​​module processes the received pixel data according to a convolutional neural network, where the convolutional neural network includes multiple convolutional layers, and the output end of the previous convolutional layer is connected to the input end of the next convolutional layer.

6. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 1, characterized in that: Step S3 specifically includes the following steps: the AI ​​module divides the received pixel data into multiple batches; the AI ​​module processes the multiple batches of pixel data batch by batch in batch order according to a convolutional neural network, and the convolutional neural network includes multiple convolutional layers, and the output end of the previous convolutional layer is connected to the input end of the next convolutional layer.

7. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 6 is characterized in that: The step of the AI ​​module dividing the received pixel data into multiple batches specifically includes the following steps: the AI ​​module divides the received pixel data into multiple batches with the same number of rows.

8. A data pipeline optimization method for multi-layer convolutional deep fusion based on compilation optimization, characterized in that: The method is applied to an artificial intelligence image signal processor, and the method comprises the following steps: S11. Initialize a convolutional neural network as an original network calculation graph, where the original network calculation graph includes multiple nodes, and each node includes one or more convolutional layers; S12. Input the first node and the second node in the original network calculation graph, as well as the node formed by merging the first node and the second node, into the AI ​​compiler respectively; S13, designing a different number of rows for each batch of pixels to be processed, and generating corresponding candidate solutions for the three nodes in S12 respectively; S14. Using a preset mathematical evaluation model, a performance evaluation is performed on all candidate solutions of the three nodes to obtain the candidate solution with the best performance for each node. If the performance of the node after the fusion of the first node and the second node is better than the performance of the two nodes before the fusion, the first node and the second node are fused; otherwise, the first node and the second node are not fused. S15. According to the result of S14, if the first node and the second node are fused, that is, become a new fused node, then the original network computation graph is updated; if no fusion occurs, the second node and the third node in the original network computation graph are fused; S16. Using the same principles as steps S12 to S15, determine whether the second node and the third node can be fused, and so on, determine whether all adjacent nodes in the original network calculation graph can be fused, until the original network calculation graph is no longer updated; S17. Use the finally obtained network calculation graph to implement the data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization as described in any one of claims 1-7.

9. The data pipeline optimization method for multi-layer convolution deep fusion based on compilation optimization according to claim 8, characterized in that: The mathematical evaluation model is used to evaluate the execution time of the artificial intelligence image signal processor on the same image.

Images