Data processing method and device, electronic equipment and non-transient computer readable storage medium

Abstract

The embodiment of the invention provides a data processing method and device, electronic equipment and a non-transient computer readable storage medium. The embodiment of the invention is used for concurrently accessing the service data and the debugging data in the last-stage cache of the artificial intelligence processor. Hardware-level isolation is carried out on the business data and the debugging data in the last-level cache, so that the read-write performance of the business data in the cache is not affected in the whole process that the artificial intelligence chip starts the debugging function and continuously generates a large amount of debugging data, and therefore the debugging efficiency of developers is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence chips, and more specifically to methods for processing data, apparatus for processing data, electronic devices, and non-transient computer-readable storage media. Background Technology

[0002] The rapid development of artificial intelligence has placed higher demands on computing power, thus giving rise to AI chips specifically designed to accelerate AI workloads. During the operation of an AI processor, in addition to handling normal business data computation tasks, it is also necessary to use a hardware-software collaborative approach for chip debugging, verification, and diagnostics. Traditional debugging methods typically involve hardware writing performance counter data distributed throughout the chip into main memory (e.g., RAM), and then software reading the main memory and parsing the performance data to obtain chip operating status information.

[0003] However, as the last level between the processor core and main memory, the last-level cache receives performance data from other modules, and it also generates performance data itself. This performance data is indiscriminately written to the last-level cache, causing it to consume its capacity and bandwidth, thus impacting the cache's performance in processing business data. This not only reduces the execution efficiency of business programs but also distorts the collected performance data, rendering it inaccurate.

[0004] Furthermore, even if the above problems are mitigated by reducing the bandwidth usage of performance data and compressing it as much as possible, performance data still often needs to be written to memory through the conventional path of the last-level cache. This approach inevitably consumes hardware resources for caching business data and severely limits the amount and accuracy of performance data, failing to meet the increasingly complex debugging needs of artificial intelligence chips.

[0005] Therefore, the data processing scheme for the last-level cache needs to be improved. Summary of the Invention

[0006] This disclosure provides a method for processing data, an apparatus for processing data, an electronic device, and a non-transient computer-readable storage medium.

[0007] This disclosure provides a method for processing data, which is used to concurrently access business data and debugging data in the last-level cache of an artificial intelligence processor. The method includes: receiving a business request through a first port of the last-level cache, processing the business request through a first hardware path connected to the first port, and routing the business data in the business request to a first storage space in the main storage area; receiving a debug write request through a second port of the last-level cache, processing the debug write request through a second hardware path connected to the second port, and routing the debug data in the debug write request to a second storage space in the main storage area; wherein the second hardware path is independent of the first hardware path, and the second storage space and the first storage space correspond to different physical address ranges in the main storage area.

[0008] This disclosure provides a data processing apparatus for concurrently accessing business data and debugging data in the final cache of an artificial intelligence processor. The apparatus includes: a first port configured to receive business requests; a first hardware path connected to the first port and configured to route business data in the business request to a first storage space in the main storage area; a second port configured to receive debug write requests; and a second hardware path connected to the second port and independent of the first hardware path, configured to route debug data in the debug write request to a second storage space in the main storage area; wherein the second storage space and the first storage space correspond to different physical address ranges in the main storage area.

[0009] This disclosure provides an artificial intelligence processor configured to execute the above-described method.

[0010] This disclosure provides an electronic device, including: a main storage area; and an artificial intelligence processor connected to the main storage area, the artificial intelligence processor including a last-level cache, the last-level cache including the aforementioned device.

[0011] This disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the above-described method.

[0012] According to another aspect of this disclosure, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the foregoing aspects or various alternative implementations of the foregoing aspects.

[0013] This disclosure isolates business data from debugging data at the hardware level in the last-level cache, so that the read and write performance of business data in the cache is not affected throughout the process of the artificial intelligence chip enabling the debugging function and continuously generating a large amount of debugging data, thereby improving the debugging efficiency of software developers. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The accompanying drawings in the following description are merely exemplary embodiments of this disclosure.

[0015] Figure 1 A schematic diagram of the architecture of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0016] Figure 2 A flowchart illustrating a method for processing data in the final cache of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0017] Figure 3 A schematic diagram of a dual-path architecture for the final-level cache of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0018] Figure 4 A schematic diagram of the physical isolation partitioning of an electronic device and a main storage area according to an embodiment of the present disclosure is shown.

[0019] Figure 5 A schematic diagram of the parallel operation timing of the business process and debugging function according to an embodiment of the present disclosure is shown on the timeline. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0021] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0022] The rapid development of artificial intelligence has placed higher demands on computing power, giving rise to AI processors specifically designed to accelerate AI workloads. These chips typically employ a many-core parallel processor architecture, leveraging their massive parallel computing capabilities to efficiently process complex algorithms, such as neural network computations in deep learning.

[0023] During the operation of an AI processor, in addition to handling normal business data calculation tasks, it is also necessary to use a hardware-software collaborative approach for chip debugging, verification, and diagnostics. Traditional debugging solutions typically involve hardware writing performance counter data distributed across the chip into memory, and then software reading the memory and parsing the performance data to obtain chip operating status information.

[0024] Figure 1 A schematic diagram of the architecture of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0025] like Figure 1 As shown, the artificial intelligence processor 101 integrates multiple processor core clusters (processor core clusters 1 to N are shown in the figure, where N is a positive integer greater than 1). Each processor core cluster contains multiple processor cores 103, and each processor core 103 is equipped with a private private cache (such as L1 cache 104). Several adjacent processor core clusters form a shared cache unit 105, which contains a shared L2 cache 106.

[0026] Multiple shared cache units 105 share a Last Level Cache (LLC) 107. Located between the L2 cache 106 and the main memory area 110, the Last Level Cache is the last-level cache in a hierarchical storage structure (e.g., L3 cache). Its capacity is typically larger than that of the L1 cache 104 and L2 cache 106. It is used to cache data frequently accessed by the processor core 103, reducing the number of accesses to the main memory area 110 and thus improving overall computational efficiency.

[0027] It should be noted that although the accompanying drawings in this disclosure distinguish between L2 cache and last-level cache, depending on the processor architecture, the last-level cache can be either L2 cache or L3 cache, and this disclosure is not limited to this.

[0028] In the artificial intelligence processor of this disclosure embodiment, each last-level cache 107-1...107-N corresponds to a relatively independent main memory area 110-1...110-N. Each main memory area 110 has a mutually distinct address space. For example, last-level cache 107-1 corresponds to main memory area 110-1, last-level cache 107-2 corresponds to main memory area 110-2, ..., and last-level cache 107-N corresponds to main memory area 110-N. The main memory area can be physically implemented as external memory connected via a memory interface (such as DDR memory on a motherboard), high-bandwidth memory (HBM) integrated into the processor package, or a combination of both; this disclosure is not limited to these forms.

[0029] Based on the above mapping relationship, each last-level cache 107-1...107-N can perform data cache management within its own main storage area, thereby reducing the data sharing requirements between different last-level caches, thus reducing cache consistency maintenance overhead and improving the storage access efficiency of the artificial intelligence processor.

[0030] Traditionally, the last-level caches 107-1...107-N cache data (such as read / write requests from processor core 103) in the cache array. If the data requested by the request is not in the cache, the last-level caches 107-1...107-N will read the corresponding cache line from the main memory area 110-1...110-N, fill it into the cache array, and then return the data to the requester.

[0031] Meanwhile, performance monitoring units may be distributed throughout the chip to collect performance metrics data during chip operation, such as cache hit rate, memory access latency, and bandwidth utilization. However, when the performance metrics data generated by the performance monitoring units need to be written to the main memory area 110-1...110-N, this data will pass through the conventional data path of the last-level cache 107-1...107-N.

[0032] The last-level caches 107-1...107-N, serving as the final stage between processor core 103 and main memory areas 110-1...110-N, have limited bandwidth and cache capacity. In traditional methods, debug data and business data share the same data path. Because performance data itself occupies the cache capacity of the last-level cache 107, the space available for caching business data in the cache array is reduced, thus decreasing the cache hit rate of business data, increasing the number of times business programs access the main memory area, and ultimately affecting the execution performance of business programs.

[0033] Meanwhile, the writing of performance data consumes the data transfer bandwidth between the last-level cache and memory. When debugging data and business data both need to access memory, they compete for limited bandwidth resources, leading to increased latency in business requests and reduced computational efficiency of the AI ​​processor.

[0034] Furthermore, the fact that debugging data and business data share the same data path further limits the methods for collecting and processing performance data. In order to reduce the impact on business performance, traditional solutions have to adopt conservative strategies such as reducing the frequency of performance data collection and compressing the amount of performance data. This results in insufficient accuracy of the collected performance data, reduced reference value, and an inability to meet the increasingly complex debugging needs of artificial intelligence processors.

[0035] Therefore, there is an urgent need for a new technical solution that can efficiently collect and store debugging data without affecting the performance of the last-level cache in processing business data, thereby improving the debugging efficiency of artificial intelligence processors and the work efficiency of software developers.

[0036] Therefore, in this embodiment, business data and debugging data are isolated at the hardware level in the last-level cache, so that the read and write performance of business data in the cache is not affected throughout the process of the artificial intelligence processor enabling the debugging function and continuously generating a large amount of debugging data, thereby improving the debugging efficiency of software developers.

[0037] Specifically, this disclosure provides an apparatus for processing data in the final cache of an artificial intelligence processor. The apparatus includes: a first port configured to receive a service request; a first hardware path connected to the first port and configured to route service data in the service request to a first storage space in the main storage area; a second port configured to receive a debug write request; and a second hardware path connected to the second port and independent of the first hardware path, configured to route debug data in the debug write request to a second storage space in the main storage area; wherein the second storage space and the first storage space correspond to different physical address ranges in the main storage area.

[0038] This disclosure uses two independent hardware paths to process different types of data. Service requests are processed through a dedicated first hardware path, and debug write requests are processed through a dedicated second hardware path. The two paths are separated in hardware structure and do not share transmission bandwidth or queues.

[0039] In an optional aspect, some embodiments of this disclosure also include a special processing mechanism for debug data. The debug data carries only the packet sequence number and not the complete address, and instead of entering the cache array, it is directly written to the second storage space of the main storage area through a bypass mechanism, thereby avoiding waste of cache capacity.

[0040] In another optional aspect, some embodiments of this disclosure also configure unequal independent request number pools for the two hardware paths to ensure that the service path can always maintain high-concurrency request delivery without being affected by the load on the debugging path.

[0041] In another alternative aspect, some embodiments of this disclosure implement strict isolation of physical address ranges at the main memory level (e.g., memory level), with business data and debugging data stored in different address spaces of the main memory area.

[0042] Next, combined Figure 2 The data processing method and flow of this disclosure are described in detail. Figure 2 A flowchart of a method 20 for processing data in the final cache of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0043] Method 20 is used to concurrently access business data and debug data in the last-level cache of the artificial intelligence processor. It is worth noting that "concurrently accessing business data and debug data in the last-level cache of the artificial intelligence processor" mentioned in this disclosure means that the last-level cache has the ability to simultaneously process, schedule, or transmit business requests and debug requests at the hardware architecture level.

[0044] "Concurrency" here refers to the ability of the last-level cache to handle both business requests and debugging requests in parallel or alternately without interference, through physically isolated first and second hardware paths, when both arrive within the same time period (e.g., the same clock cycle or adjacent clock cycles). Even if only business data or debugging data is accessed at a given moment, the last-level cache remains in a logical state with the potential for concurrent processing.

[0045] Method 20 includes operations S201 and S202. There is no strict sequential relationship between operations S201 and S202; they can occur in parallel. Of course, method 20 of this embodiment may also include more or fewer operations, and this disclosure is not limited thereto.

[0046] In operation S201, a service request is received through the first port of the last-level cache, and the service request is processed through the first hardware path connected to the first port, and the service data in the service request is routed to the first storage space of the main storage area.

[0047] Optionally, a business request is a data read / write request generated when the AI ​​processor runs an actual computation program. Its purpose is to read the data required for computation or write the results generated by the computation. Business data refers to data generated, accessed, or modified by the AI ​​processor during the execution of a business program. This data is necessary for the business program to perform its computational functions, including but not limited to the weight parameters, activation values, intermediate computation results, and input / output data of the neural network model. Business data may be repeatedly accessed by the processor core and exhibits spatial and temporal locality; therefore, it is usually cached in the last-level cache to improve access efficiency. However, this disclosure is not limited to this.

[0048] Optionally, the service request carries service data and target address information, where the target address indicates the memory address where the service data should be written, or the memory address where the data to be read is located. The service request can be a read request or a write request. A read request is used to read data from main memory for processor core computation, while a write request is used to write the computation results generated by the processor core back to main memory. This disclosure is not limited thereto.

[0049] Optionally, the first port is on the last-level cache module used to receive data from upstream components (such as...). Figure 1 The first port is a hardware interface for service requests (L1 cache, L2 cache, or processor core). The interface width and clock frequency of the first port can be configured according to actual needs to meet the requirements of high-bandwidth service data transmission. The first port is directly connected to the first hardware path; service requests entering through the first port immediately enter the first hardware path for subsequent processing. However, this disclosure is not limited to this.

[0050] Optionally, the first hardware path is a logical hardware path in the last-level cache used to process business data. This path consists of multiple hardware modules connected in series, including a business data memory access queue, a hit detection unit, and an execution unit. The first hardware path and the second hardware path, which will be detailed later, are independent in hardware implementation. They do not share any hardware resources, including data transmission paths, queue caches, and request management units, to ensure that changes in the load of the second hardware path do not affect the processing capacity of the first hardware path. Of course, this disclosure is not limited to this.

[0051] Optionally, routing refers to the process of data transmission along a transmission path in a hardware system. In this disclosure, routing specifically refers to the process of sending the business data in a business request from the first port through the processing modules of the first hardware path to the first storage space in the main storage area. This routing process is completed collaboratively by the hardware modules in the first hardware path: first, the business request is stored in a business data access queue for queuing; then, a hit check is performed based on the request address; if a hit occurs, data is read from or written to the cache array; if a hit does not occur, data is retrieved from the main storage area and processed; finally, the business data is sent to the first storage space in the main storage area through the execution unit. Of course, this disclosure is not limited to this.

[0052] Optionally, the first storage space in the main storage area refers to the address region within the main storage area specifically used for storing business data. The first storage space and the second storage space correspond to different physical address ranges within the main storage area, and they are independent of each other. The first storage space typically occupies the majority of the address range in the main storage area, providing sufficient data storage space for business programs. Of course, this disclosure is not limited to this.

[0053] Optionally, the first storage space can be configured in a high-bandwidth memory (HBM) encapsulated within an artificial intelligence processor to improve the access efficiency of business data by utilizing its high bandwidth and low latency characteristics; it can also be configured in an external memory (DDR) connected through a memory interface to meet the needs of large-capacity data storage; or it can be distributed in both HBM and DDR, and unified management and access can be achieved through address mapping or data migration mechanisms, thereby balancing performance and capacity.

[0054] Optionally, in one example, the business data in the business request is first written to the first storage space corresponding to HBM to meet the computing requirements of high bandwidth and low latency; when the data needs to be accessed by the general processing unit or processed later, it is then migrated from HBM to the CPU-side memory space corresponding to DDR.

[0055] Optionally, operation S201 includes: storing the service request to a service data access queue on the first hardware path; determining, through a hit detection unit on the first hardware path, whether the service data corresponding to the service request is hit in the cache array; in response to the data corresponding to the service request being hit in the cache array, reading or writing data to a cache line in the cache array through an execution unit on the first hardware path; in response to the data corresponding to the service request not being hit in the cache array, retrieving the corresponding cache line from the main storage area through the execution unit. (Refer to the following...) Figure 3 The process will not be detailed here.

[0056] In operation S202, debug write requests are received through the second port of the last-level cache, and the debug write requests are processed through the second hardware path connected to the second port, routing the debug data in the debug write request to the second storage space of the main storage area.

[0057] In this disclosure, there is no strict order between operation S202 and operation S201; they can occur in parallel. This means that while the AI ​​processor is executing business programs and generating a large number of business requests, the performance monitoring unit can simultaneously generate debug write requests, and the two do not interfere with each other.

[0058] Optionally, debug write requests are performance metric data write requests generated by the performance monitoring unit or the last-level cache itself. Their purpose is to persistently store debug data in memory for subsequent analysis. Unlike business requests, debug write requests are only used to write data and do not involve data reading.

[0059] Optionally, debug data refers to various performance metrics collected by the performance monitoring unit during chip operation, including but not limited to cache hit rate, memory access latency, bandwidth utilization, utilization of each computing unit, temperature sensor data, and power consumption data. Debug data is usually only used for subsequent analysis and is typically read once and then no longer needed. Therefore, it is not suitable to occupy valuable cache resources and is more suitable to be directly routed to main memory (RAM).

[0060] Optionally, the second port is a hardware interface on the last-level cache module used to receive debug write requests from the performance monitoring unit or the internal performance acquisition module. The second port is physically separate from the first port, and the two have independent interfaces and signal paths, and do not share hardware resources.

[0061] Optionally, the second hardware path is a dedicated hardware logic path within the last-level cache module for handling debug write requests. This path consists of hardware modules such as a debug data memory access queue, an address calculation unit, a second request management unit, and a debug request queue. The second hardware path is independent of the first hardware path in hardware implementation.

[0062] In operation S202, "routing" in debug data processing specifically refers to the process of sending debug data in debug write requests from the second port through the debug data access queue, address calculation unit, second request management unit, debug request queue and other processing modules, and finally directly to the second storage space of the main storage area through bypass.

[0063] Specifically, the second hardware path can receive requests using a second port independent of the first hardware path, with separate request entry points for the two. Optionally, operation S202 includes: storing the debug write request to a debug data access queue on the second hardware path; mapping the debug data packet sequence number in the debug write request to a target memory address in the second storage space using an address calculation unit on the second hardware path; and based on the target memory address, sending the debug data to the second storage space via a bypass of the cache array of the last-level cache. (Refer to...) Figure 3 The process will not be detailed here.

[0064] Therefore, the second hardware path can use an independent debug data memory access queue to store requests, without sharing cache resources with the first hardware path's business data memory access queue. Furthermore, the second hardware path uses an independent address calculation unit for address mapping, independent of the address processing logic of the first hardware path. The second hardware path can also use an independent second request management unit to manage request numbers, independent of the request number pool of the first hardware path's first request management unit. Additionally, the second hardware path bypasses data transmission to main memory (RAM), independent of the execution unit of the first hardware path. This hardware independence ensures that the processing of debug data does not cause resource contention for the processing of business data.

[0065] Optionally, the second storage space in the main storage area refers to the address region within the main storage area used for storing debug data. The second storage space and the first storage space correspond to different physical address ranges within the main storage area, and are physically isolated. The second storage space can be configured in DDR for access and debug analysis by general-purpose processing units; it can also be configured in HBM as needed, or distributed across both HBM and DDR. By independently partitioning and configuring the second storage space, isolated storage of debug data and business data, as well as flexible access control, are achieved. Of course, this disclosure is not limited to this.

[0066] Optionally, debug data in a debug write request can be first written to the second storage space in HBM. When the data needs to be accessed by a general-purpose processing unit or processed further, it can then be migrated from HBM to the CPU-side memory space corresponding to DDR. Of course, this disclosure is not limited thereto.

[0067] This physically isolates the storage areas for business data and debug data, ensuring that even a large amount of debug data generated by the second hardware path will not encroach on the capacity of the first storage space. Furthermore, it avoids cache consistency issues between business data and debug data. Since their storage areas do not overlap, the processor core does not need to worry about debug data writes affecting the state of business data in the cache array when accessing business data. Additionally, it simplifies debug data access management. After the software allocates a dedicated address space for debug data during initialization, subsequent debug data writes directly use that address range. The fact that the second and first storage spaces correspond to different physical address ranges in the main storage area also facilitates the reading and analysis of debug data. Developers can directly access the address range of the second storage space to read debug data after the business program has finished running.

[0068] Because the second storage space is physically isolated from the first storage space, and the debug data does not pass through the cache array, the collected debug data has integrity and continuity. Developers can verify whether each debug data packet is stored sequentially according to the sequence number and whether the data content is complete and undamaged. If data is found to be missing or damaged, the time point of the problem can be inferred based on the range of missing sequence numbers to locate the fault.

[0069] Therefore, this disclosure uses two independent hardware paths to process different types of data. Service requests are processed through a dedicated first hardware path, and debug write requests are processed through a dedicated second hardware path. The two paths are separated in hardware structure and do not share transmission bandwidth or queues.

[0070] Next, combined Figure 3 The device hardware architecture of this disclosure is described in detail. Figure 3 A schematic diagram of a dual-path architecture for the final-level cache of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0071] exist Figure 3 The diagram shows the internal structure of the final-level cache module, which contains two independent hardware paths: a first hardware path (black background with white text) for processing business data and a second hardware path (white background with black text) for processing debugging data.

[0072] like Figure 3 As shown, the first hardware path includes, but is not limited to, the first port 301, the service data access queue 311, the hit detection unit 312, the cache array 302, the execution unit 313, the first request management unit 314 in the execution unit 313, etc. These components can work together to perform operation S201.

[0073] Optionally, the first port 301 is a hardware interface used to receive service requests. After a service request is issued from the processor core or L2 cache, it enters the final cache module through the first port 301 for further processing. The interface width and clock frequency of the first port 301 can be configured according to actual design requirements to meet the needs of high-bandwidth service data transmission.

[0074] Optionally, the business data access queue 311 is a first-in, first-out (FIFO) cache for storing business requests. After a business request enters through the first port 301, it is first queued in the business data access queue 311. The business data access queue 311 may contain multiple queue entries, each storing information related to a business request, including the request type (read or write), target address, data length, etc. Of course, this disclosure is not limited to this.

[0075] Optionally, the hit detection unit 312 is a hardware module used to determine whether a service request hits the cache. The hit detection unit 312 maintains the tag memory and status information of the cache array 302. When a service request arrives, the hit detection unit 312 compares the address carried by the service request with the entries in the tag memory to determine whether the required data is already in the cache array 302. The specific workflow includes the following steps: First, locate the set of candidate cache lines in the cache array based on the index field of the service request address; then, read the tag and validity bit information of the candidate cache lines; next, compare the tag field of the request address with the tag of the candidate cache lines, while checking whether the validity bit is set; finally, output a hit or miss signal based on the comparison result. Of course, this disclosure is not limited thereto.

[0076] Cache array 302 is a high-speed storage structure for storing data, consisting of multiple cache lines. Each cache line has a fixed storage capacity (e.g., 64 bytes) and includes a tag field, a valid bit field, and a dirty bit field. The tag field stores the high-order bits of the memory address corresponding to the cache line, used for address comparison; the valid bit field indicates whether the data currently stored in the cache line is valid; the dirty bit field indicates whether the data in the cache line has been modified but not yet written back to memory. The total capacity of cache array 302 can be 16MB, 32MB, or 64MB, depending on the design goals and cost constraints of the artificial intelligence processor. This disclosure is not limited to these values.

[0077] The execution unit 313 is a processing module for performing data read and write operations. When the hit detection unit 312 determines that a business request has been hit, the execution unit 313 directly reads or writes data from the cache array 302. When a miss occurs, the execution unit 313 first initiates a read request to the main storage area. After the data is returned and filled into the cache array 302, subsequent data processing operations are completed. The execution unit 313 may also include a write buffer for storing dirty cache lines to be written back to the main storage area, thereby reducing the blocking of write operations on business request processing. The execution unit 313 also includes a first request management unit 314 for allocating a first request number on the bus to the first hardware path, wherein the first request management unit 314 is configured with a first number of request number pools. Of course, this disclosure is not limited thereto.

[0078] More specifically, the first request management unit 314 is a hardware module for managing memory access requests of the first hardware path. The first request management unit 314 is configured with a request number pool containing a first quantity of numbers. The request number is used to uniquely identify an in-transit memory access request on the bus, in order to track the request status and receive responses. The first quantity can be 128 or 256, enabling the first hardware path to maintain a high-concurrency service request delivery capability. When a service request obtains a first request number, it enters the request queue to wait for delivery to the main memory area; after the main memory area returns a response, the first request management unit 314 releases the first request number so that the number can be reused by subsequent service requests. Of course, this disclosure is not limited to this.

[0079] Continue to refer to Figure 3 The second hardware path includes the following components: second port 303, debug data access queue 321, etc. These components can work together to perform operation S202.

[0080] Optionally, the second port 303 is a hardware interface for receiving debug write requests. After a debug write request is issued from the performance monitoring unit or the performance acquisition module inside the last-level cache, it enters the second hardware path for processing through the second port 303. The second port 303 is physically separate from the first port 301, each having its own independent interface signal and data path. Of course, this disclosure is not limited thereto.

[0081] Optionally, the debug data access queue 321 is a first-in, first-out (FIFO) buffer for storing debug write requests. After a debug write request enters through the second port 303, it is first queued in the debug data access queue 321. Similar to the business data access queue 311, the debug data access queue 321 also contains multiple queue entries, and its depth can be configured according to the expected traffic of debug data. Of course, this disclosure is not limited thereto.

[0082] Optionally, the address calculation unit 322 is a hardware module used to map debug data packet sequence numbers to memory addresses. The address calculation unit 322 internally maintains a base address register, storing the starting physical address of the second memory space in the main memory area. When a debug data packet sequence number is received, the address calculation unit 322 calculates the target memory address as the base address plus the product of the sequence number and the size of the debug data packet. Of course, this disclosure is not limited thereto.

[0083] Specifically, assuming that during initialization, the base address allocated to the second storage space by the last-level cache is 0x1FFF8000, and the size of the debug data packet is set to 64 bytes, when the performance monitoring unit generates debug data packet number 1024, this debug data packet carries sequence number 1024 into the second hardware path. After receiving sequence number 1024, the address calculation unit 322 performs the following calculation: the target memory address equals the base address plus the sequence number multiplied by the data packet size, that is, 0x1FFF8000 plus 1024 multiplied by 64 equals 0x1FFF8000 plus 0x10000 equals 0x20008000. If the size of the second storage space is 8MB (corresponding to a hexadecimal address range of 0x800000 bytes), then the number of debug data packets that can be stored is 8MB divided by 64 bytes, which equals 131072. Therefore, the valid range of the sequence number is 0 to 131071. When the sequence number exceeds 131071, the address calculation unit 322 needs to adopt a wrap-around strategy, taking the sequence number modulo 131072 before address mapping. For example, when the sequence number is 131072, the modulo calculation result is 0, and the corresponding target address is 0x1FFF8000 plus 0 multiplied by 64, which equals 0x1FFF8000, i.e., returning to the starting address of the second memory space; when the sequence number is 200000, the modulo calculation result is 200000 modulo 131072, which equals 68928, and the corresponding offset is 68928 x 64 = 4,411,392 (i.e., 0x435000). The target address is 0x1FFF8000 plus 0x435000, which equals 0x1FFF8000 + 0x435000 = 0x2042D000, which is still within the address range of the second memory space.

[0084] Optionally, the second request management unit 323 is a hardware module for managing memory access requests from the second hardware path. The second request management unit 323 is configured with a request number pool containing a second number of numbers. The second number can be 8 or 16 to match the characteristics of bursty but small overall volume of debug data, ensuring real-time writing of debug data while avoiding excessive bus resource consumption by the debug path. The second number is less than the first number.

[0085] To more clearly illustrate the impact of the asymmetric design of the first request management unit 314 and the second request management unit 323 on system performance, a specific scenario is used as an example. Assume that at a certain moment, the AI ​​processor is running a computationally intensive AI inference task, which requires the processor core to continuously perform matrix operations, resulting in very frequent data access. At this time, the first hardware path is processing a large number of concurrent business read requests; assume there are 80 business read requests being processed simultaneously. Since the first request management unit 314 is configured with 128 request numbers, all 80 requests can obtain an independent request number and be sent to the main storage area. Simultaneously, the performance monitoring unit detects a decreasing trend in cache hit rate and begins sending debug write requests to the second hardware path, recording performance metrics. Assume that 20 debug write requests are generated within 1 millisecond. Since the second request management unit 323 is only configured with 8 request numbers, the second hardware path will, according to the first-come, first-served principle, first assign request numbers to the 8 debug write requests and send them to the main storage area. The remaining 12 debug write requests wait in the debug data access queue 321. Once a debug write request is completed and its request number is released, the next debug write request in the queue will be able to obtain a request number. During this process, debug write requests generated by the second hardware path will not significantly affect the processing of business read requests on the first hardware path. The first request management unit 314 still has available request numbers, ensuring the concurrency capability of business requests.

[0086] Optionally, the debug request queue 324 is a hardware module used to implement cache bypass functionality. The debug request queue 324 routes debug write requests directly to the memory controller, bypassing the cache array 302, thus preventing debug data from occupying the storage space of the cache array 302. Specifically, after obtaining the target memory address and completing the allocation of the second request number, the relevant debug data is sent to the debug request queue 324, which forwards it to the second storage space via a bypass path, thereby bypassing the cache array 302.

[0087] After the request is processed, when the debug request queue 324 receives a write response for the debug data from the second storage space, the second request management unit will release the previously assigned second request number accordingly, thereby completing a complete debug data bypass write / response process.

[0088] It is important to note that no cache is configured in the second hardware path. This is a special design based on the characteristic that debug data is used only once. Debug data has only one read value from its generation to its analysis, unlike business data which may be repeatedly accessed by the processor core. Therefore, not configuring a cache in the second hardware path saves hardware resources and avoids debug data occupying cache space.

[0089] After the debug data is written to the second storage space, when an external source or processor initiates a read request for the debug data through the first port, the first port receives and parses the read request to obtain the memory address information corresponding to the target debug data. Subsequently, the read request is scheduled to the first hardware path, which directly accesses the second storage space, locates and reads the corresponding debug data based on the memory address. After reading is completed, the debug data is returned to the first port via the first hardware path, and finally responds to the read request, realizing the direct return of debug data from the second storage space to the request initiator. Of course, this disclosure is not limited thereto.

[0090] An arbitration unit (not shown) is a hardware module used to handle access conflicts between service requests and debug write requests. When both the first hardware path and the second hardware path simultaneously request access to the main memory area, the arbitration unit makes a decision based on a priority strategy. In this disclosure, the priority of service requests is set higher than that of debug write requests.

[0091] Let's illustrate the working logic of the arbitration unit using a specific scenario. Suppose that at a certain moment, the write port bandwidth of the main storage area is already saturated with five business write requests from the first hardware path. At this time, the second hardware path generates a debug write request that needs to be written to the main storage area. Since the write port bandwidth is full, the debug write request needs to wait. Simultaneously, a new business write request arrives at the first hardware path. At this point, the arbitration unit intervenes to make a decision. Because the business request has a higher priority, the arbitration unit will prioritize inserting the new business write request at the front of the waiting queue, while the debug write request continues to wait for available write port bandwidth. Furthermore, suppose that during the debug write request's waiting period, several more new business write requests arrive at the first hardware path. The arbitration unit will process these business write requests sequentially, before the debug write request, ensuring that the priority of the business requests is always guaranteed.

[0092] Next, combined Figure 4 The physical isolation design of the main memory (RAM) in this disclosure is described in detail.

[0093] Figure 4 A schematic diagram of the physical isolation partitioning of an electronic device and a main storage area according to an embodiment of the present disclosure is shown.

[0094] exist Figure 4In this system, the electronic device includes an artificial intelligence processor and multiple main memory areas 411, 412, and 41N. The artificial intelligence processor internally contains multiple last-level cache modules 421, 422, and 42N, each corresponding to an independent main memory area 411, 412, or 41N. This design allows each last-level cache module to independently manage access to its corresponding main memory area, avoiding the overhead of cache coherency protocols between multiple last-level caches.

[0095] Each main storage area is divided into two independent regions in terms of physical address space: a first storage space for storing business data and a second storage space for storing debugging data. Taking main storage area 41N as an example, its physical address space is divided as follows.

[0096] The total physical capacity of the main storage area 41N can be 8GB (e.g., address range from 0x00000000 to 0x20000000). The first storage spaces 411a and 411c occupy most of the capacity of the main storage area 411 and are used to store business data. The physical address range corresponding to the first storage space 411a is from a starting address to a certain boundary address, and its total capacity is approximately the total capacity of the main storage area minus the capacity of the second storage space. The addresses of the first storage spaces 411a and 411c can be non-contiguous or contiguous.

[0097] The second storage space 411b occupies a fixed capacity area of ​​the main storage area 411 and is specifically used for storing debugging data. The physical address range corresponding to the second storage space 411b is from a preset start address to an end address, and the total capacity is a preset size (e.g., 8MB). This isolation and division of the physical address space can be pre-configured by software during the initialization phase, and the hardware can directly route according to the address range without the need for complex consistency management at runtime.

[0098] Therefore, in one aspect of this disclosure, the storage areas for business data and debug data are physically and completely isolated in the main storage area. Even if the amount of debug data generated by the second hardware path is large, it will not encroach on the capacity of the first storage space 411a or 411c, ensuring that business data always has sufficient cache space available. This also avoids cache consistency issues between business data and debug data. Since their storage areas do not overlap, the processor core does not need to worry about debug data writes affecting the state of business data in the cache array when accessing business data, reducing the complexity of hardware design. In another aspect of this disclosure, this also simplifies debug data access management. After the software allocates a dedicated address space for debug data during the initialization phase, subsequent debug data writes directly use this address range without additional address translation or permission checks. Furthermore, developers can directly access the address range of the second storage space 411b to read debug data after the business program finishes running, without distinguishing data types or worrying about access conflicts, facilitating the reading and analysis of debug data.

[0099] It should be noted that setting the capacity of the second storage space to a preset size (e.g., 8MB) is a preferred embodiment. In practical applications, it can be adjusted according to the specific chip specifications and debugging requirements. For example, for chips with high debugging requirements, the capacity of the second storage space can be appropriately increased; for edge AI processors with limited memory resources, the capacity of the second storage space can be appropriately decreased. The selection of the capacity of the second storage space needs to comprehensively consider factors such as the accuracy requirements of debugging data acquisition, the chip's memory resource constraints, and system costs.

[0100] Next, combined Figure 5 The debugging read / write timing design of this disclosure is described in detail. Figure 5 A schematic diagram illustrating the parallel operation timing of business procedures and debugging functions according to an embodiment of this disclosure is shown on the timeline. Figure 5 In the diagram, the horizontal axis represents the increase in time, and the vertical axis represents different operation types.

[0101] The time period T0 to T1 represents the startup phase of the business application. During this phase, the processor core begins running the business application and generates business requests. The last-level cache receives these business requests and processes them through the first hardware path, while business data is read and written in the first storage space. In the early stages of business application startup, the number of business requests gradually increases, and the load on the last-level cache gradually rises to a stable state. During the T0 to T1 time period, debugging functionality may also be enabled. Developers may choose to enable hardware performance monitoring while running the business application. The performance monitoring unit begins collecting performance indicator data and sends debug write requests to the last-level cache through the second port. These debug write requests are written to the second storage space through the second hardware path. Because the second hardware path is independent of the first hardware path, the writing of debug data and the processing of business requests can occur simultaneously without interference.

[0102] The time period T1 to T2 represents the parallel operation phase of the business program and debugging functions. This can be a relatively long phase, with the business program running continuously and the performance monitoring function continuously collecting debugging data. During this phase, business requests and debugging write requests are generated and processed simultaneously. The arbitration unit ensures that business requests have priority access to the main memory area based on a priority strategy. For example, suppose that during the time period T1 to T2, the main memory area receives multiple concurrent business read requests from the first hardware path and multiple debugging write requests from the second hardware path. Since business requests have higher priority, the arbitration unit will schedule business read requests in an earlier time sequence, while debugging write requests will be transmitted in the intervals between business requests. These two paths work in parallel. Throughout the parallel operation of the business program and debugging functions, the first hardware path always maintains the ability to process business requests, and the performance of business requests is not affected by changes in the load of the debugging path. Time T2 represents the end point of the business program. At this time, the processor core stops executing the business program and no longer generates new business requests. The load on the first hardware path disappears, and its corresponding request number pool becomes idle.

[0103] The time period from T3 to T4 represents the debug read operation phase. After the business application finishes running, developers call the software interface to read the debug data that has been written to disk. The debug read request enters the last-level cache through the first or second port. Since the business application has finished at this time, the first hardware path is idle, and the debug read request can reuse the resources of the first hardware path to read debug data from the second storage space and return it to the software for analysis. During this period, the hardware resource reuse feature is utilized, eliminating the need to set up a separate third hardware path to read debug data, thus saving hardware design costs and chip area. Time T4 indicates that the debug operation is complete. Developers have obtained complete debug data and can perform performance analysis and optimization of the business application.

[0104] In one aspect of this disclosure, debug write requests occur in parallel with the business application, fully utilizing the hardware's parallel processing capabilities, ensuring the business application maintains normal performance throughout its execution. Debug read requests reuse the hardware resources of the business path, eliminating the need for a separate third hardware path and saving hardware design costs. Finally, debug read requests occur after the business application has finished, at which point reusing the first hardware path will not cause performance conflicts with the business requests.

[0105] This disclosure also provides an electronic device, including a main storage area and an artificial intelligence processor connected to the main storage area. The artificial intelligence processor includes a last-level cache, which includes the aforementioned data processing device. This electronic device can be an artificial intelligence server, an artificial intelligence accelerator card, or other computing device containing an artificial intelligence processor. Specifically, the electronic device can include, but is not limited to, the following types: AI accelerator cards for data centers; AI inference chips for edge computing scenarios; and NPU modules integrated into AI systems-on-a-chip in mobile devices.

[0106] This disclosure also provides a non-transitory computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the data processing method described above. The storage medium may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, flash memory, or other non-transitory storage medium suitable for storing computer instructions.

[0107] This disclosure also provides an electronic device, including a main storage area and an artificial intelligence processor connected to the main storage area. The artificial intelligence processor includes a last-level cache, which includes the data processing device 30 described above. This electronic device may be an artificial intelligence server, an artificial intelligence accelerator card, or other computing device containing an artificial intelligence processor.

[0108] This disclosure also provides a non-transitory computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the data processing method described above. The storage medium may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, flash memory, or other non-transitory storage medium suitable for storing computer instructions.

[0109] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art should understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure.

[0110] This disclosure also provides a non-transitory computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the data processing method described above. The storage medium may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, flash memory, or other non-transitory storage medium suitable for storing computer instructions.

[0111] The processor in this disclosure embodiment can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, operations, and logic block diagrams disclosed in this disclosure embodiment. The general-purpose processor can be a microprocessor or any conventional processor, and can be based on an x86 architecture or an ARM architecture.

[0112] In general, the various exemplary embodiments of this disclosure can be implemented in hardware, dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of the embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0113] This disclosure also provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a method according to an embodiment of this disclosure.

[0114] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] In general, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of the embodiments of this disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0116] The exemplary embodiments of this disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will understand that various modifications and combinations of these embodiments or their features can be made without departing from the principles and spirit of this disclosure, and such modifications should fall within the scope of this disclosure. For example, the following provides an overview of some aspects of this disclosure, which can be combined with any other aspects.

[0117] Aspect 1: A method for processing data is provided, the method being used to concurrently access business data and debugging data in the last-level cache of an artificial intelligence processor. The method includes: receiving a business request through a first port of the last-level cache, processing the business request through a first hardware path connected to the first port, and routing the business data in the business request to a first storage space in the main storage area; receiving a debug write request through a second port of the last-level cache, processing the debug write request through a second hardware path connected to the second port, and routing the debug data in the debug write request to a second storage space in the main storage area; wherein the second hardware path is independent of the first hardware path, and the second storage space and the first storage space correspond to different physical address ranges in the main storage area.

[0118] Aspect 2: The process of the service request through the first hardware path connected to the first port includes: storing the service request in the service data access queue on the first hardware path; determining whether the service data corresponding to the service request is hit in the cache array through the hit detection unit on the first hardware path; in response to the data corresponding to the service request being hit in the cache array, reading or writing data in the cache line of the cache array through the execution unit on the first hardware path; in response to the data corresponding to the service request not being hit in the cache array, retrieving the corresponding cache line from the main storage area through the execution unit.

[0119] Aspect 3: The process of the debug write request through the second hardware path connected to the second port includes: storing the debug write request in the debug data access queue on the second hardware path; mapping the debug data packet sequence number in the debug write request to the target memory address in the second storage space through the address calculation unit on the second hardware path; and sending the debug data to the second storage space through the bypass of the cache array of the last-level cache based on the target memory address.

[0120] Aspect 4: The method further includes: allocating a first request number on the bus to the first hardware path through a first request management unit in the execution unit, wherein the first request management unit is configured with a first number of request number pools; and allocating a second request number on the bus to the second hardware path through a second request management unit on the second hardware path, wherein the second request management unit is configured with a second number of request number pools, and the first number is greater than the second number.

[0121] Aspect 5: The step of sending the debug data to the second storage space via the bypass of the cache array of the last-level cache includes: in response to obtaining the target memory address and allocating the second request number, sending the debug data into the debug request queue to send it to the second storage space through the debug request queue; in response to the debug request queue receiving a write response for the debug data returned by the second storage space, releasing the second request number in the second request management unit.

[0122] Aspect Six: The method further includes: in response to an access conflict between the service request and the debug write request to the main storage area, the service request is preferentially sent to the main storage area through an arbitration unit.

[0123] Aspect 7: The method further includes: receiving a read request for the debug data through the first port; reading the corresponding debug data from the second storage space through the first hardware path and returning it.

[0124] Aspect 8: The second storage space is a storage space with a preset capacity in the main storage area.

[0125] Aspect 9: During the execution of the business program by the artificial intelligence processor, business requests and debug write requests are received in parallel.

[0126] Aspect 10: A data processing apparatus is provided for concurrently accessing business data and debugging data in the final cache of an artificial intelligence processor. The apparatus includes: a first port configured to receive business requests; a first hardware path connected to the first port and configured to route business data in the business request to a first storage space in the main storage area; a second port configured to receive debug write requests; and a second hardware path connected to the second port and independent of the first hardware path, configured to route debug data in the debug write request to a second storage space in the main storage area; wherein the second storage space and the first storage space correspond to different physical address ranges in the main storage area.

[0127] Aspect 11: The first hardware path includes: a business data access queue, configured to store the business request; a hit detection unit, configured to determine whether the data corresponding to the business request is hit in the cache array; and an execution unit, configured to: read or write data in the cache line of the cache array in response to the data corresponding to the business request being hit in the cache array; and retrieve the corresponding cache line from the main storage area in response to the data corresponding to the business request not being hit in the cache array.

[0128] Aspect 12: The second hardware path includes: a debug data access queue, configured to store the debug write request; an address calculation unit, configured to map the debug data packet sequence number in the debug write request to a target memory address in the second storage space; wherein, based on the target memory address, the debug data is sent to the second storage space through a bypass of the cache array of the last-level cache.

[0129] Aspect 13: The execution unit further includes: a first request management unit, the first request management unit being configured to: allocate a first request number on the bus to the first hardware path, wherein the first request management unit is configured with a first number of request number pools; the second hardware path further includes: a second request management unit, the second request management unit being configured to: allocate a second request number on the bus to the second hardware path, wherein the second request management unit is configured with a second number of request number pools, the first number being greater than the second number.

[0130] Aspect 14: The second hardware path includes a debug request queue, configured to: in response to obtaining the target memory address and allocating the second request number, receive the debug write request and forward the debug write request to the second storage space; in response to receiving a write response for the debug data returned by the second storage space, request the release of the second request number in the second request management unit.

[0131] Aspect 15: The apparatus further includes: an arbitration unit configured to, in response to an access conflict between the service request and the debug write request to the main storage area, prioritize sending the service request to the main storage area.

[0132] Aspect 16: The first port is also configured to receive read requests for the debug data, and the first hardware path is also configured to read the corresponding debug data from the second storage space and return it.

[0133] Aspect 17: The second storage space is a storage space with a preset capacity in the main storage area.

[0134] Aspect 18: The first port receiving service requests and the second port receiving debug write requests are configured to occur in parallel during the execution of the service program by the artificial intelligence processor.

[0135] Aspect 19: An electronic device is provided, comprising: a main storage area; and an artificial intelligence processor connected to the main storage area, the artificial intelligence processor including a last-level cache, the last-level cache including the aforementioned means.

[0136] Aspect 20: A non-transient computer-readable storage medium is provided, on which computer instructions are stored, which, when executed by a processor, implement the above method.

Claims

1. A method for processing data, the method being used for concurrently accessing business data and debugging data in the final cache of an artificial intelligence processor, characterized in that, The method includes: The service request is received through the first port of the last-level cache, and the service request is processed through the first hardware path connected to the first port, and the service data in the service request is routed to the first storage space of the main storage area. The debug write request is received through the second port of the last-level cache, and the debug write request is processed through the second hardware path connected to the second port, and the debug data in the debug write request is routed to the second storage space of the main storage area. The second hardware path is independent of the first hardware path, and the second storage space and the first storage space correspond to different physical address ranges in the main storage area.

2. The method as described in claim 1, characterized in that, The process of processing the service request through the first hardware path connected to the first port includes: The service request is stored in the service data access queue on the first hardware path; The hit detection unit on the first hardware path determines whether the business data corresponding to the business request is hit in the cache array. When the data corresponding to the service request is hit in the cache array, the execution unit on the first hardware path reads or writes the data in the cache line of the cache array. If the data corresponding to the service request is not found in the cache array, the corresponding cache line is retrieved from the main storage area by the execution unit.

3. The method as described in claim 2, characterized in that, The process of processing the debug write request through the second hardware path connected to the second port includes: The debug write request is stored in the debug data access queue on the second hardware path; The address calculation unit on the second hardware path maps the debug data packet sequence number in the debug write request to the target memory address in the second storage space. Based on the target memory address, the debugging data is sent to the second storage space via a bypass of the cache array of the last-level cache.

4. The method as described in claim 3, characterized in that, The method further includes: The first request management unit in the execution unit assigns a first request number to the first hardware path on the bus, wherein the first request management unit is configured with a first number of request number pools; A second request number is assigned to the second hardware path on the bus by a second request management unit on the second hardware path, wherein the second request management unit is configured with a second number of request number pools, the first number being greater than the second number.

5. The method as described in claim 4, characterized in that, The step of sending the debugging data to the second storage space via the bypass of the cache array of the last-level cache includes: In response to obtaining the target memory address and allocating the second request number, the debugging data is sent to the debugging request queue so as to be sent to the second storage space through the debugging request queue; In response to the debug request queue receiving a write response for the debug data returned by the second storage space, the second request number is released in the second request management unit.

6. The method as described in claim 1, characterized in that, The method further includes: In response to a conflict between the service request and the debug write request in accessing the main storage area, the arbitration unit prioritizes sending the service request to the main storage area.

7. The method as described in claim 1, characterized in that, The method further includes: Receive read requests for the debug data through the first port; The corresponding debugging data is read from the second storage space through the first hardware path and returned.

8. The method as described in claim 1, characterized in that, The second storage space is a storage space with a preset capacity in the main storage area.

9. The method as described in claim 1, characterized in that, During the execution of the business program by the artificial intelligence processor, business requests and debug write requests are received in parallel.

10. A data processing apparatus, the apparatus being used for concurrently accessing business data and debugging data in the final cache of an artificial intelligence processor, characterized in that, The device includes: The first port is configured to receive service requests; A first hardware path, connected to the first port, is configured to route the service data in the service request to the first storage space of the main storage area; The second port is configured to receive debug write requests; The second hardware path, connected to the second port and independent of the first hardware path, is configured to route debug data in the debug write request to the second storage space of the main storage area. The second storage space and the first storage space correspond to different physical address ranges in the main storage area.

11. The apparatus as claimed in claim 10, characterized in that, The first hardware path includes: The business data access queue is configured to store the business request. The hit detection unit is configured to: determine whether the data corresponding to the service request is hit in the cache array; The execution unit is configured as follows: When the data corresponding to the service request is hit in the cache array, the data is read or written to the cache line of the cache array. If the data corresponding to the service request is not found in the cache array, the corresponding cache line is retrieved from the main storage area.

12. The apparatus as claimed in claim 11, characterized in that, The second hardware path includes: The debug data memory access queue is configured to store the debug write requests; The address calculation unit is configured to map the debug data packet sequence number in the debug write request to the target memory address in the second storage space. Specifically, based on the target memory address, the debugging data is bypassed and sent to the second storage space through the cache array of the last-level cache.

13. The apparatus as claimed in claim 12, characterized in that, The execution unit further includes: a first request management unit, the first request management unit being configured to: allocate a first request number on the bus for the first hardware path, wherein the first request management unit is configured with a first number of request number pools; The second hardware path further includes a second request management unit, which is configured to allocate a second request number on the bus for the second hardware path, wherein the second request management unit is configured with a second pool of request numbers, the first pool being greater than the second pool.

14. The apparatus as claimed in claim 13, characterized in that, The second hardware pathway includes a debug request queue, configured as follows: In response to obtaining the target memory address and allocating the second request number, the debug write request is received and forwarded to the second storage space; In response to receiving a write response for the debug data returned by the second storage space, a request is made to release the second request number in the second request management unit.

15. The apparatus as claimed in claim 10, characterized in that, The device further includes: An arbitration unit is configured to prioritize sending the service request to the main storage area in response to an access conflict between the service request and the debug write request.

16. The apparatus as claimed in claim 10, characterized in that, The first port is also configured to receive read requests for the debug data, and the first hardware path is also configured to read the corresponding debug data from the second storage space and return it.

17. The apparatus as claimed in claim 10, characterized in that, The second storage space is a storage space with a preset capacity in the main storage area.

18. The apparatus as claimed in claim 10, characterized in that, Receiving service requests via the first port and receiving debug write requests via the second port are configured to occur in parallel during the execution of the service program by the AI ​​processor.

19. An electronic device, characterized in that, The electronic device includes: Main storage area; and An artificial intelligence processor, connected to the main memory area, the artificial intelligence processor including a last-level cache, the last-level cache including the means as described in any one of claims 10 to 18.

20. A non-transient computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the method described in any one of claims 1 to 9 is implemented.

Images