An instruction fetch apparatus, processor, board card and instruction fetch method

Abstract

The present disclosure discloses an instruction fetching device, a processor, an integrated circuit chip, a board card, an electronic device and a method for fetching instructions. In one embodiment, the aforementioned instruction fetching device can be included in a computing device, and the computing device can be included in a combined processing device, which can further include a general-purpose interconnection interface and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the device and the other processing devices, respectively, for storing data of the device and the other processing devices. The scheme of the present disclosure can improve the instruction fetching efficiency in various computing fields, including, for example, the field of artificial intelligence, thereby accelerating the operation and reducing the overall overhead and cost of the operation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of computing. More specifically, the present disclosure relates to an instruction fetch apparatus, a processor, an integrated circuit chip, a board card, an electronic device, and a method for fetching instructions. BACKGROUND

[0002] A conventional central processing unit ("CPU") needs to refill instructions from a dynamic random access memory ("DRAM") to a cache after a cache miss event occurs, before it can continue to fetch instructions from the cache for operations such as decoding. In some scenarios, in order to improve the efficiency of fetching instructions (referred to as "instruction fetch" hereinafter), the CPU often performs a hardware prefetch operation when refilling, i.e., in addition to refilling the instruction that is currently missed (referred to as "missed instruction" hereinafter), it also refills a number of instructions following the missed instruction to the cache, which will affect the efficiency of instruction fetch. SUMMARY

[0003] In order to at least solve the problems existing in the prior art, the present disclosure provides a scheme for efficiently fetching instructions. By using the scheme disclosed in the present disclosure, the received instructions can be sent out (e.g., sent to a decoder) during the refill operation of the missed instructions, thereby significantly improving the efficiency of instruction fetch. Due to the improvement of instruction fetch efficiency, the scheme of the present disclosure can also obtain technical advantages in enhancing the processing performance of hardware such as a processor, reducing power consumption, and improving the execution efficiency of computing operations, etc.

[0004] In a first aspect, the present disclosure provides an instruction fetch apparatus, comprising: a cache configured to cache a plurality of instructions fetched from an external memory; an instruction fetch circuit configured to fetch a plurality of instructions to be sent out from the cache; a refill request circuit configured to: detect whether a cache miss event occurs during the instruction fetch apparatus sending out instructions; and in response to detecting that the cache miss event occurs, send a refill request to the external memory; the instruction fetch circuit is further configured to: receive a plurality of refill instructions as a response to the refill request from the external memory in batches; and send the received refill instructions out during the interval of receiving the refill instructions in batches.

[0005] In a second aspect, the present disclosure provides a processor, comprising: an instruction fetch apparatus as described above and to be described in a plurality of embodiments below; a decoder configured to receive instructions from the instruction fetch apparatus and decode them to obtain a plurality of microinstructions and / or control signals; and a processing circuit configured to perform operations according to the plurality of microinstructions and / or control signals.

[0006] In a third aspect, the present disclosure provides an integrated circuit chip comprising the instruction fetch apparatus or the processor as described above and to be described in the following embodiments.

[0007] In a fourth aspect, the present disclosure provides a board card comprising the integrated circuit chip as described above and to be described in the following embodiments.

[0008] In a fifth aspect, the present disclosure provides an electronic device comprising the integrated circuit chip as described above and to be described in the following embodiments.

[0009] In a sixth aspect, the present disclosure provides a method for fetching instructions, the method comprising: detecting whether a cache miss event occurs during fetching a plurality of instructions to be sent out from a buffer; sending a backfill request to an external memory in response to detecting that the cache miss event occurs; receiving backfill instructions from the external memory in batches as a response to the backfill request; and sending out the received backfill instructions during intervals of receiving the backfill instructions in batches.

[0010] By utilizing the instruction fetch apparatus, the integrated circuit chip, the board card, the electronic device and the method of the present disclosure, the received backfill instructions can be sent out during intervals of backfill operation, thereby significantly improving the efficiency of instruction fetching. Due to the improvement of instruction fetching efficiency, the scheme of the present disclosure further improves the operation speed of operations related to instructions, thereby improving the overall performance of hardware and reducing the computational overhead. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the disclosed example embodiments will become more apparent from the following detailed description read in conjunction with the accompanying drawings. In the drawings, several embodiments of the disclosure are shown exemplarily but not limitatively, and identical or corresponding reference numbers indicate identical or corresponding parts thereof:

[0012] Figure 1 is a block diagram exemplarily showing an instruction fetch apparatus according to an embodiment of the present disclosure;

[0013] Figure 2 is a block diagram exemplarily showing a processor according to an embodiment of the present disclosure;

[0014] Figure 3 is a flowchart exemplarily showing a method for fetching instructions according to an embodiment of the present disclosure;

[0015] Figure 4 is a structural diagram of a combined processing apparatus according to an embodiment of the present disclosure; and

[0016] Figure 5 is a structural diagram of a board card according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] The present disclosure provides a hardware architecture to accelerate instruction fetch. When implemented in an instruction fetch apparatus, the instruction fetch apparatus comprises a refill request circuit configured to detect whether a cache miss event occurs during the instruction fetch apparatus sending out instructions to an external memory, and send a refill request to the external memory when the cache miss event occurs. In one embodiment, the instruction fetch apparatus of the present disclosure further comprises an instruction fetch circuit configured to receive batched refill instructions from the external memory as a response to the refill request, and send out the received refill instructions to an external entity, such as a decoder, during the interval of batched receiving. With the instruction fetch architecture of the present disclosure, the instruction fetch operation can be performed efficiently, thereby accelerating the execution of instructions. When the instruction fetch apparatus of the present disclosure is applied in a computing apparatus, the processing performance and computing efficiency of the computing apparatus can be significantly improved, in turn reducing the computing overhead.

[0018] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of, rather than all of, the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0019] Figure 1 is an exemplary block diagram illustrating an instruction fetch apparatus 100 according to an embodiment of the present disclosure. For ease of understanding, Figure 1 The external memory 104 interacting with the instruction fetch apparatus 100 is also shown in

[0020] As shown in Figure 1 The instruction fetch apparatus 100 of the present disclosure can comprise a cache 102 configured to cache a plurality of instructions fetched from the external memory 104 through a bus 106, as shown in

[0021] In the process of sending instructions out to the outside, such as to a decoder, the buffer can deliver to the instruction fetch circuit 108 a plurality of instructions to be sent out, such as the instructions 110, 112 and 114 shown in the figure and the instructions to be filled back, i.e. the fill-back instructions 116, 118 and 120 in the present disclosure, between them. In an embodiment, the instruction fetch circuit can be configured to queue the plurality of instructions to be sent out in a sending order to form an instruction fetch pipeline to send out in sequence. When it is determined that the instructions required for a calculation operation or code execution are not cached in the buffer, a cache miss event in the present disclosure occurs. The occurrence of the event means that the required instruction or instructions are not present in the current buffer and instruction fetch pipeline, for example Figure 1 The missing and to-be-filled-back fill-back instructions 116, 118 and 120 are shown in a virtual box in the figure. It needs to be understood that Figure 1 Three fill-back instructions are shown in the figure only by way of example. In different scenarios, according to the frequency of the occurrence of the cache miss event, more fill-back instructions can be required to be filled back from the external memory, and the order of the fill-back instructions can not be the adjacent arrangement with other instructions as shown in the figure, but can appear at any position in the instruction fetch pipeline. Figure 1 The three fill-back instructions are shown in the figure only by way of example. In different scenarios, according to the frequency of the occurrence of the cache miss event, more fill-back instructions can be required to be filled back from the external memory, and the order of the fill-back instructions can not be the adjacent arrangement with other instructions as shown in the figure, but can appear at any position in the instruction fetch pipeline.

[0022] In order to determine whether the cache miss event as described above occurs, the instruction fetch apparatus of the present disclosure can further include a fill-back request circuit 122, which can be configured to detect whether a cache miss event occurs during the instruction fetch apparatus sends out instructions. When it is detected that a cache miss event occurs, for example, the instruction fetch pipeline does not have the related instructions because the required instructions are not cached in the buffer, the fill-back request circuit can send a fill-back request for the missing instructions, i.e. the fill-back instructions, to the external memory. In response to receiving the fill-back request, the external memory, such as a DRAM, performs a fill-back operation for the buffer, i.e. fills back the fill-back instructions that trigger the cache miss event into the buffer.

[0023] In actual application scenarios, because the number of fill-back requests is often large, the fill-back instructions returned by the external memory need to be transmitted in batches. In view of this situation, the present disclosure proposes that while filling back the fill-back instructions that trigger the cache miss event into the buffer, the instruction fetch circuit also fills them back to a suitable position in the instruction fetch pipeline, such as the positions indicated by the virtual boxes 116, 118 and 120 in the figure, for queuing and sending out. Because the external memory performs the transmission of the fill-back instructions in batches, the instruction fetch circuit of the present disclosure can be configured to send out the fill-back instructions that have been received during the interval of receiving the fill-back instructions in batches. In this way, the efficiency of the instruction fetch operation of the present disclosure is significantly improved, thereby speeding up the instruction fetch and subsequent instruction execution operations. Figure 1 ​

[0024] In one embodiment, upon detecting the occurrence of the cache miss event, the buffer is configured to suspend the delivery of the instructions to be sent out to the instruction fetch circuit, and to receive the refill instructions from the external memory in batches. In one scenario, after the instruction fetch circuit sends out the received refill instructions, the buffer can be configured to receive the next batch of refill instructions from the external memory. In another embodiment, the instruction fetch apparatus of the present disclosure further comprises a buffer 124, which can be configured to receive the refill instructions from the external memory in batches, and to refill the received refill instructions into the buffer after the instruction fetch circuit finishes sending out the received refill instructions in batches. In this way, it can be ensured that the instruction fetch circuit sends out all the refill instructions in the instruction fetch pipeline before performing the refill operation to the buffer, so as to also reduce the frequent instruction access operations between the buffer and the external memory.

[0025] In one embodiment, the refill request circuit can be further configured to send only one refill request to the external memory for the same refill instruction in response to detecting the occurrence of multiple cache miss events triggered by the same refill instruction. According to the scheme of the present disclosure, since the instruction fetch circuit does not wait until all the refill instructions are refilled into the buffer before performing the operation of sending out the instructions through the instruction fetch pipeline, it is possible that the refill request circuit detects multiple cache miss events triggered by the same refill instruction. In order to avoid sending the refill request for the same refill instruction to the external memory repeatedly, the refill request circuit of the present disclosure will send only one refill request to the external memory for the same refill instruction. In this way, the number of refill requests and the communication overhead between the refill request circuit and the external memory can be reduced. In contrast, when a new cache miss event triggered by a new refill instruction is detected during the interval of receiving the refill instructions in batches, the refill request circuit can be configured to send a new refill request to the external memory for the new refill instruction.

[0026] The above description is combined with Figure 1 The instruction fetch apparatus of the present disclosure and its embodiments are described in detail. By using the instruction fetch apparatus of the present disclosure, the instruction fetch operation can be accelerated, so as to achieve efficient instruction fetching. Further, when a cache miss event occurs, the instruction fetch apparatus of the present disclosure can also reduce unnecessary refill requests, and save the I / O throughput for instruction delivery. When the instruction fetch apparatus of the present disclosure sends instructions to the decoder, it is also helpful for the efficient decoding of the decoder, so as to improve the efficiency of instruction execution.

[0027] Figure 2 is a block diagram of a processor 200 according to an embodiment of the present disclosure. As shown in Figure 2 The processor 200 comprises the above-mentionedFigure 1 The described instruction fetch apparatus 100 operates to fetch the buffered instructions from the buffer and to obtain the refill instructions from the external memory when a buffer miss event occurs, and to continue sending out instructions through the internal instruction fetch pipeline during the refill interval. As previously described in conjunction with Figure 1 The instruction fetch apparatus 100 is described in detail, and will not be repeated here.

[0028] In one embodiment, the processor of the present disclosure further comprises a decoder 204 configured to receive the instructions from the instruction fetch apparatus and to decode the instructions to obtain a plurality of microinstructions and / or control signals, for example, which can be executed internally within the processor or processing circuit. In the context of the present disclosure, the aforementioned plurality of microinstructions and / or control signals can also be collectively referred to as operation instructions, which can include (or indicate) one or more operation operations to be performed by the processor. Depending on different operation scenarios, the operation operations can include but are not limited to addition operations, multiplication operations, convolution operation operations, pooling operations, and various operations. For the aforementioned various operations, the processor of the present disclosure further comprises a processing circuit 206 for performing various operations, which in one or more embodiments can comprise one or more processing sub-circuits. In the case of multiple processing sub-circuits, these processing sub-circuits can be connected in a regular structure, for example, in the form of a multi-dimensional array, to perform operations such as parallel operations or multi-stage pipelined operations. Depending on different implementations and computing scenarios, the processing circuit can include but is not limited to random number processing circuit, addition and subtraction circuit, subtraction circuit, lookup table circuit, parameter configuration circuit, multiplier, pooler, comparator, absolute value circuit, logic operator, position index circuit, or filter, and various operators or operation circuits.

[0029] The above-described instruction fetch apparatus 100 is described in conjunction with Figure 2 The processor of the present disclosure comprising the instruction fetch apparatus is described, but it should be noted that the description here is only exemplary and not limiting, and those skilled in the art can also think of various modifications or alternatives according to the teachings herein. For example, although the instruction fetch apparatus, the decoder and the processing circuit are described as three separate entities in the processor, depending on the implementation scenario, the instruction fetch apparatus and the decoder can also be combined or arranged in the control circuit which controls the processing circuit. For example, in some scenarios, the decoder can be arranged in the processing circuit as an entity or unit of decoding function, so that the processing circuit also supports the decoding function of the fetched instructions.

[0030] Figure 3 is a flowchart showing a method 300 for fetching instructions according to an embodiment of the present disclosure. As previously described in conjunction with Figure 1 and Figure 2 described by those skilled in the art can be understoodFigure 3 The method 300 shown can be combined Figure 1 Instruction extraction device 100 or Figure 2 The processor 200 implements

[0031] like Figure 3 As shown, in step 302, during the process of retrieving multiple instructions to be sent from the cache, method 300 checks whether a cache miss event has occurred. As previously mentioned, the occurrence of this cache miss event means that the cache does not have the required fetched instructions cached, and a fill operation needs to be requested from external memory, such as DRAM. Therefore, in step 304, in response to detecting the occurrence of the cache miss event, method 300 sends a fill request to external memory, for example... Figure 1 The diagram illustrates the use of a backfill request circuit to send a backfill request. Next, at step 306, method 300 receives backfill instructions in batches from the external memory as responses to the backfill request. Finally, at step 308, method 300 sends the received backfill instructions outwards during the intervals between the batch receptions of the backfill instructions. As previously mentioned, this sending can be achieved via... Figure 1 The instruction fetch circuit shown utilizes a sequentially queued instruction fetch pipeline to execute instruction sending operations. Furthermore, this sending can be directed to… Figure 2 The decoder shown performs a transmission operation so that the decoder can decode or parse the instruction to obtain, for example, the microinstructions and / or control signals required to perform arithmetic operations.

[0032] In one embodiment, in response to the detection of multiple cache miss events triggered by the same backfill instruction, the method 300 sends a backfill request to the external memory only once for the same backfill instruction. In another embodiment, in response to the detection of a new cache miss event triggered by a new backfill instruction during the interval of receiving backfill instructions in batches, the method 300 sends a new backfill request to the external memory for the new backfill instruction.

[0033] For the sake of brevity, the above combines... Figure 3 The steps shown describe the method for extracting instructions disclosed herein. Those skilled in the art, based on the disclosure, will also realize that this method may include further steps, and that the execution of these steps can achieve the combined effect described above. Figure 1 and Figure 2 The various operations described in this disclosure will not be elaborated upon here.

[0034] Figure 4is a structural diagram illustrating a combined processing apparatus 400 according to embodiments of the present disclosure. As shown, the combined processing apparatus 400 includes a computing apparatus 402, which can include the instruction extraction apparatus or processor described in the present disclosure in connection with the accompanying drawings. In one or more embodiments, the computing apparatus can also be implemented as an integrated circuit chip, board, or electronic device including the instruction extraction apparatus of the present disclosure. In addition, the combined processing apparatus also includes a general interconnect interface 404 and other processing apparatus 406. The computing apparatus 402 of the present disclosure can interact with the other processing apparatus 406 through the general interconnect interface 404 to jointly complete user-specified operations, including, for example, the instruction extraction operations, subsequent decoding and / or execution operations of the present disclosure.

[0035] According to the present disclosure, the other processing apparatus can include one or more types of processors, such as central processing units (“CPUs”), graphics processing units (“GPUs”), artificial intelligence processors, and other general and / or special-purpose processors, the number of which can not be limited but determined according to actual needs. In one or more embodiments, the other processing apparatus can serve as an interface for the computing apparatus of the present disclosure to external data and controls, performing basic controls including but not limited to data transfer, starting, stopping, and the like of the computing apparatus; the other processing apparatus can also cooperate with the computing apparatus to jointly complete computational tasks. In one implementation scenario, the computing apparatus according to the present disclosure can be implemented as a machine learning computing apparatus.

[0036] According to the present disclosure, the general interconnect interface can be used to transmit data and control instructions, such as the computing instructions of the present disclosure, between the computing apparatus and the other processing apparatus. For example, the computing apparatus can obtain the required input data from the other processing apparatus via the general interconnect interface, and write to the on-chip storage apparatus of the computing apparatus, such as the external memory of the present disclosure, e.g., DRAM. Further, the computing apparatus can obtain control instructions from the other processing apparatus via the general interconnect interface, and write to the on-chip control cache of the computing apparatus, such as the cache of the present disclosure. Alternatively or optionally, the general interconnect interface can also read data from the storage apparatus of the computing apparatus and transmit it to the other processing apparatus.

[0037] Optionally, the combined processing apparatus can also include storage apparatus 408, which can be connected to the computing apparatus and the other processing apparatus, respectively. In one or more embodiments, the storage apparatus can be used to save data of the computing apparatus and the other processing apparatus, especially those that cannot be saved in the internal or on-chip storage apparatus of the computing apparatus or the other processing apparatus.

[0038] According to different application scenarios, the combined processing device of the present disclosure can be used as a SOC (System on Chip) of a mobile phone, a robot, a drone, a video monitoring device, and the like, effectively reducing the core area of the control part, improving the processing speed, and reducing the overall power consumption. In this case, the general interconnection interface of the combined processing device is connected to some components of the device. Some components are, for example, a camera, a display, a mouse, a keyboard, a network card, or a WiFi interface.

[0039] In some embodiments, the present disclosure also discloses a chip or an integrated circuit chip comprising the computing device or the combined processing device described above. In other embodiments, the present disclosure also discloses a chip packaging structure comprising the chip described above.

[0040] In some embodiments, the present disclosure also discloses a board card comprising the chip packaging structure described above. Referring to Figure 5 which provides the exemplary board card described above, in addition to comprising the chip 502, the board card can also comprise other matching components, including but not limited to: a memory device 504, an interface device 506, and a control device 508.

[0041] The memory device is connected to the chip in the chip packaging structure through a bus for storing data. The memory device can comprise a plurality of groups of storage units 510. Each group of storage units is connected to the chip through a bus. It can be understood that each group of storage units can be a DDR SDRAM (“Double Data Rate SDRAM”).

[0042] DDR does not need to increase the clock frequency to double the speed of SDRAM. DDR allows reading data on the rising and falling edges of the clock pulse. The speed of DDR is twice that of standard SDRAM. In one embodiment, the memory device can comprise four groups of storage units. Each group of storage units can comprise a plurality of DDR4 particles (chips). In one embodiment, the chip can internally comprise four 72-bit DDR4 controllers, of which 64 bits are used for data transmission and 8 bits are used for ECC verification.

[0043] In one embodiment, each group of storage units comprises a plurality of double-rate synchronous dynamic random access memories arranged in parallel. DDR can transmit data twice in one clock cycle. A controller for controlling DDR is arranged in the chip for controlling the data transmission and data storage of each storage unit.

[0044] The interface device is electrically connected with the chip in the chip package structure. The interface device is used to realize data transmission between the chip and an external device 512 (such as a server or a computer). For example, in an embodiment, the interface device can be a standard PCIE interface. For example, the data to be processed is transmitted to the chip by the server through the standard PCIE interface, so as to realize data transfer. In another embodiment, the interface device can also be other interfaces, and the disclosure does not limit the specific forms of the above-mentioned other interfaces. The interface unit can only realize the switching function. In addition, the calculation result of the chip is still transmitted back to the external device (such as a server) by the interface device.

[0045] The control device is electrically connected with the chip. The control device is used to monitor the state of the chip. Specifically, the chip and the control device can be electrically connected through an SPI interface. The control device can include a micro controller unit (MCU). In one or more embodiments, the chip can include multiple processing chips, multiple processing cores or multiple processing circuits, and can drive multiple loads. Therefore, the chip can be in different working states such as multiple loads and light loads. Through the control device, the working states of the multiple processing chips, the multiple processing and / or the multiple processing circuits in the chip can be regulated.

[0046] In some embodiments, the disclosure also discloses an electronic device or apparatus including the above-mentioned board card. According to different application scenarios, the electronic device or apparatus can include a data processing apparatus, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a vehicle record instrument, a navigation instrument, a sensor, a camera, a server, a cloud server, a camera, a video camera, a projector, a watch, a headset, a mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device. The vehicle includes an airplane, a ship and / or a vehicle; the household appliance includes a television, an air conditioner, a microwave oven, a refrigerator, an electric rice cooker, a humidifier, a washing machine, an electric lamp, a gas stove, an oil smoke exhauster; the medical device includes a nuclear magnetic resonance instrument, a B-ultrasound instrument and / or an electrocardiograph.

[0047] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the disclosure is not limited by the order of the described actions, because according to the disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily necessary for the disclosure.

[0048] In the above-described embodiments, the description of each embodiment focuses on different aspects, and parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0049] In several embodiments provided in the present disclosure, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. For example, the division of the units is merely a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, optical, acoustic or other forms.

[0050] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0051] In addition, each functional unit in each embodiment of the present disclosure can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software program module.

[0052] When the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, when the technical solutions of the present disclosure can be embodied in the form of a software product, the computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0053] In the foregoing embodiments of the present disclosure, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments. The technical features of the foregoing embodiments can be combined in any manner. In order to make the description concise, not all possible combinations of the technical features in the foregoing embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, it should be considered as the scope of the present disclosure.

[0054] The foregoing can be better understood in light of the following description.

[0055] An instruction fetch apparatus comprising:

[0056] a cache configured to cache a plurality of instructions fetched from an external memory;

[0057] a fetch circuit configured to fetch a plurality of instructions to be sent out from the cache;

[0058] a refill request circuit configured to:

[0059] detect whether a cache miss event occurs during the instruction fetch apparatus sending out instructions; and

[0060] in response to detecting that the cache miss event occurs, send a refill request to the external memory;

[0061] the fetch circuit is further configured to:

[0062] receive, from the external memory, refill instructions as a response to the refill request in batches; and

[0063] send out the received refill instructions during an interval of receiving the refill instructions in batches.

[0064] In some embodiments, the fetch circuit is further configured to:

[0065] queue the plurality of instructions to be sent out in a sending order to form a fetch pipeline; and

[0066] queue the received refill instructions in a corresponding order for sending out through the fetch pipeline.

[0067] In some embodiments, when detecting that the cache miss event occurs, the cache is configured to:

[0068] suspend delivery of instructions to be sent out to the fetch circuit; and

[0069] receive the refill instructions from the external memory in batches.

[0070] In some embodiments, the instruction fetch circuit is configured to send out the received refill instructions during the interval when the buffer receives the refill instructions in batches, and the buffer is configured to receive the next batch of refill instructions from the external memory after the instruction fetch circuit sends out the received refill instructions.

[0071] In some embodiments, the instruction fetch apparatus further comprises a buffer configured to:

[0072] receive the refill instructions from the external memory in batches; and

[0073] refill the received refill instructions into the buffer after the instruction fetch circuit finishes sending out the received refill instructions in batches.

[0074] In some embodiments, the refill request circuit is further configured to:

[0075] in response to detecting that multiple cache miss events triggered by the same refill instruction occur, send only one refill request for the same refill instruction to the external memory.

[0076] In some embodiments, the refill request circuit is further configured to:

[0077] in response to detecting that a new cache miss event triggered by a new refill instruction occurs during the interval when refill instructions are received in batches, send a new refill request for the new refill instruction to the external memory.

[0078] A processor comprising:

[0079] the instruction fetch apparatus as previously described;

[0080] a decoder configured to receive instructions from the instruction fetch apparatus and decode them to obtain a plurality of microinstructions and / or control signals; and

[0081] processing circuitry configured to perform operations according to the plurality of microinstructions and / or control signals.

[0082] An integrated circuit chip comprising the instruction fetch apparatus as previously described or the processor as previously described.

[0083] A board card comprising the integrated circuit chip as previously described.

[0084] An electronic device comprising the integrated circuit chip as previously described.

[0085] A method for fetching instructions comprising:

[0086] detecting whether a cache miss event occurs during fetching of the plurality of instructions from the cache for sending out;

[0087] in response to detecting that the cache miss event occurs, sending a refill request to an external memory;

[0088] receiving refill instructions from the external memory in batches as responses to the refill request; and

[0089] during an interval in which the refill instructions are received in batches, sending out the received refill instructions.

[0090] In some embodiments, sending out instructions comprises:

[0091] queuing the plurality of instructions to be sent in a sending order to form an instruction fetch pipeline; and

[0092] queuing the received refill instructions in a corresponding order for sending out through the instruction fetch pipeline.

[0093] In some embodiments, the method further comprises, upon detecting that the cache miss event occurs, pausing the cache from delivering instructions to be sent out, and receiving the refill instructions from the external memory in batches.

[0094] In some embodiments, during an interval in which the cache receives the refill instructions in batches, the method comprises sending out the received refill instructions, and upon sending out, pausing the cache from receiving a next batch of refill instructions from the external memory.

[0095] In some embodiments, the method further comprises buffering the received refill instructions from the external memory in batches, and upon completing sending out the received refill instructions in batches, filling the received refill instructions in batches into the cache.

[0096] In some embodiments, sending the refill request to the external memory comprises:

[0097] in response to detecting that a plurality of cache miss events triggered by a same refill instruction occur, sending only one refill request for the same refill instruction to the external memory.

[0098] In some embodiments, sending the refill request to the external memory comprises:

[0099] in response to detecting that a new cache miss event triggered by a new refill instruction occurs during an interval in which refill instructions are received in batches, sending a new refill request for the new refill instruction to the external memory.

[0100] The above detailed description of the embodiments of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form described, and many modifications, variations, and alternatives are possible. The concept of the present disclosure has been described with reference to particular embodiments thereof. The specification and drawings should be regarded as illustrative rather than restrictive. It will be apparent to those with skill in the art that many modifications, adaptations, and variations can be made within the scope of the present disclosure and that the scope of the present disclosure is not limited to what has been specifically set forth. Thus, the disclosure is not to be limited by the specific examples provided and the general description above.

[0101] It should be understood that the terms “first,” “second,” “third,” and “fourth” and the like in the claims, specification, and drawings of this disclosure are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. The terms “including” and “comprising” as used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0102] It should also be understood that the terms used in the specification and claims of this disclosure are for the purpose of describing particular embodiments only and are not intended to be limiting. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0103] As used in this specification and claims, the terms “if’ and “when” can each be interpreted to mean “upon determination” or “in response to a determination” or “upon detection” or “in response to a detection” depending on the context. Similarly, the phrase “if determined” or “if detected” can each be interpreted to mean “upon determination” or “in response to a determination” or “upon detection” or “in response to a detection” depending on the context.

[0104] The above detailed description of the embodiments of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form described, and many modifications, variations, and alternatives are possible. The concept of the present disclosure has been described with reference to particular embodiments thereof. The specification and drawings should be regarded as illustrative rather than restrictive. It will be apparent to those with skill in the art that many modifications, adaptations, and variations can be made within the scope of the present disclosure and that the scope of the present disclosure is not limited to what has been specifically set forth. Thus, the disclosure is not to be limited by the specific examples provided and the general description above.

Claims

1. An instruction fetch apparatus, comprising: a buffer configured to buffer a plurality of instructions fetched from an external memory; a fetch circuit configured to fetch a plurality of instructions to be sent out from the buffer; a refill request circuit configured to: detect whether a cache miss event occurs during the instruction fetch apparatus sending out instructions; and send a refill request to the external memory in response to detecting that the cache miss event occurs; the fetch circuit is further configured to: receive a plurality of refill instructions from the external memory in batches as a response to the refill request; and send out the received refill instructions during intervals in which the refill instructions are received in batches.

2. The instruction fetch apparatus of claim 1, wherein the fetch circuit is further configured to: queue the plurality of instructions to be sent out in a sending order to form a fetch pipeline; and queue the received refill instructions in a corresponding order for sending out through the fetch pipeline.

3. The instruction fetch apparatus of claim 1, wherein upon detecting that the cache miss event occurs, the buffer is configured to: suspend passing instructions to be sent out to the fetch circuit; and receive the refill instructions from the external memory in batches.

4. The instruction fetch apparatus of claim 3, wherein the fetch circuit is configured to send out the received refill instructions during intervals in which the buffer receives the refill instructions in batches, and the buffer is configured to receive a next batch of refill instructions from the external memory after the fetch circuit sends out the received refill instructions.

5. The instruction fetch apparatus of claim 1, further comprising a buffer configured to: receive the refill instructions from the external memory in batches; and refill the received refill instructions in batches into the buffer after the fetch circuit finishes sending out the received refill instructions in batches.

6. The instruction fetch apparatus of any one of claims 1-5, wherein the refill request circuit is further configured to: send a refill request for a same refill instruction to the external memory only once in response to detecting that a plurality of cache miss events triggered by the same refill instruction occur.

7. The instruction fetch apparatus of any one of claims 1-5, wherein the refill request circuit is further configured to: send a new refill request for a new refill instruction to the external memory in response to detecting that a new cache miss event triggered by the new refill instruction occurs during intervals in which refill instructions are received in batches.

8. A processor, comprising: the instruction fetch apparatus of any one of claims 1-7; a decoder configured to receive and decode instructions from the instruction fetch apparatus to obtain a plurality of micro-instructions and / or control signals; and a processing circuit configured to perform operations according to the plurality of micro-instructions and / or control signals. ​ 9. An integrated circuit chip comprising the instruction fetch apparatus according to any one of claims 1-7 or the processor according to claim 8.

10. A board card comprising the integrated circuit chip according to claim 9.

11. An electronic device comprising the integrated circuit chip according to claim 9.

12. A method for fetching instructions, comprising: detecting whether a cache miss event occurs during fetching of a plurality of instructions to be sent out from a buffer; sending a fillback request to an external memory in response to detecting that the cache miss event occurs; receiving fillback instructions from the external memory in batches as a response to the fillback request; during the interval of receiving the fillback instructions in batches, sending the received fillback instructions out.

13. The method according to claim 12, wherein sending instructions out comprises: queuing the plurality of instructions to be sent out in a sending order to form a fetch pipeline; and queuing the received fillback instructions in a corresponding order for sending out through the fetch pipeline.

14. The method according to claim 12, further comprising pausing the buffer from delivering instructions to be sent out upon detecting that the cache miss event occurs, and receiving the fillback instructions from the external memory in batches.

15. The method according to claim 14, wherein during the interval of the buffer receiving the fillback instructions in batches, the method comprises sending the received fillback instructions out, and after sending out, pausing the buffer from receiving the next batch of fillback instructions from the external memory.

16. The method according to claim 12, further comprising buffering the received fillback instructions from the external memory in batches, and after completing sending out the received fillback instructions in batches, filling the received fillback instructions in batches into the buffer.

17. The method according to any one of claims 12-16, wherein sending the fillback request to the external memory comprises: sending a fillback request for the same fillback instruction to the external memory only once in response to detecting that multiple cache miss events triggered by the same fillback instruction occur.

18. The method according to any one of claims 12-16, wherein sending the fillback request to the external memory comprises: sending a new fillback request for a new fillback instruction to the external memory in response to detecting that a new cache miss event triggered by the new fillback instruction occurs during the interval of receiving fillback instructions in batches. ​ ​

Images