A bridge, processing unit, and computing system

Abstract

The application discloses a kind of bridge, processing unit and computing system, the bridge includes first circuit board, the first circuit board is connected to first function card by first interface;Second circuit board, the second circuit board is connected to second function card by second interface;Circuit communication mechanism is arranged between the first circuit board and the second circuit board, for the first circuit board and the second circuit board are communicated;Distance adjusting mechanism is arranged between the first circuit board and the second circuit board, for adjusting the distance between the first circuit board and the second circuit board.According to the bridge provided in the application, the first circuit board and the second circuit board are communicated by the circuit communication mechanism, and the distance between the first circuit board and the second circuit board is adjusted by the distance adjusting mechanism, so that the user can change the interface spacing of the bridge according to the demand, and the bridge can be applied to a wider range of scenarios, improving the utilization rate of the bridge.

Description

Technical Field

[0001] The present invention relates to the field of computer hardware, and in particular to a bridge, a processing unit and a computing system. Background Art

[0002] With the continuous advancement of computer technology, the computing demands of artificial intelligence (AI) and high-performance computing (HPC) are constantly increasing. Therefore, there is a growing need for multi-processor systems that support seamless connections between processors (such as GPUs) so that they can work together as a giant accelerator. Due to the limited PCIe bandwidth, this often creates a bottleneck. To build a powerful end-to-end computing platform, a faster and more scalable interconnect is required.

[0003] NVLink is an interconnect and communication protocol developed and launched by NVDIA. NVLink uses a point-to-point architecture and serial transmission for connecting central processing units (CPUs) and graphics processing units (GPUs), and can also be used to connect multiple GPUs. Connecting two GPUs via an NVLink bridge achieves one-way transmission speeds of up to 50GB / s and two-way transmission speeds of up to 100GB / s, far exceeding the current bandwidth of the PCIe bus. This enables GPU performance expansion and meets the demands of high-quality display computing workloads.

[0004] However, the interface spacing of current bridges is usually fixed and cannot be changed (for example, 3 slots, 4 slots, etc.), and users cannot adjust it according to their needs. Therefore, it is necessary to propose a new bridge to solve the above problems. Summary of the Invention

[0005] The Summary of the Invention introduces a series of simplified concepts that will be further described in the Detailed Description of the Invention. The Summary of the Invention is not intended to limit the key features and essential features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0006] The present invention provides a bridge, comprising:

[0007] a first circuit board connected to the first function card via a first interface;

[0008] a second circuit board connected to the second function card via a second interface;

[0009] a circuit connection mechanism, provided between the first circuit board and the second circuit board, for connecting the first circuit board and the second circuit board;

[0010] A distance adjusting mechanism is arranged between the first circuit board and the second circuit board, and is used to adjust the distance between the first circuit board and the second circuit board.

[0011] Further, the first circuit board is provided with at least two connection terminals, and the second circuit board is provided with at least two connection terminals.

[0012] Further, the circuit communication mechanism comprises a third circuit board, and the third circuit board is provided with two connection terminals at two ends thereof.

[0013] Further, the distance between the two connection terminals of the third circuit board is one or more slot spacings.

[0014] Further, the distance adjusting mechanism comprises a first guide column and a second guide column arranged in parallel.

[0015] Further, the distance adjusting mechanism further comprises a base, and the base is provided with a clamping structure.

[0016] Further, the clamping structure comprises a cylinder, and the side surface of the cylinder is in contact with the surface of the first guide column and the second guide column.

[0017] Further, the functional card comprises a display card, a sound card or a network card.

[0018] The application further provides a processing unit comprising the bridge.

[0019] The application further provides a computing system comprising at least one processor and a memory coupled to the at least one processor, and the computing system comprises the bridge.

[0020] According to the bridge provided by the application, the first circuit board and the second circuit board are communicated through the circuit communication mechanism, and the distance between the first circuit board and the second circuit board is adjusted through the distance adjusting mechanism, so that the user can change the interface spacing of the bridge according to the requirement, and the bridge can be applied to a wider range of scenarios, and the utilization rate of the bridge is improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] The following drawings for the present application are hereby incorporated as part of the present application for the purpose of understanding the present application. The embodiments of the present application and their description shown in the drawings are used to explain the principles of the present application.

[0022] In the drawings:

[0023] Figure 1A A structural schematic diagram of a bridge according to one embodiment is shown;

[0024] Figure 1B A top view of a bridge according to one embodiment is shown;

[0025] Figure 1C A bottom view of a bridge according to one embodiment is shown;

[0026] Figure 2A A front view of a bridge in a first mode according to one embodiment is shown;

[0027] Figure 2B A front view of a bridge in a second mode according to one embodiment is shown;

[0028] Figure 3 A parallel processing unit according to one embodiment is shown;

[0029] Figure 4A A general processing cluster within a parallel processing unit according to one embodiment is shown; Figure 3

[0030] A memory partition unit of a parallel processing unit according to one embodiment is shown; Figure 4B Figure 3 A streaming multiprocessor according to one embodiment is shown;

[0031] Figure 5A Figure 4A A processing system according to one embodiment, implemented using a parallel processing unit (PPU);

[0032] Figure 5B A conceptual diagram of a processing system according to one embodiment, implemented using a parallel processing unit (PPU); Figure 3

[0033] Figure 5C An exemplary system that can implement various architectures and / or functions of various previous embodiments is shown. ​​​DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present invention more apparent, exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0035] Example 1

[0036] Currently, the interface spacing of a bridge is usually fixed and cannot be changed (for example, 3 slots, 4 slots, etc.), and users cannot adjust it according to usage requirements.

[0037] To solve the above problems, this embodiment provides a bridge. Figures 1A-2B The bridge includes a first circuit board 110 connected to the first function card, a second circuit board 120 connected to the second function card, and a circuit connection mechanism 130 and a distance adjustment mechanism 140 arranged between the first circuit board 110 and the second circuit board 120, wherein the circuit connection mechanism 130 is used to connect the first circuit board 110 and the second circuit board 120, and the distance adjustment mechanism 140 is used to adjust the distance between the first circuit board 110 and the second circuit board 120.

[0038] Exemplarily, the first circuit board 110 and the second circuit board 120 can be various types of circuit boards, such as a ceramic circuit board, an alumina ceramic circuit board, an aluminum nitride ceramic circuit board, a circuit board, a printed circuit board (PCB), an aluminum substrate, a high-frequency board, a thick copper board, an ultra-thin circuit board, etc., and the present invention is not limited to this.

[0039] Exemplarily, the first circuit board 110 and the second circuit board 120 are two independent circuit boards, and the two can move relative to each other.

[0040] In one embodiment, the first circuit board 110 and the second circuit board 120 are both “L”-shaped, with their long sides arranged in parallel and their short sides arranged opposite to each other.

[0041] Illustratively, the first circuit board 110 is provided with a first interface 113 for connecting to a first function card (not shown), and the second circuit board 120 is provided with a second interface 123 for connecting to a second function card (not shown).

[0042] Exemplarily, the function cards include, but are not limited to, graphics cards, sound cards, and network cards, but this is not limited in the present invention. Function cards are typically inserted into a computer motherboard, also known as a mainboard, system board, logic board, motherboard, or backplane. A motherboard is the center or main circuit board of a complex electronic system, such as a computer. A typical motherboard provides a series of slots for devices such as the processor, graphics card, sound card, network card, hard drive, memory, and external devices to connect to. These devices typically plug directly into the corresponding slots or connect via wires. The most important component on a motherboard is the chipset, which provides a universal platform for connecting different devices and controls communication between them. The chipset also includes support for various expansion slots, such as processor, PCI, ISA, AGP, and PCI Express. The chipset also provides additional functionality for the motherboard, such as an integrated graphics core, integrated sound card, integrated infrared communication technology, Bluetooth, and Wi-Fi.

[0043] In one embodiment, a graphics card is composed of a graphics processor (GPU), video memory, a circuit board, and BIOS firmware. NVLink bridges are used to interconnect multiple GPUs to form a parallel processing unit (PPU), as shown below. Figure 3 Described in detail.

[0044] Exemplarily, at least two connection terminals 111 and 112 are provided on the first circuit board 110 , and at least two connection terminals 121 and 122 are provided on the second circuit board 120 .

[0045] Furthermore, one of the connection terminals 111 , 112 on the first circuit board 110 is connected to one of the connection terminals 121 , 122 on the second circuit board 120 through a circuit connection mechanism 130 .

[0046] Exemplarily, the circuit connection mechanism 130 includes a third circuit board, and connection terminals 131 and 132 are respectively arranged at both ends of the third circuit board, one connection terminal 13 of the third circuit board is connected to one of the connection terminals 111 and 112 on the first circuit board 110, and the other connection terminal 132 of the third circuit board is connected to one of the connection terminals 121 and 122 on the second circuit board 120.

[0047] Furthermore, the distance between the two connection terminals 131, 132 of the third circuit board is one or more slot spacings. In one embodiment, each slot spacing is approximately 20.3 mm. Preferably, the distance between the two connection terminals 111, 112 of the first circuit board is 1 / 2 of the slot spacing, and the distance between the two connection terminals 121, 122 of the second circuit board is 1 / 2 of the slot spacing.

[0048] In one embodiment, the third circuit board connects the short sides of the first circuit board 110 and the second circuit board 120, and the first circuit board 110, the second circuit board 120 and the third circuit board together form a "concave" shape arrangement, as shown in Figures 1A-1C .

[0049] Referring to Figure 2A and 2B , the bridge includes a first mode and a second mode.

[0050] In one embodiment, as shown in Figure 2A , when the connecting terminal 131 of the third circuit board is connected to the connecting terminal 111 of the first circuit board 110, and the connecting terminal 132 of the third circuit board is connected to the connecting terminal 121 of the second circuit board 120, the bridge is in the first mode, i.e. the interface narrow pitch mode.

[0051] In one embodiment, as shown in Figure 2B , when the connecting terminal 131 of the third circuit board is connected to the connecting terminal 112 of the first circuit board 110, and the connecting terminal 132 of the third circuit board is connected to the connecting terminal 122 of the second circuit board 120, the bridge is in the second mode, i.e. the interface wide pitch mode.

[0052] In one embodiment, in the interface narrow pitch mode, the pitch between the first interface 113 and the second interface 123 is 3 slots (3-slot), and in the interface wide pitch mode, the pitch between the first interface 113 and the second interface 123 is 4 slots (4-slot).

[0053] Exemplarily, the distance adjusting mechanism 140 includes a first guide column 141, a second guide column 142, a base 143 and a clamping structure 144. The first end of the first guide column 141 is fixedly connected to the first circuit board 110, and the first end of the second guide column 141 is fixedly connected to the second circuit board 120. Under the action of an external force, the first guide column 141 can move relative to the second guide column 142.

[0054] Further, the first guide column 141 and the second guide column 142 penetrate through the base 143, and the base 143 is provided with the clamping structure 144, so that the first guide column 141 and the second guide column 142 can be prevented from sliding arbitrarily in the base 143.

[0055] In one embodiment, the clamping structure 144 includes a cylinder, which is spring-connected to the base 143. The side surface of the cylinder is in contact with the surface of the first guide column 141 and the second guide column 142. The contact surfaces of the cylinder and the first guide column 141 and the second guide column 142 are provided with anti-skid lines, which include but are not limited to protrusions, wavy lines, gear-like protrusions, etc.

[0056] In one embodiment, when the bridge is subjected to external force to switch from the first mode to the second mode, the first circuit board 110 and the second circuit board 120 are subjected to tension, the first guide column 141 and the second guide column 142 move backwards, the connection terminal 131 of the third circuit board is separated from the connection terminal 111 of the first circuit board 110 and connected to the connection terminal 112 of the first circuit board 110, and the connection terminal 132 of the third circuit board is separated from the connection terminal 121 of the second circuit board 120 and connected to the connection terminal 122 of the second circuit board 120.

[0057] In one embodiment, blocking components are further provided at the ends of the first guide pillar 141 and the second guide pillar 142 to limit the distance of the backward movement of the first guide pillar 141 and the second guide pillar 142 to prevent the first guide pillar 141 and the second guide pillar 142 from sliding out of the base.

[0058] In one embodiment, when the bridge is switched from the second mode to the first mode by external force, the first circuit board 110 and the second circuit board 120 are subjected to pressure, the first guide column 141 and the second guide column 142 move toward each other, the connection terminal 131 of the third circuit board is separated from the connection terminal 112 of the first circuit board 110 and connected to the connection terminal 111 of the first circuit board 110, and the connection terminal 132 of the third circuit board is separated from the connection terminal 122 of the second circuit board 120 and connected to the connection terminal 121 of the second circuit board 120.

[0059] In one embodiment, the distance adjustment mechanism 140 connects the middle parts of the first circuit board 141 and the second circuit board 142 , so that the force is more uniform and the mode switching process is smoother.

[0060] Example 2

[0061] Reference Figure 3 , a parallel processing unit (PPU) 300 is a multi-threaded processor implemented on one or more integrated circuit devices. The PPU 300 is a latency-hiding architecture designed for processing many threads in parallel. A thread (i.e., an execution thread) is an instance of an instruction set configured to be executed by the PPU 300. In one embodiment, the PPU 300 is a graphics processing unit (GPU) that is configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data in order to generate two-dimensional (2D) image data for display on a display device (such as a liquid crystal display (LCD) device). In other embodiments, the PPU 300 can be used to perform general-purpose computations. Although an exemplary parallel processor is provided herein for illustrative purposes, it should be specifically noted that the processor is described for illustrative purposes only and any processor can be used to supplement and / or replace the processor.

[0062] One or more PPUs 300 can be configured to accelerate thousands of high-performance computing (HPC), data center, and machine learning applications. PPUs 300 can be configured to accelerate numerous deep learning systems and applications, including autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analysis, molecular simulations, drug discovery, disease diagnosis, weather forecasting, big data analytics, astronomy, molecular dynamics simulations, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations, among others.

[0063] like Figure 3 As shown, the PPU 300 includes an input / output (I / O) unit 305, a front-end unit 315, a scheduler unit 320, a work distribution unit 325, a hub 330, a crossbar switch (Xbar) 370, one or more general processing clusters (GPCs) 350, and one or more partition units 380. The PPU 300 can be connected to a host processor or other PPUs 300 via one or more high-speed NVLink 310 interconnects. The PPU 300 can be connected to a host processor or other peripheral devices via interconnect 302. The PPU 300 can also be connected to a local memory including multiple memory devices 304. In one embodiment, the local memory can include multiple dynamic random access memory (DRAM) devices. The DRAM devices can be configured as a high-bandwidth memory (HBM) subsystem, in which multiple DRAM dies are stacked within each device.

[0064] The NVLink 310 interconnect enables the system to scale and include one or more PPUs 300 in conjunction with one or more CPUs, supporting cache coherency between the PPU 300 and the CPU, and CPU mastering. Data and / or commands can be sent by the NVLink 310 through the hub 330 to or from other units of the PPU 300, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). Figure 5B NVLink 310 is described in more detail.

[0065] The I / O unit 305 is configured to send and receive communications (e.g., commands, data, etc.) from a host processor (not shown) over the interconnect 302. The I / O unit 305 can communicate with the host processor directly via the interconnect 302, or through one or more intermediate devices such as a memory hub. In one embodiment, the I / O unit 305 can communicate with one or more other processors (e.g., one or more PPUs 300) via the interconnect 302. In one embodiment, the I / O unit 305 implements a Peripheral Component Interconnect Express (PCIe) interface for communications over a PCIe bus, and the interconnect 302 is a PCIe bus. In alternative embodiments, the I / O unit 305 can implement other types of known interfaces for communicating with external devices.

[0066] The I / O unit 305 decodes packets of data received via the interconnect 302. In one embodiment, the packets of data represent commands configured to cause the PPU 300 to perform various operations. The I / O unit 305 sends the decoded commands to various other units of the PPU 300 as specified by the commands. For example, some commands can be sent to the front-end unit 315. Other commands can be sent to the hub 330 or other units of the PPU 300 such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). In other words, the I / O unit 305 is configured to route communications between and among various logical units of the PPU 300.

[0067] In one embodiment, a program executed by the host processor encodes a stream of commands in a buffer that provides a workload for processing by the PPU 300. The workload can include a number of instructions and data to be processed by those instructions. The buffer is a region of memory that is accessible (e.g., read / write) by both the host processor and the PPU 300. For example, the I / O unit 305 can be configured to access the buffer in a system memory connected to the interconnect 302 via memory requests transmitted over the interconnect 302. In one embodiment, the host processor writes the stream of commands to the buffer and then sends the PPU 300 a pointer to a beginning of the stream of commands. The front-end unit 315 receives the pointer(s) to the stream(s) of commands. The front-end unit 315 manages the stream(s), reading commands from the stream(s) and forwarding the commands to various units of the PPU 300.

[0068] Front-end unit 315 is coupled to scheduler unit 320, which configures various GPCs 350 to process tasks defined by one or more streams. Scheduler unit 320 is configured to track state information related to the various tasks managed by scheduler unit 320. The state may indicate which GPC 350 a task is assigned to, whether the task is active or inactive, the priority associated with the task, and the like. Scheduler unit 320 manages the execution of multiple tasks on one or more GPCs 350.

[0069] Scheduler unit 320 is coupled to work distribution unit 325, which is configured to dispatch tasks for execution on GPCs 350. Work distribution unit 325 can track a number of scheduled tasks received from scheduler unit 320. In one embodiment, work distribution unit 325 manages a pending task pool and an active task pool for each GPC 350. The pending task pool can include a number of time slots (e.g., 32 time slots) containing tasks assigned to be processed by a particular GPC 350. The active task pool can include a number of time slots (e.g., 4 time slots) for tasks being actively processed by GPC 350. When a GPC 350 completes execution of a task, the task is evicted from the active task pool of GPC 350, and one of the other tasks from the pending task pool is selected and scheduled for execution on GPC 350. If an active task on GPC 350 has become idle, such as while waiting for a data dependency to be resolved, the active task may be evicted from GPC 350 and returned to the pending task pool, while another task in the pending task pool is selected and scheduled for execution on GPC 350 .

[0070] Work distribution unit 325 communicates with one or more GPCs 350 via XBar (crossbar) 370. XBar 370 is an interconnect network that couples many units of PPU 300 to other units of PPU 300. For example, XBar 370 can be configured to couple work distribution unit 325 to a specific GPC 350. Although not explicitly shown, one or more other units of PPU 300 can also be connected to XBar 370 via hub 330.

[0071] Tasks are managed by the scheduler unit 320 and dispatched to the GPCs 350 by the work distribution unit 325. The GPCs 350 are configured to process tasks and generate results. The results can be consumed by other tasks within the GPC 350, routed to different GPCs 350 via the XBar 370, or stored in the memory 304. The results can be written to the memory 304 via the partition unit 380, which implements a memory interface for reading data from and writing data to the memory 304. The results can be sent to another PPU 304 or CPU via the NVLink 310. In one embodiment, the PPU 300 includes a number U of partition units 380, which is equal to the number of independent and different memory devices 304 coupled to the PPU 300. Figure 4B The partition unit 380 is described in more detail.

[0072] In one embodiment, the host processor executes a driver kernel that implements an application programming interface (API) that enables one or more applications to be executed on the host processor to schedule operations for execution on the PPU 300. In one embodiment, multiple computing applications are executed simultaneously by the PPU 300, and the PPU 300 provides isolation, quality of service (QoS), and independent address spaces for the multiple computing applications. The application can generate instructions (e.g., API calls) that cause the driver kernel to generate one or more tasks to be executed by the PPU 300. The driver kernel outputs the tasks to one or more streams being processed by the PPU 300. Each task can include one or more related groups of threads, referred to herein as warps. In one embodiment, a warp includes 32 related threads that can execute in parallel. Collaborating threads can refer to multiple threads that include instructions to perform tasks and that can exchange data through shared memory. In combination Figure 5A Describes threads and cooperative threads in more detail.

[0073] Figure 4A According to one embodiment, Figure 3 PPU 300 GPC 350. Figure 4A As shown, each GPC 350 includes multiple hardware units for processing tasks. In one embodiment, each GPC 350 includes a pipeline manager 410, a pre-raster operation unit (PROP) 415, a raster engine 425, a work distribution crossbar (WDX) 480, a memory management unit (MMU) 490, and one or more data processing clusters (DPCs) 420. It should be understood that Figure 4A The GPC 350 may include instead Figure 4A Other hardware units or other than the units shown in Figure 4AOther hardware units besides those shown in .

[0074] In one embodiment, the operation of GPC 350 is controlled by pipeline manager 410. Pipeline manager 410 manages the configuration of one or more DPCs 420 for processing tasks assigned to GPC 350. In one embodiment, pipeline manager 410 can configure at least one of one or more DPCs 420 to implement at least a portion of a graphics rendering pipeline. For example, DPC 420 can be configured to execute vertex shading programs on programmable streaming multiprocessors (SMs) 440. Pipeline manager 410 can also be configured to route packets received from work distribution unit 325 to appropriate logic units within GPC 350. For example, some packets can be routed to fixed-function hardware units within PROP 415 and / or raster engine 425, while other packets can be routed to DPC 420 for processing by primitive engine 435 or SM 440. In one embodiment, pipeline manager 410 can configure at least one of one or more DPCs 420 to implement a neural network model and / or a compute pipeline.

[0075] PROP unit 415 is configured to route data generated by raster engine 425 and DPC 420 to the raster operations (ROP) unit, in conjunction with Figure 4B The PROP unit 415 may also be configured to perform optimizations for color blending, organize pixel data, perform address translation, and the like.

[0076] The raster engine 425 includes several fixed-function hardware units configured to perform various raster operations. In one embodiment, the raster engine 425 includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, and a tile aggregation engine. The setup engine receives the transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices. The plane equations are sent to the coarse raster engine to generate coverage information for the primitives (e.g., the x, y coverage mask of the tile). The output of the coarse raster engine is sent to the culling engine, where fragments associated with primitives that fail the z-test are culled, and unculled fragments are sent to the clipping engine, where fragments outside the viewing frustum are clipped. Those fragments remaining after clipping and culling can be passed to the fine raster engine to generate attributes for the pixel fragments based on the plane equations generated by the setup engine. The output of the raster engine 425 includes, for example, fragments to be processed by the fragment shader implemented in the DPC 420.

[0077] Each DPC 420 included in the GPC 350 includes an M pipeline controller (MPC) 430, a primitive engine 435, and one or more SMs 440. The MPC 430 controls the operation of the DPC 420 and routes packets received from the pipeline manager 410 to appropriate units in the DPC 420. For example, packets associated with vertices may be routed to the primitive engine 435, which is configured to fetch vertex attributes associated with the vertices from the memory 304. Conversely, packets associated with shading programs may be sent to the SM 440.

[0078] SM 440 includes a programmable streaming processor configured to process tasks represented by multiple threads. Each SM 440 is multi-threaded and configured to simultaneously execute multiple threads (e.g., 32 threads) from a particular thread group. In one embodiment, SM 440 implements a SIMD (single instruction, multiple data) architecture, in which each thread in a thread group (e.g., a warp) is configured to process a different data set based on the same instruction set. All threads in a thread group execute the same instructions. In another embodiment, SM 440 implements a SIMT (single instruction, multiple thread) architecture, in which each thread in a thread group is configured to process a different data set based on the same instruction set, but in which individual threads in a thread group are allowed to diverge during execution. In one embodiment, a program counter, call stack, and execution state are maintained for each warp, enabling concurrency between warps and serial execution within warps when threads within the warp diverge. In another embodiment, a program counter, call stack, and execution state are maintained for each individual thread, thereby achieving equal concurrency between all threads within and between warps. When execution state is maintained for each individual thread, threads executing the same instruction can be converged and executed in parallel for maximum efficiency. Figure 5A Describe SM440 in more detail.

[0079] MMU 490 provides an interface between GPC 350 and partition unit 380. MMU 490 can provide virtual to physical address translation, memory protection, and arbitration of memory requests. In one embodiment, MMU 490 provides one or more translation lookaside buffers (TLBs) for performing translations from virtual addresses to physical addresses in memory 304.

[0080] Figure 4B According to one embodiment, Figure 3 The memory partition unit 380 of the PPU 300. Figure 4BAs shown, the memory partition unit 380 includes a raster operations (ROP) unit 450, a level 2 (L2) cache 460, and a memory interface 470. The memory interface 470 is coupled to the memory 304. The memory interface 470 can implement a 32-, 64-, 128-, or 1024-bit data bus for high-speed data transfer. In one embodiment, the PPU 300 incorporates U memory interfaces 470, one for each pair of partition units 380, where each pair of partition units 380 is connected to a corresponding memory device 304. For example, the PPU 300 can be connected to up to Y memory devices 304, such as a high-bandwidth memory stack or graphics double data rate version 5 synchronous dynamic random access memory or other types of persistent memory.

[0081] In one embodiment, memory interface 470 implements an HBM2 memory interface, and Y is equal to half of U. In one embodiment, the HBM2 memory stack is located on the same physical package as PPU 300, providing significant power and area savings compared to conventional GDDR5 SDRAM systems. In one embodiment, each HBM2 stack includes four memory dies and Y is equal to 4, where the HBM2 stack includes two 128-bit channels per die, for a total of 8 channels and a data bus width of 1024 bits.

[0082] In one embodiment, memory 304 supports single-error correction, double-error detection (SECDED) error correction code (ECC) to protect data. ECC provides increased reliability for computing applications sensitive to data corruption. Reliability is particularly important in large cluster computing environments where PPU 300 processes very large data sets and / or long-running applications.

[0083] In one embodiment, the PPU 300 implements a multi-level memory hierarchy. In one embodiment, the memory partitioning unit 380 supports unified memory to provide a single, unified virtual address space for the CPU and PPU 300 memory, enabling data sharing between virtual memory systems. In one embodiment, the frequency of PPU 300 accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU 300 where the pages are accessed more frequently. In one embodiment, NVLink 310 supports address translation services that allow the PPU 300 to directly access the CPU's page tables and provide full access to the CPU's memory by the PPU 300.

[0084] In one embodiment, the copy engine transfers data between multiple PPUs 300 or between a PPU 300 and a CPU. The copy engine can generate a page fault for an address that is not mapped to a page table. The memory partition unit 380 can then service the page fault, map the address into a page table, and then the copy engine can perform the transfer. In conventional systems, multiple copy engines operate on fixed memory (e.g., non-pageable) between multiple processors, which significantly reduces the available memory. Due to hardware page faults, addresses can be passed to the copy engine without worrying about whether the memory page is resident, and the copy process is transparent.

[0085] Data from memory 304 or other system memory can be retrieved by memory partition unit 380 and stored in L2 cache 460, which is located on-chip and shared between various GPCs 350. As shown, each memory partition unit 380 includes a portion of L2 cache 460 associated with the corresponding memory device 304. Lower-level caches can then be implemented in multiple units within GPC 350. For example, each SM 440 can implement a level 1 (L1) cache. The L1 cache is a dedicated memory dedicated to a specific SM 440. Data from L2 cache 460 can be retrieved and stored in each L1 cache for processing in the functional units of SM 440. L2 cache 460 is coupled to memory interface 470 and XBar 370.

[0086] The ROP unit 450 performs graphics raster operations related to pixel color such as color compression, pixel blending, etc. The ROP unit 450 also implements depth testing in conjunction with the raster engine 425, receiving the depth of the sample position associated with the pixel fragment from the culling engine of the raster engine 425. The depth of the sample position associated with the fragment is tested against the corresponding depth in the depth buffer. If the fragment passes the depth test for the sample position, the ROP unit 450 updates the depth buffer and sends the result of the depth test to the raster engine 425. It will be understood that the number of partition units 380 can be different than the number of GPCs 350, and therefore each ROP unit 450 can be coupled to each GPC 350. The ROP unit 450 keeps track of packets received from different GPCs 350 and determines to which GPC 350 the results generated by the ROP unit 450 are routed via the Xbar 370. Although in Figure 4B In the embodiment shown, ROP unit 450 is included within memory partition unit 380, but in other embodiments, ROP unit 450 may be external to memory partition unit 380. For example, ROP unit 450 may reside in GPC 350 or another unit.

[0087] Figure 5A According to one embodiment, Figure 4A Streaming multiprocessor 440. Figure 5A As shown, SM 440 includes an instruction cache 505, one or more scheduler units 510, a register file 520, one or more processing cores 550, one or more special function units (SFUs) 552, one or more load / store units (LSUs) 554, an interconnect network 580, and a shared memory / L1 cache 570.

[0088] As described above, the work distribution unit 325 schedules tasks for execution on the GPCs 350 of the PPU 300. Tasks are assigned to specific DPCs 420 within the GPC 350 and, if the task is associated with a shader program, may be assigned to an SM 440. The scheduler unit 510 receives tasks from the work distribution unit 325 and manages the scheduling of instructions for one or more thread blocks assigned to the SM 440. The scheduler unit 510 schedules thread blocks for execution as warps of parallel threads, where each thread block is assigned at least one warp. In one embodiment, each warp executes 32 threads. The scheduler unit 510 can manage multiple different thread blocks, assign warps to different thread blocks, and then dispatch instructions from multiple different cooperative groups to various functional units (i.e., cores 550, SFUs 552, and LSUs 554) during each clock cycle.

[0089] Cooperative Groups is a programming model for organizing groups of communicating threads that allows developers to express the granularity at which threads are communicating, enabling the expression of richer and more efficient decompositions of parallelism. The cooperative launch API supports synchronization between thread blocks to execute parallel algorithms. Conventional programming models provide a single, simple construct for synchronizing cooperating threads: a barrier across all threads of a thread block (e.g., the syncthreads() function). However, programmers often want to define thread groups at a granularity smaller than the thread block granularity and synchronize within the defined group, enabling higher performance, design flexibility, and software reuse in the form of a collective group-wide function interface.

[0090] Cooperative Groups enable programmers to explicitly define thread groups at sub-block (e.g., as small as a single thread) and multi-block granularity and perform collective operations, such as synchronization, on threads in a cooperative group. The programming model supports clean composition across software boundaries so that libraries and utility functions can safely synchronize in their local environment without making assumptions about convergence. Cooperative Group primitives enable new patterns of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across the entire grid of thread blocks.

[0091] The dispatch unit 515 is configured to deliver instructions to one or more functional units. In this embodiment, the scheduler unit 510 includes two dispatch units 515, which enable scheduling of two different instructions from the same warp during each clock cycle. In alternative embodiments, each scheduler unit 510 may include a single dispatch unit 515 or additional dispatch units 515.

[0092] Each SM 440 includes a register file 520 that provides a set of registers for the functional units of the SM 440. In one embodiment, the register file 520 is divided between each functional unit so that each functional unit is allocated a dedicated portion of the register file 520. In another embodiment, the register file 520 is divided between the different warps executed by the SM 440. The register file 520 provides temporary storage for operands connected to the data paths of the functional units.

[0093] Each SM 440 includes L processing cores 550. In one embodiment, the SM 440 includes a large number (e.g., 128, etc.) of different processing cores 550. Each core 550 may include a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit, including a floating-point arithmetic logic unit (FLU) and an integer arithmetic logic unit (ALU). In one embodiment, the FLU implements the IEEE 754-2008 standard for floating-point operations. In one embodiment, the core 550 includes 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0094] Tensor cores are configured to perform matrix operations, and in one embodiment, one or more tensor cores are included in core 550. Specifically, the tensor cores are configured to perform deep learning matrix operations, such as convolution operations for neural network training and inference. In one embodiment, each tensor core operates on a 4×4 matrix and performs a matrix multiplication and accumulation operation D=A×B+C, where A, B, C, and D are 4×4 matrices.

[0095] In one embodiment, the matrix multiplication inputs A and B are 16-bit floating point matrices, while the accumulation matrices C and D can be 16-bit floating point or 32-bit floating point matrices. The tensor cores operate on 16-bit floating point input data as well as 32-bit floating point accumulations. The 16-bit floating point multiplication requires 64 operations to produce a full-precision product, which is then accumulated using 32-bit floating point additions with other intermediate products of the 4×4×4 matrix multiplication. In practice, tensor cores are used to perform larger two-dimensional or higher-dimensional matrix operations built from these smaller elements. APIs (such as the CUDA 9 C++ API) expose specialized matrix load, matrix multiplication and accumulation, and matrix store operations to efficiently use tensor cores from CUDA-C++ programs. At the CUDA level, the warp-level interface assumes that the 16×16 size matrix spans all 32 threads of the warp.

[0096] Each SM 440 also includes M SFUs 552 that perform special functions (e.g., attribute evaluation, reciprocal square root, etc.). In one embodiment, the SFUs 552 may include a tree traversal unit configured to traverse a hierarchical tree data structure. In one embodiment, the SFUs 552 may include a texture unit configured to perform texture map filtering operations. In one embodiment, the texture unit is configured to load a texture map (e.g., a 2D array of texels) from memory 304 and sample the texture map to generate sampled texture values ​​for use in shader programs executed by the SM 440. In one embodiment, the texture map is stored in shared memory / L1 cache 470. The texture unit implements texture operations, such as filtering operations using mip maps (i.e., texture maps at different levels of detail). In one embodiment, each SM 440 includes two texture units.

[0097] Each SM 440 also includes N LSUs 554, which implement load and store operations between the shared memory / L1 cache 570 and the register file 520. Each SM 440 includes an interconnect network 580 that connects each functional unit to the register file 520 and the LSUs 554 to the register file 520 and the shared memory / L1 cache 570. In one embodiment, the interconnect network 580 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 520 and to connect the LSUs 554 to memory locations in the register file and the shared memory / L1 cache 570.

[0098] Shared memory / L1 cache 570 is an on-chip memory array that allows data storage and communication between the SM 440 and the geometry engine 435 and between threads within the SM 440. In one embodiment, shared memory / L1 cache 570 includes 128 KB of storage capacity and is in the path from the SM 440 to the partition unit 380. Shared memory / L1 cache 570 can be used for cache reads and writes. One or more of shared memory / L1 cache 570, L2 cache 460, and memory 304 are backed up by main memory.

[0099] Combining data cache and shared memory functionality into a single memory block provides the best overall performance for both types of memory accesses. This capacity can be used by a program as a cache that does not use shared memory. For example, if the shared memory is configured to use half the capacity, then texture and load / store operations can use the remaining capacity. The integration within shared memory / L1 cache 570 causes shared memory / L1 cache 570 to function as a high-throughput pipeline for streaming data and, at the same time, provide high bandwidth and low latency access of frequently reused data.

[0100] When configured for general-purpose parallel computation, a simpler configuration can be used compared to graphics processing. Specifically, Figure 3 The illustrated fixed function graphics processing units are bypassed, creating a simpler programming model. In a general-purpose parallel computation configuration, the work distribution unit 325 assigns and dispatches thread blocks directly to the DPCs 420. The threads in a block execute the same program, use the unique thread ID in the computation to ensure each thread generates a unique result, use the SM 440 to execute the program and perform the computation, use shared memory / L1 cache 570 to communicate between threads, and use the LSUs 554 to read and write global memory through shared memory / L1 cache 570 and the memory partition unit 380. When configured for general-purpose parallel computation, the SM 440 can also write commands that the scheduler unit 320 can use to launch new work on the DPCs 420.

[0101] The PPU 300 may be included in a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, etc. In one embodiment, the PPU 300 is included on a single semiconductor substrate. In another embodiment, the PPU 300 is included on a system-on-chip (SoC) along with one or more other devices (such as an additional PPU 300, a memory 304, a reduced instruction set computer (RISC) CPU, a memory management unit (MMU), a digital-to-analog converter (DAC), etc.).

[0102] In one embodiment, PPU 300 may be included on a graphics card that includes one or more memory devices 304. The graphics card may be configured to interface with a PCIe slot on a motherboard of a desktop computer. In another embodiment, PPU 300 may be an integrated graphics processing unit (iGPU) or parallel processor included in a chipset of the motherboard.

[0103] Example 3

[0104] Systems with multiple GPUs and CPUs are being used across various industries as developers expose and exploit greater parallelism in applications such as artificial intelligence computing. High-performance GPU-accelerated systems with tens to thousands of computing nodes are deployed in data centers, research institutions, and supercomputers to solve larger problems. As the number of processing devices within high-performance systems increases, communication and data transmission mechanisms need to scale to support this increased bandwidth.

[0105] Figure 5B According to one embodiment, the Figure 3 A conceptual diagram of a processing system 500 implemented by a PPU 300. The exemplary system 500 may be configured to implement Figure 2A The processing system 500 includes a CPU 530, a switch 510, and each of a plurality of PPUs 300 and a corresponding memory 304. The NVLink 310 provides a high-speed communication link between each PPU 300. Figure 5B A specific number of NVLink 310 and interconnect 302 connections are shown, but the number of connections connected to each PPU 300 and CPU 530 can vary. Switch 510 interfaces between interconnect 302 and CPU 530. PPU 300, memory 304, and NVLink 310 can be located on a single semiconductor platform to form a parallel processing module 525. In one embodiment, switch 510 supports two or more protocols that interface between various different connections and / or links.

[0106] In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between each PPU 300 and CPU 530, and switch 510 interfaces between interconnect 302 and each PPU 300. PPU 300, memory 304, and interconnect 302 can be located on a single semiconductor platform to form a parallel processing module 525. In yet another embodiment (not shown), interconnect 302 provides one or more communication links between each PPU 300 and CPU 530, and switch 510 interfaces between each PPU 300 using NVLink 310 to provide one or more high-speed communication links between PPUs 300. In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between PPU 300 and CPU 530 through switch 510. In yet another embodiment (not shown), interconnect 302 provides one or more communication links between each PPU 300 directly. One or more NVLink 310 high-speed communication links can be implemented as physical NVLink interconnects or on-chip or on-die interconnects using the same protocol as NVLink 310.

[0107] In the context of this specification, a single semiconductor platform can refer to a sole unitary semiconductor-based integrated circuit that is fabricated in a single fabrication operation. It should be noted that the term single semiconductor platform can also refer to multi-chip modules with increased connectivity which simulate on-chip operation, and make substantial improvements over utilizing a conventional bus implementation. Of course, the various circuits or devices can alternatively be placed individually or in various combinations of two or more of the circuits or devices.

[0108] In one embodiment, the signaling rate of each NVLink 310 is 20 to 25 gigabits per second, and each PPU 300 includes six NVLink 310 interfaces (as shown in FIG. 5A). Each NVLink 310 provides a data transfer rate of 25 gigabits per second in each direction, with six links providing 300 gigabits per second. When CPU 530 also includes one or more NVLink 310 interfaces, NVLink 310 can be dedicated to PPU-to-PPU communication, as shown in FIG. 5B, or some combination of PPU-to-PPU and PPU-to-CPU. Figure 5B In one embodiment, the signaling rate of each NVLink 310 is 20 to 25 gigabits per second, and each PPU 300 includes six NVLink 310 interfaces (as shown in FIG. 5A). Each NVLink 310 provides a data transfer rate of 25 gigabits per second in each direction, with six links providing 300 gigabits per second. When CPU 530 also includes one or more NVLink 310 interfaces, NVLink 310 can be dedicated to PPU-to-PPU communication, as shown in FIG. 5B, or some combination of PPU-to-PPU and PPU-to-CPU. Figure 5B In one embodiment, the signaling rate of each NVLink 310 is 20 to 25 gigabits per second, and each PPU 300 includes six NVLink 310 interfaces (as shown in FIG. 5A). Each NVLink 310 provides a data transfer rate of 25 gigabits per second in each direction, with six links providing 300 gigabits per second. When CPU 530 also includes one or more NVLink 310 interfaces, NVLink 310 can be dedicated to PPU-to-PPU communication, as shown in FIG. 5B, or some combination of PPU-to-PPU and PPU-to-CPU.

[0109] In one embodiment, NVLink 310 allows direct load / store / atomic access from the CPU 530 to the memory 304 of each PPU 300. In one embodiment, NVLink 310 supports coherency operations, allowing data read from memory 304 to be stored in the cache hierarchy of the CPU 530, reducing cache access latency for the CPU 530. In one embodiment, NVLink 310 includes support for Address Translation Services (ATS), allowing the PPU 300 to directly access page tables within the CPU 530. One or more NVLinks 310 can also be configured to operate in a low-power mode.

[0110] Figure 5C An exemplary system 565 is shown in which various architectures and / or functionalities of various previous embodiments may be implemented. The exemplary system 565 may be configured to implement Figure 2A The method 200 shown in FIG.

[0111] As shown, a system 565 is provided that includes at least one central processing unit 530 connected to a communication bus 575. The communication bus 575 can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or one or more point-to-point communication protocols. The system 565 also includes a main memory 540. Control logic (software) and data are stored in the main memory 540, which can take the form of random access memory (RAM).

[0112] System 565 also includes an input device 560, a parallel processing system 525, and a display device 545, such as a conventional CRT (cathode ray tube), an LCD (liquid crystal display), an LED (light emitting diode), a plasma display, etc. User input can be received from input device 560 (e.g., a keyboard, a mouse, a touchpad, a microphone, etc.). Each of the aforementioned modules and / or devices can even be located on a single semiconductor platform to form system 565. Alternatively, the modules can be located separately or in various combinations of semiconductor platforms, depending on the user's needs.

[0113] Furthermore, system 565 can be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) through network interface 535 for communication purposes.

[0114] System 565 may also include auxiliary storage (not shown). Auxiliary storage 610 includes, for example, a hard drive and / or a removable storage drive, representative of a floppy disk drive, a tape drive, an optical disk drive, a digital versatile disk (DVD) drive, a recording device, or a universal serial bus (USB) flash memory. The removable storage drive reads from and / or writes to a removable storage unit in a well-known manner.

[0115] Computer programs or computer control logic algorithms may be stored in the main memory 540 and / or the secondary storage. These computer programs, when executed, enable the system 565 to perform various functions. The memory 540, storage, and / or any other storage are possible examples of computer-readable media.

[0116] The architecture and / or functionality of the various preceding figures can be implemented in the context of a general-purpose computer system, a circuit board system, a game console system dedicated for entertainment purposes, a dedicated system, and / or any other desired system. For example, system 565 can take the form of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, a mobile telephone device, a television, a workstation, a game console, an embedded system, and / or any other type of logic.

[0117] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely illustrative and are not intended to limit the scope of the present invention. Various changes and modifications may be made therein by those skilled in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as claimed in the appended claims.

[0118] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0119] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this approach to the present invention should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More precisely, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with fewer features than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0120] It will be understood by those skilled in the art that, except where mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus disclosed herein may be combined in any combination. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature providing the same, equivalent, or similar purpose.

[0121] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

[0122] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0123] The foregoing description is merely a specific embodiment of the present invention or an illustration of a specific embodiment. The scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present invention are intended to be encompassed by the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A bridge, characterized in that: include: a first circuit board connected to the first function card via a first interface; a second circuit board connected to the second function card via a second interface; a circuit connection mechanism, provided between the first circuit board and the second circuit board, for connecting the first circuit board and the second circuit board; The distance adjustment mechanism is provided between the first circuit board and the second circuit board, and is used to adjust the distance between the first circuit board and the second circuit board.

2. The bridge according to claim 1, wherein: At least two connection terminals are provided on the first circuit board, and at least two connection terminals are provided on the second circuit board. One of the connection terminals on the first circuit board is connected to one of the connection terminals on the second circuit board through the circuit connection mechanism.

3. The bridge according to claim 2, wherein: The circuit connection mechanism includes a third circuit board, and connection terminals are respectively provided at both ends of the third circuit board. One connection terminal of the third circuit board is connected to one of the connection terminals on the first circuit board, and the other connection terminal of the third circuit board is connected to one of the connection terminals on the second circuit board.

4. The bridge according to claim 3, wherein: The distance between the two connection terminals of the third circuit board is one or more slot spacings.

5. The bridge according to claim 1, wherein: The distance adjustment mechanism includes a first guide post and a second guide post arranged in parallel, one end of the first guide post is fixed relative to the first circuit board, and one end of the second guide post is fixed relative to the second circuit board.

6. The bridge according to claim 5, wherein: The distance adjustment mechanism further includes a base, in which a locking structure is provided. The first guide column and the second guide column pass through the base, and the locking structure prevents the first guide column and the second guide column from sliding arbitrarily in the base.

7. The bridge according to claim 6, wherein: The locking structure includes a cylinder, a side surface of the cylinder contacts the surfaces of the first guide pillar and the second guide pillar, and the contact surfaces of the cylinder, the first guide pillar and the second guide pillar are both provided with anti-slip patterns.

8. The bridge according to claim 1, wherein: The function card includes a graphics card, a sound card or a network card, and the function card is installed in a card slot of the host.

9. A processing unit, characterized in that: The processing unit includes the bridge according to any one of claims 1-8.

10. A computing system comprising at least one processor and a memory coupled to the at least one processor, characterized in that: The computing system comprises the bridge according to any one of claims 1-8.

Images