AI inference offloading scheduling method and device, electronic equipment and readable storage medium

Abstract

This application discloses an AI inference offloading scheduling method, apparatus, electronic device, and readable storage medium, belonging to the field of AI inference technology. It includes: acquiring multiple user requests; when the number of user requests reaches a threshold, determining the key-value cache corresponding to each user request and the priority of each user request; and then offloading the key-value cache corresponding to each user request based on the order of their priorities. Through this method, new user requests can be cached according to a certain prediction window size for use in LLM inference, and their offloading priorities can be analyzed simultaneously, thus avoiding any impact on LLM inference.

Description

Technical Field

[0001] This application belongs to the field of AI inference technology, specifically relating to an AI inference offloading and scheduling method, an AI inference offloading and scheduling device, an electronic device, and a readable storage medium. Background Technology

[0002] Large Language Models (LLMs) have ushered in a new era of generative applications. To ensure the accuracy and logical consistency of LLM responses, most LLMs have a parameter scale exceeding 1 billion, with some even reaching hundreds of billions. When these large-scale models are deployed on resource-constrained edge GPUs or in private scenarios, they can lead to severe problems with insufficient GPU memory resources.

[0003] KV cache offloading technology is currently the main method to alleviate insufficient video memory resources. It can save intermediate results during LLM inference to other storage media such as DRAM in the form of KV cache, and when needed, it can be reloaded from DRAM to GPU to reuse existing calculation results, thereby alleviating the pressure of insufficient video memory.

[0004] However, the current KV cache offloading mechanism has problems. After completing the prefill inference phase, the GPU directly offloads the KV cache. For latency performance reasons, multiple GPUs are usually connected to the CPU's PCIe root port via a PCIe switch or directly. However, the total bandwidth of these ports / switches is limited. When multiple GPUs simultaneously offload their KV cache to DRAM, insufficient PCIe bandwidth can easily occur. This prevents subsequent KV cache entries from being offloaded to DRAM in time, affecting subsequent reuse and ultimately degrading the performance of the entire AI inference system. Summary of the Invention

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide an AI inference offloading and scheduling method, the method comprising: Retrieve multiple user requests; When the number of user requests reaches a certain threshold, determine the key-value cache corresponding to each user request, as well as the priority of each user request; The key-value cache corresponding to each user request is unloaded based on the priority order.

[0006] In one or more embodiments, determining the key-value cache corresponding to each user request, and the priority of each user request, includes: A large language model inference is used to determine the key-value cache corresponding to each user request; Determine the first and second candidate priorities for each user request, and integrate the first and second candidate priorities to obtain the final priority of each user request.

[0007] In one or more embodiments, the step of using a large language model to determine the key-value cache corresponding to each user request includes: Multiple user requests are sent to at least one GPU for inference, generating key-value caches corresponding to each user request.

[0008] In one or more embodiments, determining a first candidate priority for each user request includes: Select the first unprocessed user request with the fewest tokens from among all user requests; Determine whether the first user request and each of the other user requests have the same pre-pass token; If yes, then determine the number M of other user requests and increment the priority of the first user request by M; if no, then perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the first user request with the fewest unprocessed tokens from among the user requests is executed; otherwise, the process ends.

[0009] In one or more embodiments, determining a second candidate priority for each user request includes: Select the second unprocessed user request from among the user requests; Determine whether the target key-value cache exists in the key-value cache database of the dynamic random access memory; the target key-value cache has the same pre-pass token as the second user request; If yes, determine the number N of the target key-value cache and decrement the priority of the second user request by N; otherwise, perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the unprocessed second user request from among the user requests is executed; otherwise, the process ends.

[0010] In one or more embodiments, unloading the key-value cache of the graphics processor corresponding to each user request based on the priority order includes: The key-value caches are sorted according to their priority from highest to lowest to generate a key-value cache unloading queue. The corresponding graphics processor is invoked according to the key-value cache unloading queue so that the corresponding graphics processor can unload the corresponding key-value cache.

[0011] Secondly, embodiments of this application provide an AI inference offloading scheduling device, the device comprising: The acquisition module is used to acquire multiple user requests; The determination module is used to determine the key-value cache corresponding to each user request and the priority of each user request when the number of user requests reaches a certain threshold. The unloading module is used to unload the key-value cache corresponding to each user request based on the priority order.

[0012] In one or more embodiments, the determining module includes: The first determination submodule is used to use large language model reasoning to determine the key-value cache corresponding to each user request; The second determining submodule is used to determine the first candidate priority and the second candidate priority of each user request, and to integrate the first candidate priority and the second candidate priority to obtain the final priority of each user request.

[0013] In one or more embodiments, the first determining submodule is specifically used for: Multiple user requests are sent to at least one GPU for inference, generating key-value caches corresponding to each user request.

[0014] In one or more embodiments, the second determining submodule is specifically used for: Select the first unprocessed user request with the fewest tokens from among all user requests; Determine whether the first user request and each of the other user requests have the same pre-pass token; If yes, then determine the number M of other user requests and increment the priority of the first user request by M; if no, then perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the first user request with the fewest unprocessed tokens from among the user requests is executed; otherwise, the process ends.

[0015] In one or more embodiments, the second determining submodule is further configured to: Select the second unprocessed user request from among the user requests; Determine whether the target key-value cache exists in the key-value cache database of the dynamic random access memory; the target key-value cache has the same pre-pass token as the second user request; If yes, determine the number N of the target key-value cache and decrement the priority of the second user request by N; otherwise, perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the unprocessed second user request from among the user requests is executed; otherwise, the process ends.

[0016] In one or more embodiments, the unloading module is specifically used for: The key-value caches are sorted according to their priority from highest to lowest to generate a key-value cache unloading queue. The corresponding graphics processor is invoked according to the key-value cache unloading queue so that the corresponding graphics processor can unload the corresponding key-value cache.

[0017] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0018] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0019] In this embodiment, the AI ​​inference system can acquire multiple user requests. When the number of user requests reaches a threshold, it determines the key-value cache corresponding to each user request and the priority of each user request. Then, it unloads the key-value cache corresponding to each user request based on the order of priority. In this way, new user requests can be cached according to a certain prediction window size for LLM inference, and their unloading priority can be analyzed simultaneously, thus avoiding any impact on LLM inference.

[0020] Furthermore, LLM inference and prefix matching within the database can be performed on the tokens of each request to determine their offloading priority. Once the LLM inference is completed and the KV cache is output, the tokens can be offloaded according to the obtained priority order, thus avoiding PCIe bandwidth congestion caused by concurrent offloading. Attached Figure Description

[0021] Figure 1This is a flowchart illustrating the steps of an embodiment of the AI ​​inference offloading scheduling method of this application; Figure 2 This is a structural block diagram of an embodiment of an AI inference unloading scheduling device according to this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0024] The AI ​​inference offloading scheduling provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0025] The following explains the relevant definitions involved in the embodiments of this application: KV cache: Key-Value Cache is a mechanism for caching intermediate computation results (keys and values) during AI inference computation. It can be reused to avoid repeated computation for the same user requests, thereby significantly improving inference efficiency; it can also be offloaded from the GPU (Graphics Processing Unit) to DRAM (Dynamic Random Access Memory) to save video memory.

[0026] AI inference systems are hardware and software systems used to deploy and implement inference for large language models. They primarily include GPUs, CPUs (Central Processing Units), system DRAM, SSDs (Solid State Disks or Solid State Drives) storage, and an AI inference framework to enable large language inference capabilities. Generally, it supports mainstream key-value cache offloading and reuse mechanisms.

[0027] Prefill computation, also known as the forward computation process, is one of the computational stages in the AI ​​inference process. This stage involves numerous matrix multiplications and additions, placing extremely high demands on the GPU's computing power. Therefore, the reuse of the KV cache mainly occurs during this stage to alleviate the computational burden on the GPU. If the KV cache is pre-unloaded to DRAM, it needs to be loaded onto the GPU first to complete the prefill computation.

[0028] Reference Figure 1 This document illustrates a flowchart of an embodiment of an AI inference offloading and scheduling method according to this application. The method can be applied to an AI inference system, which may include hardware such as GPUs, CPUs, and DRAMs, and an AI inference framework. It can be used to process user requests and offload the key-value cache. Specifically, the method may include the following steps: Step 101: Obtain multiple user requests.

[0029] Step 102: Determine the key-value cache corresponding to each user request, and the priority of each user request.

[0030] Specifically, the AI ​​inference system can acquire user requests in real time. When the number of acquired user requests reaches a threshold, it can use LLM (Large Language Model) inference to generate a KV cache corresponding to each user request, and determine the priority of each user request through matching.

[0031] When using LLM inference to generate the KV cache corresponding to each user request, each user request can be sent to each GPU for inference. After each GPU performs prefill calculations, it can generate the KV cache corresponding to each request.

[0032] Furthermore, when sending user requests to individual GPUs, each user request can be sent directly to the individual GPU; alternatively, these user requests can be packaged into a batch and then sent to the individual GPUs according to the batch. Of course, the specific sending method can be set according to actual needs, and this application embodiment does not limit it in this way.

[0033] Furthermore, when detecting whether the number of acquired user requests has reached a threshold, it can be determined directly whether the number of user requests has reached the threshold; alternatively, a prediction window of size K (K is a positive integer) can be pre-set. When user requests are acquired in real time, they can be filled into the prediction window. When the prediction window is full, it can be determined that the number of user requests has reached the threshold. Of course, other detection methods can also be used. In practical applications, the specific detection method can be set according to actual needs, and this application embodiment does not impose any limitations on this.

[0034] Furthermore, when determining the priority of any user request, it can be done simultaneously through two dimensions: for any user request, the priority of each user request can be determined by the pre-existing token (denoted as "first candidate priority"), and the second candidate priority of each user request can be determined by database matching (denoted as "second candidate priority").

[0035] Furthermore, when determining the first candidate priority of each user request through the pre-existing token, the user request with the fewest unprocessed tokens (denoted as "first user request") can be selected from all user requests. Then, it is checked whether the first user request has the same pre-existing token as each of the other user requests. Among them, the other user requests are all user requests other than the first user request.

[0036] In this process, matching can be performed between requests within the prediction window or at other locations. In practical applications, the specific matching location can be set according to actual needs, and this application embodiment does not impose any restrictions on this.

[0037] If other user requests with the same pre-set token exist, the number of such requests, M (where M is a positive integer), can be determined. The priority of the first user request is then incremented by M. This ensures that the key-value cache for user requests most likely to be used in the future needs to be unloaded first. For example, if the priority of the first user request is 3, and the number of other user requests with the same pre-set token is determined to be 3, then the first candidate priority for the first user request is 3 + 3 = 6. Then, the step of checking for unprocessed user requests is executed. If no other user requests with the same pre-set token exist, the step of checking for unprocessed user requests can be executed directly.

[0038] Check if there are any unprocessed user requests among the user requests. If so, proceed to the next round of the loop by selecting the first unprocessed user request with the fewest tokens from among the user requests. If not, the process can end.

[0039] Furthermore, when determining the second candidate priority of each user request by using database matching, an unprocessed user request (denoted as "second user request") can be selected from each user request. Then, it is determined whether a target key-value cache exists in the DRAM KVcache database, which has the same pre-token as the second user request.

[0040] If the target key-value cache exists in the DRAM KV cache database, then the number of target key-value caches N (N is a positive integer) can be determined, and the priority of the second user request is reduced by N. This ensures that even if the KV cache is not unloaded to the DRAM KV cache database in time, there are still other shorter KV caches that can be reused. For example, if the priority of the second user request is 3, and the number of target key-value caches is determined to be 4, then the second candidate priority of the second user request is 3 + (-4) = -1. Then, the step of checking whether there are any unprocessed user requests among the user requests is executed. If the target key-value cache does not exist, then the step of checking whether there are any unprocessed user requests among the user requests can be executed directly.

[0041] The process checks if there are any unprocessed user requests among the user requests. If so, it proceeds to the next iteration by selecting a second unprocessed user request from among the user requests. If not, the process ends.

[0042] After obtaining the first and second candidate priorities, they can be integrated to obtain the final priority of each user request. For example, continuing from the previous example, if a user request has a first candidate priority of 6 and a second candidate priority of -1, then the final priority of the user request is 6 + (-1) = 5.

[0043] Step 103: Unload the key-value cache of the graphics processor corresponding to each user request based on the priority order.

[0044] Specifically, after determining the final priority of each user request, the key-value caches can be sorted according to their priority to generate a key-value cache unloading queue. Then, the corresponding GPU is called according to the key-value cache unloading queue so that the corresponding GPU can unload the corresponding KV cache and store the unloaded KV cache into the DRAM KVcache database.

[0045] Furthermore, before each GPU unloads its corresponding KV cache, it can wait for the prefill phase calculation of each GPU to complete.

[0046] In this embodiment, the AI ​​inference system can acquire multiple user requests. When the number of user requests reaches a threshold, it determines the key-value cache corresponding to each user request and the priority of each user request. Then, it unloads the key-value cache corresponding to each user request based on the order of priority. In this way, new user requests can be cached according to a certain prediction window size for LLM inference, and their unloading priority can be analyzed simultaneously, thus avoiding any impact on LLM inference.

[0047] Furthermore, LLM inference and prefix matching within the database can be performed on the tokens of each request to determine their offloading priority. Once the LLM inference is completed and the KV cache is output, the tokens can be offloaded according to the obtained priority order, thus avoiding PCIe bandwidth congestion caused by concurrent offloading.

[0048] It should be noted that the AI ​​inference offloading and scheduling method provided in this application embodiment can be executed by a virtual device, or a control module in the virtual device for executing the method of loading AI inference offloading and scheduling. This application embodiment uses the execution of the method of loading AI inference offloading and scheduling by a virtual device as an example to illustrate the AI ​​inference offloading and scheduling method provided in this application embodiment.

[0049] Reference Figure 2 The diagram illustrates a structural block diagram of an embodiment of an AI inference offloading scheduling device according to this application, which may specifically include the following modules: Module 201 is used to acquire multiple user requests; The determination module 202 is used to determine the key-value cache corresponding to each user request and the priority of each user request when the number of user requests reaches the number threshold. The unloading module 203 is used to unload the key-value cache corresponding to each user request based on the priority order.

[0050] In this embodiment of the application, the determining module includes: The first determination submodule is used to use large language model reasoning to determine the key-value cache corresponding to each user request; The second determining submodule is used to determine the first candidate priority and the second candidate priority of each user request, and to integrate the first candidate priority and the second candidate priority to obtain the final priority of each user request.

[0051] In this embodiment of the application, the first determining submodule is specifically used for: Multiple user requests are sent to at least one GPU for inference, generating key-value caches corresponding to each user request.

[0052] In this embodiment of the application, the second determining submodule is specifically used for: Select the first unprocessed user request with the fewest tokens from among all user requests; Determine whether the first user request and each of the other user requests have the same pre-pass token; If yes, then determine the number M of other user requests and increment the priority of the first user request by M; if no, then perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the first user request with the fewest unprocessed tokens from among the user requests is executed; otherwise, the process ends.

[0053] In this embodiment of the application, the second determining submodule is further configured to: Select the second unprocessed user request from among the user requests; Determine whether the target key-value cache exists in the key-value cache database of the dynamic random access memory; the target key-value cache has the same pre-pass token as the second user request; If yes, determine the number N of the target key-value cache and decrement the priority of the second user request by N; otherwise, perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the unprocessed second user request from among the user requests is executed; otherwise, the process ends.

[0054] In this embodiment of the application, the unloading module is specifically used for: The key-value caches are sorted according to their priority from highest to lowest to generate a key-value cache unloading queue. The corresponding graphics processor is invoked according to the key-value cache unloading queue so that the corresponding graphics processor can unload the corresponding key-value cache.

[0055] In this embodiment, the AI ​​inference system can acquire multiple user requests. When the number of user requests reaches a threshold, it determines the key-value cache corresponding to each user request and the priority of each user request. Then, it unloads the key-value cache corresponding to each user request based on the order of priority. In this way, new user requests can be cached according to a certain prediction window size for LLM inference, and their unloading priority can be analyzed simultaneously, thus avoiding any impact on LLM inference.

[0056] Furthermore, LLM inference and prefix matching within the database can be performed on the tokens of each request to determine their offloading priority. Once the LLM inference is completed and the KV cache is output, the tokens can be offloaded according to the obtained priority order, thus avoiding PCIe bandwidth congestion caused by concurrent offloading.

[0057] The virtual device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. A non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not impose specific limitations.

[0058] This application also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the AI ​​inference offloading scheduling method embodiments described above and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0059] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0060] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described AI inference offloading scheduling method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0061] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0062] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0063] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0064] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An AI inference offloading scheduling method, characterized in that, include: Retrieve multiple user requests; When the number of user requests reaches a certain threshold, determine the key-value cache corresponding to each user request, as well as the priority of each user request; The key-value cache corresponding to each user request is unloaded based on the priority order.

2. The AI ​​inference offloading scheduling method according to claim 1, characterized in that, The determination of the key-value cache corresponding to each user request, and the priority of each user request, includes: A large language model inference is used to determine the key-value cache corresponding to each user request; Determine the first and second candidate priorities for each user request, and integrate the first and second candidate priorities to obtain the final priority of each user request.

3. The AI ​​inference offloading scheduling method according to claim 2, characterized in that, The step of using a large language model to determine the key-value cache corresponding to each user request includes: Multiple user requests are sent to at least one GPU for inference, generating key-value caches corresponding to each user request.

4. The AI ​​inference offloading scheduling method according to claim 2, characterized in that, Determine the first candidate priority for each user request, including: Select the first unprocessed user request with the fewest tokens from among all user requests; Determine whether the first user request and each of the other user requests have the same pre-pass token; If yes, then determine the number M of other user requests and increment the priority of the first user request by M; if no, then perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the first user request with the fewest unprocessed tokens from among the user requests is executed; otherwise, the process ends.

5. The AI ​​inference offloading scheduling method according to claim 2, characterized in that, Determine the second candidate priority for each user request, including: Select the second unprocessed user request from among the user requests; Determine whether the target key-value cache exists in the key-value cache database of the dynamic random access memory; the target key-value cache has the same pre-pass token as the second user request; If yes, determine the number N of the target key-value cache and decrement the priority of the second user request by N; otherwise, perform a check to see if there are any unprocessed user requests among the user requests. Detect whether there are any unprocessed user requests among the various user requests; If yes, then the process of selecting the unprocessed second user request from among the user requests is executed; otherwise, the process ends.

6. The AI ​​inference offloading scheduling method according to claim 1, characterized in that, The unloading of the key-value cache of the graphics processor corresponding to each user request based on the priority order includes: The key-value caches are sorted according to their priority from highest to lowest to generate a key-value cache unloading queue. The corresponding graphics processor is invoked according to the key-value cache unloading queue so that the corresponding graphics processor can unload the corresponding key-value cache.

7. An AI inference offloading scheduling device, characterized in that, include: The acquisition module is used to acquire multiple user requests; The determination module is used to determine the key-value cache corresponding to each user request and the priority of each user request when the number of user requests reaches a certain threshold. The unloading module is used to unload the key-value cache corresponding to each user request based on the priority order.

8. The AI ​​inference offloading scheduling device according to claim 7, characterized in that, The determining module includes: The first determination submodule is used to use large language model reasoning to determine the key-value cache corresponding to each user request; The second determining submodule is used to determine the first candidate priority and the second candidate priority of each user request, and to integrate the first candidate priority and the second candidate priority to obtain the final priority of each user request.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the AI ​​inference offloading scheduling method as described in claims 1-6.

10. A readable storage medium, characterized in that, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, they implement the steps of the AI ​​inference offloading scheduling method as described in claims 1-6.

Images